I want to make some software which relies heavily on Plugins. I plan on writing it in C++ and will most likely be getting plugins built in C++, Python and possibly Java But most likely C++ and python but since I don't want to rule out any languages that may be slightly more obscure or uncommon. How do I design a plugin that I can use in my C++ program that is unbound to language (or allows for the broadest compatibility)?
One way that I have thought that this could be done is to offer an API in C++ (as that allows for native code to work) and design a system that will call executable files with parameters being passed as arguments. As long as my program and the Plugin have a specification of what will be passed in and what should happen with the output this should (theoretically) be a valid way of getting maximum compatibility.
The plugins will be responsible for processing data in the form of file(s) and output more files. The actual task that the plugin completes is dependent on the Plugin but the output should always be a file. The plugin will run in parallel to my program with it being called when there is a file to process. There could be multiple instances of a single plugin running at the same time, working of different files. There is no limit to the amount of time that it takes to complete the task (from my programs point of view), but the faster it is done the better. Plugins do not affect the main program directly, only when being given files to work on and collecting results, which is why I think my executable strategy may work for my specific use case.
Best Answer
You cannot do that for exactly the same reasons explained in my answer to your previous question on Writing a language agnostic API? (which is a duplicate of this one).
You need to tell more about your context, your motivation your domain. You are seeking for a holy grail design which does not (and probably cannot) exist.
If universal designs existed, you'll already read about them.
Software design is always a matter of trade-off and compromises for a particular problem in a particular domain.
As I told in my other answer, plugins for GCC (a free software compiler, you can study all its source code) are different in their design and goals than plugins for Firefox (a free software web browser).
BTW both GCC and Clang/LLVM are free software compilers for C++ accepting plugins, and even if these two software are similar in function, their plugin design is very different and incompatible. The zsh and fish shells also accept plugins, and so does Python or Guile or Lua interpreters, or the emacs or vim editors.
There are many languages which have very different views on file IO. Look for example into the IO monad of Haskell (which I don't know well). It is very different of IO in Ocaml or in C++.
I guess that your usage of "plugin" is wrong.
You should try to find a better word, or give your (unconventional) definition of plugins, or at least illustrate that word by existing examples.
A plug-in generally involves some dynamically loaded (using
dlopenon POSIX,Load Libraryon Windows) code (which extends the virtual address space on a process). On Linux, every program written in C++ or C claiming to accept plugins (e.g.gcc,firefox,vim,gedit,vlc,xmms2, Qt and GTK applications, Apache or Lighttpd web servers, ....) is using dlopen(3) (perhaps indirectly) to load the plugin at runtime, and all the loaded plugins are technically shared objects in ELF.(I don't know Windows, but I guess that all plugins loaded by C++ programs on Windows are DLLs, and they are loaded with
LoadLibraryor similar function)You might not even need any plugin, and you could instead consider inter-process communication.
You might be thinking of software pipelines (but they are not plugins). Then think more in terms of communication protocols. Read about named pipes and anonymous pipes. Look (perhaps for inspiration) into the old unix pipelines (you might want to mimic them on Windows, and some C++ frameworks such as POCO or Qt could be helpful). Read about message passing.
A software pipeline (and notably unix pipelines) enable to make various programs written in different programming languages work together. But you do need to specify some communication protocol. Old Unix utilities supposed a line-by-line protocol, but you might define your messages to be something very different (e.g. JSON, see JSON/RPC for inspiration).
You could also make your C++ application extensible by embedding some interpreter (like Lua, Guile, or even Python ...) or some bytecode VM (like Parrot, NekoVM, ...) in it. But you should not speak of plugins in that case! You probably should speak of extensions, not plugins. For an extremely well known -and decades old- example, Emacs is an extensible editor (I'm using it daily since the 1990s). You can improve its behavior by customizing it in extension written in E-Lisp. Another example (less familiar to me): AutoCAD is a proprietary program extensible in AutoLISP; MicroSoft Word is rumored to be extensible thru macros coded in some Visual Basic dialect. A common way to make a software extensible is indeed to embed some interpreter in it. Of course such an approach is not language agnostic, it is tied to the scripting language accepted by that interpreter.
So my concrete recommendation is to embed some good enough interpreter (like Lua or Guile) in your product, and accept the fact that such extensions won't be language agnostic. In practice, if you choose a good enough extension language, it does not matter much. Both Lua and Guile have been designed as extension languages (and so was Tcl, but I don't recommend using it). My personal preference is Guile (because Scheme is a very elegant and powerful language, quite small and easy to learn, read SICP).
If it is only for that (and nothing more ...), you don't need any plugin. You could simply decide that if an output file path name starts with e.g. an exclamation point
!or a vertical bar|your program would pipe to some external command (BTW such conventions are already used in several Unix programs), or have some program argument conventions to behave like that. On POSIX systems, you'll just for example decide to use popen(3) instead of fopen(3), and that external command could be implemented in any language. You then have a primitive form of software pipelines (which you should of course not call a plugin).However, I still believe that extending your program by embedding an interpreter is preferable.