C++ – How to design a C++ program to allow for runtime import of functions

creflectionruntime

today, I like to ask you a question towards the capabilities of C++ to realize a specific software architecture.

Of course, I have used the search but have not found any directly linked answer.

Basically, my goal is to build a program which allows the user for modeling and simulation of arbitrarily composed physical systems, e.g. a driving car. I assume to have a library of physical models (functions within classes). Each function may have some inputs and return some outputs depending on the underlying physical description, e.g. a combustion engine model, an aerodynamic drag model, a wheel model, etc.

Now, the idea is to provide the user a framework which allows him to compose any functions according to his needs, i.e. to map any physical behavior. The framework should provide functionalities to connect the outputs and inputs of different functions. Therefore, the framework provides a container class. I call it COMPONENT, which is able to hold one or many model objects (FUNCTION). These containers can also hold other components (cf. composite pattern) as well as the connections (CONNECTOR) between the function parameters. Additionally, the component class provides some general numeric functionalities such as math solver and so on.

The composition of functions should be done during runtime. In the first instance, the user should be able to set up a composition through importing an XML which defines the composition structure. Later, one could think of adding a GUI.

To give you a better understanding here is a very simplified example:

<COMPONENT name="Main">
  <COMPONENT name="A">
    <FUNCTION name="A1" path="lib/functionA1" />
  </COMPONENT>
  <COMPONENT name="B">
    <FUNCTION name="B1" path="lib/functionB1" />
    <FUNCTION name="B2" path="lib/functionB2" />
  </COMPONENT>
  <CONNECTIONS>
    <CONNECTOR source="A1" target="B1" />
    <CONNECTOR source="B1" target="B2" />
  </CONNECTIONS>        
</COMPONENT>

It is not necessary to dive deeper into the framework's capabilities because my problem is much more general. When the framework code/program is compiled, the physical problem description, as well as the user-defined functions, are not known. When the user selects (via XML or later via a GUI) a function, the framework should read the function information, i.e. should get the information of input and output parameters, in order to offer the user the option to interconnect the functions.

I know the principles of reflection and I am aware that C++ does not provide this feature. However, I am sure that the concept of "building objects during runtime" is very often required. How should I set up my software architecture in C++ to achieve my goal? Is C++ the right language? What do I overlook?

Thanks in advance!

Cheers,
Oliver

Best Answer

In pure standard C++, you cannot "allow for runtime import of functions"; according to the standard, the set of C++ functions is statically known at build-time (in practice, link-time) since fixed from the union of all translation units composing your program.

In practice, most of the time (excluding embedded systems) your C++ program runs above some operating system. Read Operating Systems: Three Easy Pieces for a good overview.

Several modern operating systems allow dynamic loading of plugins. POSIX notably specifies dlopen & dlsym. Windows has something different LoadLibrary (and an inferior linking model; you need to explicitly annotate the functions concerned, provided, or used by plugins). BTW on Linux you can practically dlopen a big lot of plugins (see my manydl.c program, with enough patience it can generate then load nearly a million of plugins). So your XML thing could drive the loading of plugins. Your multi-component / multi-connector description reminds me of Qt signals and slots (which requires a moc preprocessor; you might need something like that too).

Most C++ implementations use name mangling. Because of it, you'll better declare as extern "C" the functions related to plugins (and defined in them, and accessed by dlsym from the main program). Read the C++ dlopen mini HowTo (for Linux at least).

BTW, Qt and POCO are C++ frameworks providing some portable and higher-level approach to plugins. And libffi enables you to call functions whose signature is only known at runtime.

Another possibility is to embed some interpreter, like Lua or Guile, in your program (or write your own one, like Emacs did). This is a strong architectural design decision. You may want to read Lisp In Small Pieces and Programming Language Pragmatics for more.

There are variants or mixes of those approaches. You could use some JIT compilation library (like libgccjit or asmjit). You could generate at runtime some C and C++ code in a temporary file, compile it as a temporary plugin, and dynamically load that plugin (I used such an approach in GCC MELT).

In all these approaches, memory management is a significant concern (it is a "whole program" property, and what actually is the "envelope" of your program is "changing"). You'll need at least some culture about garbage collection. Read the GC handbook for the terminology. In many cases (arbitrary circular references where weak pointers are not predictable), the reference counting scheme dear to C++ smart pointers might not be enough. See also this.

Read also about dynamic software updating.

Notice that some programming languages, notably Common Lisp (and Smalltalk), are more friendly to the idea of runtime importing functions. SBCL is a free software implementation of Common Lisp, and compiles to machine code at every REPL interaction (and is even able to garbage collect machine code, and can save an entire core image file which can be later easily restarted).

Related Topic