C++ – Memory Management for Fast Message Passing Between Threads

cgarbage-collectionlow-levelperformance

Suppose there are two threads, which communicate by asynchronously sending data messages to each other. Each thread has some kind of message queue.

My question is very low level: What can be expected to be the most efficient way to manage the memory? I can think of several solutions:

  1. Sender creates the object via new. Receiver calls delete.
  2. Memory pooling (to transfer the memory back to the sender)
  3. Garbage collection (e.g., Boehm GC)
  4. (if the objects are small enough) copy by value to avoid heap allocation completely

1) is the most obvious solution, so I'll use it for a prototype. Chances are that it is already good enough. But independent of my specific problem, I wonder which technique is most promising if you are optimizing for performance.

I would expect pooling to be theoretically the best, especially because you can use extra knowledge about the flow of information between the threads. However, I fear that it is also the most difficult to get right. Lots of tuning… 🙁

Garbage collection should be quite easy to add afterwards (after solution 1), and I would expect it to perform very well. So, I guess that it is the most practical solution if 1) turns out to be too inefficient.

If the objects are small and simple, copy by value might be the fastest. However, I fear that it forces unnecessary limitations on the implementation of the supported messages, so I want to avoid it.

Best Answer

If the objects are small and simple, copy by value might be the fastest. However, I fear that it forces unnecessary limitations on the implementation of the supported messages, so I want to avoid it.

If you can anticipate an upper bound char buf[256], e.g. A practical alternative if you cannot which only invokes heap allocations in the rare cases:

struct Message
{
    // Stores the message data.
    char buf[256];

    // Points to 'buf' if it fits, heap otherwise.
    char* data;
};
Related Topic