C Linux Multithreading – Using Locks with Multi Cores/CPUs

It sounds to me like your concurrency issue could be solved by using one queue per producer, where each producer queue refuses to dole out another work item until the previous one has completed. Then you just need each consumer to select fairly from amongst the non-empty producer queues. This should be pretty efficient if all producers generate work items at roughly the same rate and all work items require roughly the same amount of processing by the consumer.

If different producers generate work items at different rates or require different amounts of processing, you may need to change the priority to stop lower rate producers (or those producing higher complexity work units) effectively getting priority over higher rate (or low workload) producers.

One example would be for consumers to select the producer queue with the most items, not just those with no items, unfortunately this could produce other kinds of inefficiencies. How you balance your next work item priorities will thus require tweaking according to the dynamics of the system.

Firstly, nothing is atomic in multi-processing situations. Do not expect to be able to ignore synchronisation just because you're using a relatively small payload. (even a 4-byte payload needs protection).

You might like to look at existing systems for this, I'd recommend ZeroMQ that offers you everything you need for this question.