Linux – Which Message Queue should I choose (must run on Linux)

linuxmessage-queuing

There are many open source Message queues for Linux, and I need some help deciding what I should go for.

My problem is simple – I get sent a list of files that needs to be processed. Each job can't be split up, but they are self contained and can be spread to multiple computers.

I'm thinking of solving this using a message queue. Multiple clients send a message to a central queue. Each queue has a number of subscribers that will take jobs from that queue when they have finished processing the current job.

Ideally it should have the following qualities

  • Message queue must be able to store unprocessed messages in case of a shutdown/reboot
  • A job can only be processed by a single subscriber (don't want duplicate jobs)
  • The subscribers should be able to send jobs of their own, that will be processed by a different set of subscribers.

Can anyone suggest a simple to use message queue?

Best Answer

You have RabbitMQ and ZeroMQ, but afaik ZeroMQ doesn't store unprocessed messages in case of a crash. They're both open source and use AMQP, an open messaging protocol.