My team is discussing using RabbitMQ for microservice to microservice communication (among other things). We are also supporting a SAP backend that will be pushing data to us in a method undecided at this time (likely through Boomi or Idocs). We know next to nothing about RabbitMQ, except the general concepts of a message queue and what we've been able to determine from a high level look at their documentation. The general pattern seems to be a consumer is subscribed to one or more queues, and Rabbit pushes to those queues based on rules that are highly configurable, but messages on an individual queue only get read completely "once" before being removed (this is a generalization, as there are a lot of options around confirmations and stuff).
This is a very simple image of a setup where each consumer only cares about distinct topics for them:
Whereas this is a more complex one, where a single topic gets split accross multiple queues. However, when we configure the queues, we need to know where to send the topics, and we still need one queue per consumer of that message
What I am envisioning instead is one huge message pipeline that all messages go through, with individual subscribers reading the messages relevant to them. Something closer to this concept:
My goal for this is to keep everything as loosely coupled as possible. With designs 1 or 2, we have to update the queueing system or exchange every time a new consumer gets added. With the third design, the queueing system is agnostic to what is consuming it, and instead just knows it needs to send messages. The consumers can then just read off the queue, grabbing copies of relevant messages to them.
Does this design make sense? We can't be the first ones to desire a system like this, as it should be a pretty common need (changes occur in a system, and multiple consumers need to act on them).
Best Answer
In the third diagram you are proposing, if Consumer 1 gets to the message first (and ACKS it), Consumers 2 and 3 will never receive it. If that is what you want then fine. If you want each Consumer to receive the message, then you need separate queue for each consumer.
The 3rd diagram is pretty much the scenario when you have multiple servers with the same consumer deployed. If Consumer 1 is deployed on 3 servers you don't care which servers gets to the message first.
To answer your concern, in diagram 1 and 2, your queuing system doesn't care about what queues are created. Or rather, your producer doesn't care because it's the consumer who should be creating and binding queues not the producer. The producer pushes a message with a routing key to an exchange. That's where its job ends. Then when you add a new consumer, it should create the queue and bind to the exchange with the routing key.