We have a rabbitmq cluster that receives messages from one application that then get picked up by a second application. We have ran into an issue where we have a large number of 'unacked' messages in our queue. We have three total servers that run the second app and have restarted all of them but the unacked messages are not picked up. We receive an error that looks like an issue with our code (which we are looking into separately).
After doing some research we've realized these messages could have potentially already been sent but not acknowledged properly. We are wondering if there is a way to view the unacked messages similar to ones in the ready queue so we can investigate further.
Best Answer
We found a few things that helped us with this.
First, we created a catchall queue to collect all events and keep them in the ready queue so we could figure out what was going on. It turned out one producer was sending invalid messages. We shut this producer down and observed all new messages looked good and were processing.
Second, we shutdown all consumers (fine in our environment) for a short time. This caused all of the messages to move from unacked to ready. At this point we could view and see they were all invalid. Since they were in the ready queue we were able to flush the queue.