Is it Safe to Disable Magento Message Queues (Magento_Amqp)?

magento2magento2.3.2module

Upon checking my server logs I see that in the system.log I have a lot of records with this error: main.INFO: Consumer "async.operations.all" skipped as required connection "amqp" is not configured. Unknown connection name amqp [] []

Could someone please confirm if I can disable this module I am not able to understand what it is even used for. I've read this article from the official Magento docs: Manage message queues
 but still I am not able to understand do I have to leave it enabled or I can disable it. If someone could please explain what this module does would be great as well.

I am using Magento 2.3.2 on a VPS.

Best Answer

I keep bumping in this so I read up a bit more and now feel there is a need for better answers

Solutions that tell you to do cron_run = 'false" will disable other queues you do want, like bulk attribute update or export file generation.

Option 1 disable package

php bin/magento module:disable Magento_WebapiAsync
php bin/magento module:disable Magento_Amqp

They are useful for large site for bulk update and integrations, but for smaller sites totally fine without

Option 2 Tweak your queue cron, dont disable it

  1. List your existing queues

    bin/magento queue:consumers:list

  2. Add all of them (actually, those that make sense, read update below) except async to your cron consumer, in env.php. If there is functionality you never use, skip that one (but remember you did...)

    'cron_consumers_runner' => [
       'cron_run' => true,
       'max_messages' => 2,
       'single_thread' => true,
        'consumers' => [
        'product_action_attribute.update',
        'product_action_attribute.website.update',
        'exportProcessor'
        ]
    ],

Most places have a higher max_messages but if you are not having rabbitmq you are possibly on a low budget host too and short queues more often are better. Up the number as you want.

UPDATE: these queues can end up staying up forever and there are a lot of them so if your server is not very busy and not very meaty that can take a permanent slice of memory off for no valuable use. Also if you don't set 'single thread' you end up with multiple queues waiting for the same thing.

It is useful to REMOVE any of the queues that are not relevant to your usage, put them in a slower cron with very low message numbers or run them manually should you need them once. Eg: I don't use coupon generation so removed codegenerator. I also don't do mass inventory updates often, so just run it manually as bin/magento queue:consumers:start --single-thread --max-messages=1 inventory.mass.update

UPDATE: there is a "process whats there don't wait around" option now, though it cannot be set via command line, or only at installation time. It is called queue/consumers_wait_for_messages and technically should mean your queue runner clears the queue then exists, not waiting around. This will use a bit more CPU but free memory

I have tried to add it to my env.php in two places and it seems to have worked.

'queue' => [
    'consumers_wait_for_messages' => 0,
],
'cron_consumers_runner' => [
    'cron_run' => true,
    'max_messages' => 2,
    'single_thread' => true,
    'consumers-wait-for-messages' => 0,
    'consumers' => [
        'product_action_attribute.update',
        'product_action_attribute.website.update',
        'exportProcessor',
        'inventory.source.items.cleanup',
        'inventory.mass.update',
        'inventory.reservations.cleanup',
        'inventory.reservations.update'
    ]
],

ALSO: queue names have changed and there are some new ones in 5.4

product_action_attribute.update
product_action_attribute.website.update
exportProcessor
inventory.source.items.cleanup
inventory.mass.update
inventory.reservations.cleanup
inventory.reservations.update
media.storage.catalog.image.resize
codegeneratorProcessor  (coupon codes)
inventory.reservations.updateSalabilityStatus
inventory.indexer.sourceItem
inventory.indexer.stock
media.content.synchronization
media.gallery.synchronization
async.operations.all (meant for a real queue system with third party integrations)

** option 3 - out of internal cron and manually run or individually in cron**

The command to run them individually in cron on on command line is

bin/magento queue:consumers:start --single-thread --max-messages=1 inventory.source.items.cleanup

Related Topic