RabbitMQ – EPMD Reports: Node ‘rabbit’ Not Running at All

rabbitmq

My RabbitMQ used to run just fine on my Ubuntu 16.04 LTS server, but suddenly it just refuses to start.

The hostname of the server is canvas1.

I installed it using standard apt-get approach:

sudo apt-get update
sudo apt-get install rabbitmq-server

I've tried sudo rm -rf /var/log/rabbitmq/* suggested by this thread and it doesn't help.

I've tried sudo rm -rf /var/lib/rabbitmq/mnesia/* then restart, and it didn't work.

I've also tried adding 127.0.0.0 canvas1 to /etc/hosts, and adding export ERL_EPMD_ADDRESS=127.0.0.1 to /etc/default/rabbitmq-server, but none of them worked.

I've tried uninstall the rabbitmq-server, purge it, and autoremove and autopurge, then reinstall the rabbitmq-server, and it doesn't work, either.

Here is the log shown by sudo service rabbitmq-server status:

● rabbitmq-server.service - RabbitMQ Messaging Server
   Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2019-05-20 16:00:11 CST; 7s ago
  Process: 5830 ExecStop=/usr/sbin/rabbitmqctl stop (code=exited, status=2)
  Process: 5523 ExecStartPost=/usr/lib/rabbitmq/bin/rabbitmq-server-wait (code=exited, status=0/SUCCESS)
  Process: 5522 ExecStart=/usr/sbin/rabbitmq-server (code=exited, status=1/FAILURE)
 Main PID: 5522 (code=exited, status=1/FAILURE)

May 20 16:00:11 canvas1 rabbitmq[5830]:   * epmd reports: node 'rabbit' not running at all
May 20 16:00:11 canvas1 rabbitmq[5830]:                   no other nodes on canvas1
May 20 16:00:11 canvas1 rabbitmq[5830]:   * suggestion: start the node
May 20 16:00:11 canvas1 rabbitmq[5830]: current node details:
May 20 16:00:11 canvas1 rabbitmq[5830]: - node name: 'rabbitmq-cli-5838@canvas1'
May 20 16:00:11 canvas1 rabbitmq[5830]: - home dir: .
May 20 16:00:11 canvas1 rabbitmq[5830]: - cookie hash: KCGs/YQo59BuMnMdMM416A==
May 20 16:00:11 canvas1 systemd[1]: rabbitmq-server.service: Control process exited, code=exited status=2
May 20 16:00:11 canvas1 systemd[1]: rabbitmq-server.service: Unit entered failed state.
May 20 16:00:11 canvas1 systemd[1]: rabbitmq-server.service: Failed with result 'exit-code'.

and the content in /var/log/rabbitmq/startup_log:

BOOT FAILED
===========

Error description:
   noproc

Log files (may contain more information):
   /var/log/rabbitmq/rabbit@canvas1.log
   /var/log/rabbitmq/rabbit@canvas1-sasl.log

Stack trace:
   [{gen,do_for_proc,2,[{file,"gen.erl"},{line,228}]},
    {gen_event,rpc,2,[{file,"gen_event.erl"},{line,239}]},
    {rabbit,ensure_working_log_handlers,0,
            [{file,"src/rabbit.erl"},{line,697}]},
    {rabbit,'-boot/0-fun-0-',0,[{file,"src/rabbit.erl"},{line,328}]},
    {rabbit,start_it,1,[{file,"src/rabbit.erl"},{line,356}]},
    {init,start_em,1,[]},
    {init,do_boot,3,[]}]

=INFO REPORT==== 20-May-2019::16:00:09.809084 ===
Error description:
   noproc

Log files (may contain more information):
   /var/log/rabbitmq/rabbit@canvas1.log
   /var/log/rabbitmq/rabbit@canvas1-sasl.log

Stack trace:
   [{gen,do_for_proc,2,[{file,"gen.erl"},{line,228}]},
    {gen_event,rpc,2,[{file,"gen_event.erl"},{line,239}]},
    {rabbit,ensure_working_log_handlers,0,
            [{file,"src/rabbit.erl"},{line,697}]},
    {rabbit,'-boot/0-fun-0-',0,[{file,"src/rabbit.erl"},{line,328}]},
    {rabbit,start_it,1,[{file,"src/rabbit.erl"},{line,356}]},
    {init,start_em,1,[]},
    {init,do_boot,3,[]}]


{"init terminating in do_boot",noproc}

and the content in /var/log/rabbitmq/startup_err:

init terminating in do_boot (noproc)

Crash dump is being written to: erl_crash.dump...done

and there is no /var/log/rabbitmq/rabbit@canvas1.log. All the files in /var/log/rabbitmq/ is startup_log and startup_err.

There's no dets files in /var/lib/rabbitmq/mnesia either. Only a pid file there, and there's no process with that pid.

The erl_crash.dump can be found here: https://drive.google.com/file/d/1aRSQ-DXkMUZr2domAB-oc6AbeI6-y07c/view?usp=sharing

How can I solve this problem?

Best Answer

It seems that RabbitMQ does not work well with esl-erlang on Ubuntu, I don't know why though.

After completely uninstalling both RabbitMQ and esl-erlang, then reinstall RabbitMQ only, everything works fine again.