Python – Celery daemon as a Ubuntu service does not consume tasks while running from terminal does

djangopython

On Ubuntu 11.10,

I have to issue python tasks from django using celery.

I'm currently testing on the same machine but eventually the celery worker should run on a remote machine.

django uses the following settings:

BROKER_HOST = "127.0.0.1"
BROKER_PORT = 5672
BROKER_VHOST = "/my_vhost"
BROKER_USER = "celery"
BROKER_PASSWORD = "celery"

I can also see my task queued in
http://localhost:55672/#/queues

the celery daemon uses the following configuration (celeryconfig.py):

BROKER_HOST = "127.0.0.1"
BROKER_PORT = 5672
BROKER_USER = "celery"
BROKER_PASSWORD = "celery"
BROKER_VHOST = "/my_vhost"
CELERY_RESULT_BACKEND = "amqp"
import os
import sys
sys.path.append(os.getcwd())
CELERY_IMPORTS = ("tasks", )

running

celeryd -l info

works well and now I want to run it as a service.

I've followed the instructions from
http://ask.github.com/celery/cookbook/daemonizing.html

and now I'm trying to run it using:

sudo /etc/init.d/celeryd start

But the message is not being consumed, no error in the celery log either.

/etc/default/celeryd

CELERYD_NODES="w1"
CELERYD_CHDIR="/path/to/django/project"
CELERYD_OPTS="--time-limit=300 --concurrency=1"
CELERY_CONFIG_MODULE="celeryconfig"
# %n will be replaced with the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers should run as an unprivileged user.
CELERYD_USER="celery"
CELERYD_GROUP="celery"

I've also created user celery in Ubuntu not sure if its necessary.

Any help will be appreciated,
Thanks,
Guy

Best Answer

We're using supervisor to run Celery on Ubuntu, and it works pretty well, including periodic status checking and restarting of dead processes.