I'm wondering what the best way to restart Celery is without losing any tasks during the restart.
I'm currently running it as a daemon using the init.d/ script provided with celery – everything is working great.
Nonetheless, I will need to restart it (I think?) in order for it to see code updates. My concern is if someone uploads a photo and we want to do processing on it during that 5s or so window that celery is restarting the task will be lost forever and we'll start seeing weird issues.
Is there a suggested way to reload code for celery without losing any transactions in the meantime?
Thanks,
Phil
Best Answer
You can safely restart celery worker without losing any tasks with the TERM signal. So the celery will end the current tasks and die.
If you want your tasks to be retried if something goes wrong you can also use the acks_late option (Task.acks_late / CELERY_ACKS_LATE).
Q: How can I safely shut down the worker? (from: http://celery.readthedocs.org/en/latest/faq.html#how-can-i-safely-shut-down-the-worker)
To stop the worker (from: http://celery.readthedocs.org/en/latest/userguide/workers.html#worker-stopping)
Q: Stopping the worker
To send a TERM signal, use the linux command KILL (from: http://linux.about.com/od/commands/l/blcmdl1_kill.htm)