Gunicorn not fully reloading after an HUP signal

djangogunicornsupervisordvagrant

I'm trying to get a working supervisor/gunicorn/django setup on vagrant. I am using python-watchdog to try and restart gunicorn when code changes happen.

I am using the following supervisor configuration for gunicorn:

[program:someapp]
environment=PYTHONPATH='/vagrant/libs/unmodified/django-error-capture-middleware/src:/vagrant:/home/vagrant/.virtualenvs/someapp/lib/python2.7/site-packages:/vagrant/wsgi',DJANGO_SETTINGS_MODULE=someapp.settings.vagrant
command=/home/vagrant/.virtualenvs/someapp/bin/gunicorn --bind 0.0.0.0:80 --pid=/home/vagrant/.gunicorn.pid --preload --workers=1 --debug --log-level debug --error-logfile - --access-logfile - vagrant_wsgi:application
user=root
group=root
redirect_stderr=true
stdout_logfile = /vagrant/logs/gunicorn.log
stderr_logfile = /vagrant/logs/gunicorn.log
stdout_logfile_maxbytes=0
autostart=true
autorestart=true
stdout_events_enabled=true
loglevel=debug

This all works just fine. The watchdog also works fine. However, when I use watchdog to run a kill -HUP [pidofgunicorn], sometimes it won't actually reload fully. Sometimes django will even report that modules are missing when they were previously there before (I haven't modified sys.path at all).

If I use watchdog to run a supervisorctl restart someapp, it works fine. However, it takes a lot longer, especially on the virtualbox instance.

Is there something I can do to get gunicorn to reload gracefully, and actually see the changes that were made?

Best Answer

The problem is the --preload parameter.

The first solution is not to use the --preload.

The second solution is to follow this:

# Reload a new master with new workers
kill -s USR2 $PID
# Graceful stop old workers
kill -s WINCH $OLDPID
# Graceful stop old master
kill -s QUIT $OLDPID

The third solution is to use the package https://github.com/flupke/rainbow-saddle that works well with supervisord.