Change path
in your django.wsgi
to /srv/www/django_site
.
It's expecting to be able to import your modules into library code elsewhere, which it can't do without having them in the path.
Enable mod_headers in Apache and then add to your VirtualHost:
RequestHeader add X-Queue-Start "%t"
New Relic will then show you queueing time in the main overview chart.
This is the time between when request is first accepted by Apache child worker process and when mod_wsgi daemon process gets to handle the request. This can be used as one indicator of requests getting backlogged which in turn can indicate thread starvation in daemon process due to deadlocked threads or threads waiting on an external resource.
Unfortunately New Relic relies on requests completing to report data for that request. So if a request gets stuck you will not know about it. If all threads get stuck then daemon process will stop handling more requests.
Problem is that if number of processes/threads across Apache child worker processes is less than 100, the daemon process listener backlog, then all those threads can also get stuck and you will not know from Apache error logs because they will just sit there waiting for daemon to accept connection which never happens. Only the HTTP browser client will know as it will get connection refused when Apache child worker socket backlog fills up.
In mod_wsgi 4.0 I will be adding ability to configure the listener backlog for daemon process so can be reduced so you might get an error of some sort. There are already new options in mod_wsgi 4.0 to look for blocked threads and to restart daemon processes automatically and also dump a stack trace of where blocked threads were in code at the time.
To get that you would need to use mod_wsgi 4.0 dev code from mod_wsgi repo. You could then set blocked-timeout on WSGIDaemonProcess to 60 seconds and when all threads get stuck it will do the restart and recover, plus dump stack traces. Am still tweaking this and there are other configuration options related to this why don't describe here.
The mod_wsgi 4.0 code also has some other features which can be used with custom charting in New Relic to track growing number of blocked threads. Am not happy with it and needs to be changed a bit, but is stable.
Anyway, jump onto the mod_wsgi mailing list to discuss further.
Best Answer
The error messages were misleading. The actual cause of the bug was that the server database was being served by
sqlite
, and the file had gotten the wrong ownership during an update. It was now owned byroot
and wasn't writable by the Django Apache user (www-data
):The Django admin panel updates the database upon user login into the admin panel (for record keeping), the failure to be able to do this results in a 500 error.
To fix this, just change the ownership back to normal: