WSGI – Truncated or Oversized Response Headers from Daemon Process

Apache2djangomod-wsgi

System Configuration : Apache2, Django 1.10, Python 3, Ubuntu 16.04 LTS

Django debug=True.

/var/log/apache2/error.log

[52:53.057967] [wsgi:error] [pid 4303] [client 1.1.1.22:24409] Timeout when reading response headers from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py
[52:58.466726] [wsgi:error] [pid 4305] [client 1.1.1.10:9787] Truncated or oversized response headers received from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py
[52:58.466729] [wsgi:error] [pid 4304] [client 1.1.1.4:18417] Truncated or oversized response headers received from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py
[52:58.466726] [wsgi:error] [pid 4307] [client 1.1.1.22:35116] Truncated or oversized response headers received from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py
[52:58.466756] [wsgi:error] [pid 4306] [client 1.1.1.22:19242] Truncated or oversized response headers received from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py
[52:58.467164] [wsgi:error] [pid 4336] [client 1.1.1.4:34187] Truncated or oversized response headers received from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py
[52:58.467212] [wsgi:error] [pid 4342] [client 1.1.1.22:28212] Truncated or oversized response headers received from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py, referer: http://example.org/
[52:58.467282] [wsgi:error] [pid 4331] [client 1.1.1.22:31045] Truncated or oversized response headers received from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py
[52:58.467426] [wsgi:error] [pid 4341] [client 1.1.1.70:22784] Truncated or oversized response headers received from daemon process 'example.org': /home/user/dir/project/main_app/wsgi.py, referer: http://example.org/

I do not know the cause of the error. But I have narrowed it down to the Django wsgi process. Since the server is hosting static files properly.

While Cloudflare sometimes shows 502 : Bad Gateway Error, the server itself shows 500 : Internal Server Error.

I have already tried restarting the server and checking Django's (debug) log file. There is no error information in the Django log files (at all).

How should I debug the problem? Since Django didn't log anything, I assume the problem could be caused in wsgi.

Note : The server was working fine earlier. I made some changes* (which are reverted back as-is); The Django shell works fine.

Changes*

  1. Installed django-pandas, django-model-utils, numpy, scikit-learn
  2. A program that utilises above libraries. (This change is reverted to original)

In other similar questions, the problem is caused during upload of a big file.

Best Answer

The cause of the problem was numpy.

Python C extension modules, like numpy, are known to cause timeouts when used under mod_wsgi.

Source : Answer by Sean F on Timeout when reading response headers from daemon process

A similar question which I didn't find on initial search answered and explained by author of mod_wsgi says -

Some third party packages for Python which use C extension modules, and this includes scipy and numpy, will only work in the Python main interpreter and cannot be used in sub interpreters as mod_wsgi by default uses. The result can be thread deadlock, incorrect behaviour or processes crashes.

Source : Answer by Graham Dumpleton on Non-responsive apache + mod_wsgi after installing scipy

Solution

Add the below line to your httpd.conf. In my case the file was /etc/apache2/apache2.conf.

WSGIApplicationGroup %{GLOBAL}
Related Topic