Nginx gunicorn django 502 worker timeout just on some pages

502-errordjangogunicornnginxtimeout

I configured a django app with gunicorn and nginx all was working perfectly until the installation of SSL certifiate on the server. firstly all pages were served perfectly but after some time some pages were showing 502 Bad gateway while others are still working nicely.

I am not trying to upload a big file or to call a page that has a big loading time. the page should be served instantly. I tried everything but cant find the problem.maybe its a configuration error. Please if you can help me

The error was in error.log of gunicorn

[2019-04-20 14:38:24 +0200] [14828] [CRITICAL] WORKER TIMEOUT (pid:21460)
[2019-04-20 12:38:24 +0000] [21460] [INFO] Worker exiting (pid: 21460)
[2019-04-20 14:38:24 +0200] [21500] [INFO] Booting worker with pid: 21500

this is my gunicorn configuration

import multiprocessing

timeout = 120
bind = 'unix:/tmp/gunicorn.sock'
workers = multiprocessing.cpu_count() * 2 + 1
reload = True
daemon = True
accesslog = './access.log'
errorlog = './error.log'

nginx config

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 1024;
        # multi_accept on;
}

http {
        fastcgi_buffers 8 16k;
        fastcgi_buffer_size 32k;
        fastcgi_connect_timeout 300;
        fastcgi_send_timeout 300;
        fastcgi_read_timeout 300;
        ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    # server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}


#mail {
#       # See sample authentication script at:
#       # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
#       # auth_http localhost/auth.php;
#       # pop3_capabilities "TOP" "USER";
#       # imap_capabilities "IMAP4rev1" "UIDPLUS";
#
#       server {
#               listen     localhost:110;
#               protocol   pop3;
#               proxy      on;
#       }
#
#       server {
#               listen     localhost:143;
#               protocol   imap;
#               proxy      on;
#       }
#}

/etc/nginx/sites-available/example

upstream your-gunicorn {
  server unix:/tmp/gunicorn.sock fail_timeout=0;
}

# Catch all requests with an invalid HOST header

server {
    server_name "";
    listen      80;
    return      444;
}

server {
  listen 80;
  server_name example.com www.example.com;
  return 301 https://www.example.com$request_uri;
}

server {
  listen 443 default ssl;
  server_name example.com www.example.com;

  ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

  client_max_body_size 4G;
  keepalive_timeout 70;

  access_log /var/log/nginx/example.access_log;
  error_log /var/log/nginx/example.error_log warn;

  root /var/www/django_projects/example;

  location /static/ {
    autoindex off;
    alias /var/www/django_projects/example/static/;
    expires 1M;
    access_log off;
    add_header Cache-Control "public";
    proxy_ignore_headers "Set-Cookie";
  }

  location @proxy_to_app {
    proxy_set_header Host $host;

    proxy_set_header X-Real-IP $remote_addr;

    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header X-Forwarded-Proto $scheme;

    proxy_pass http://your-gunicorn;

    proxy_read_timeout 90;

    proxy_redirect http://your-gunicorn https://example.com;
  }
   location / {
    try_files $uri @proxy_to_app;
  }

  location /.well-known/acme-challenge/ {
    root /var/www/django_projects/example/static/;
  }

}

nginx error log :

2019/04/21 14:26:28 [error] 7897#7897: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 105.159.137.50, server: example.com, request: "GET /login/ HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock/login/", host: $
$host: "www.example.com"

now instead of 502 BAd Gateway i'm having 504 Gateway Time-out

and there is nothing in my django app error log

Best Answer

I was using django-instagram who scap pictures from an instagram profile and it was the problem.

I removed the django-instagram code and its working like a charm.

But i want to know why scrapping from instagram did that errors, is it because of an nginx configuration ? Because it is working perfectly on my local machine

Related Topic