NGINX reverse proxy load balance client multi request

nginx

After spending the better part of yesterday trying to figure out why my load balancing and reverse proxy wasnt working in NGINX, it turns out it was and I was running into a different error all together.

Im serving a CPU intensive REST API (Built in Flask) running behind multiple Tornado servers that were spawned by supervisord. All on the same virtual machine. NGINX listens on 80 and redirects to 8080, 8081, 8082….

The intent of all of this was to server lot of requests from a small number of actual clients – API will be consumed by other applications – so Im trying to offload the processing as it comes in from an source to a new worker. One application might make a significant number of calls to the API since they will be serving multiple human users.

Now,
If I send multiple ~simultaneous requests from different machines it functions as it should. Each worker process/cpu goes active and all is well.

If I send multiple ~simultaneous requests from different tabs in the same browser, NGINX serializes them and sends to the same CPU. So if I send 3 requests from 3 tabs (in chrome or firefox) it cycles through the upstream as it should (round robin) but it only fulfills one request at a time.

Is there a way to tell NGINX to treat all requests the same regardless of the source?

Thanks in advance.

worker_processes 4;

## Number of file descriptors
worker_rlimit_nofile 8192;
worker_cpu_affinity 0001 0010 0100 1000;

user www-data;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log;

events {
    worker_connections 4096;
    use epoll;
    multi_accept off;
}

http {
    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 30;
    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

    ##
    # Reverse Proxy Server
    ##
    upstream tornado_flask {
        least_conn;
        server 127.0.0.1:8080 max_fails=1  fail_timeout=3s;
        server 127.0.0.1:8081 max_fails=1  fail_timeout=3s;
        server 127.0.0.1:8082 max_fails=1  fail_timeout=3s;
        server 127.0.0.1:8083 max_fails=1  fail_timeout=3s;
    }

    ##
    # Start Inf API
    ##
    server {
        server_name inf.domain.com;
        listen 192.168.1.70;
        location / {
            proxy_pass http://tornado_flask;
        }
    }
    ##
    # End Inf API
    ##
}

Best Answer

I sorted this. Turns out that the error I was seeing was a result of using the same browser. NGINX doesn't considers multiple tabs in the same browser to be the same connection, so processes serially. If I send a request from two different browsers on the same machine it works fine.

Chalk it up to error in testing methodology.