Nginx – worker_connections are not enough while have idle workers

connectionload-testingnginx

I try to increase number of concurrent connections my server can handle.

$uname -a
Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux

$ulimit -Sn
65536

$ulimit -Hn
65536

Nginx config has the next lines:

worker_processes  4; # or 8
worker_rlimit_nofile 60000;

events {
    worker_connections  1024;
}

Also there is a sleepy (sleeps infinitely) python asyncio HTTP backend used via upstream.

And after a few seconds of load testing:

2016/11/03 05:13:44 [alert] 15#15: 1024 worker_connections are not enough
2016/11/03 05:13:44 [alert] 15#15: 1024 worker_connections are not enough

while 

$netstat -anvpt | wc -l  
1156   # or twice bigger if there is 8 worker processes.

htop shows that there is 4 (or 8) nginx worker processes and only 50% of 1 CPU core used (while 3 other cores are idle). More detailed investigation of the netstat output shows that only 2 worker processes are used and both of them took no more than a half of 1024 allowed connections (as from clients to nginx as from nginx to the upstream).

Additionally, nginx and backend works inside docker containers.

UPD: If I increase worker_connections up to 10240, I'll handle much more clients and the CPU performance will be a bottleneck. But the question is why I faced the message 1024 worker_connections are not enough while still have 2 or 6 idle workers.

Best Answer

Arriving late here but for Googlers, try increasing the worker processes limit (or use automatic):

worker_processes auto;

As per the Nginx documentation, they directly affect each other:

It should be kept in mind that this number includes all connections (e.g. connections with proxied servers, among others), not only connections with clients. Another consideration is that the actual number of simultaneous connections cannot exceed the current limit on the maximum number of open files, which can be changed by worker_rlimit_nofile.

Related Topic