Linux – nginx rejects connections under fairly low load

I have a web service running behind an instance of nginx running as a reverse proxy. Nginx is configured to load balance between 10 app server processes running on two separate hosts.

The problem I'm seeing is that at around 150 concurrent connections nginx starts rejecting all new incoming connections. At 140 connections everything is fast and stable, adding just a few more the server will start rejecting all incoming connections not just those above 150. Once all connections have been broken, it will start accepting again.

This does not seem to change at all as I modify worker_processes, worker_connections or multi_accept settings. When the rejections start, there is very little CPU load (>10%) and plenty of network bandwidth available. There are no messages in the error logs.

What am I doing wrong here?

Here's the config:

worker_processes 8;
worker_rlimit_nofile 65536;

events {
    worker_connections  8192;
    multi_accept on;
    use epoll;
}

http {
    include       /etc/nginx/mime.types;
    access_log  /var/log/nginx/access.log;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;
    tcp_nodelay        on;

    gzip  on;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

In the conf.d directory there are just mappings from host names to backend servers. Like this:

upstream api {
    server 10.0.0.1:8000;
    server 10.0.0.1:8001;
    server 10.0.0.2:8000;
    server 10.0.0.2:8001;
}

server {
    listen 80;
    server_name api.example.com;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_redirect off;
        proxy_pass http://api;
    }
}