Nginx – increase nginx active connection more than 65535

connectionsnginxperformance-tuning

currently I'm doing setup for nginx to accept C1M, I do in my Virtual BOX with Centos 7.0 and RAM 10G.

Here my sysctl.conf configuration. 

fs.file-max                    = 9999999
fs.nr_open                     = 9999999

net.core.netdev_max_backlog    = 4096
net.core.rmem_max              = 16777216
net.core.somaxconn             = 3240000
net.core.wmem_max              = 16777216
net.core.netdev_max_backlog    = 16384

net.ipv4.ip_forward            = 0
net.ipv4.ip_local_port_range   = 1      65535
net.ipv4.tcp_rmem              = 8192    873800  8388608
net.ipv4.tcp_wmem              = 4096    655360  8388608
net.ipv4.tcp_mem               = 8388608 8388608 8388608
net.ipv4.tcp_fin_timeout       = 7
net.ipv4.tcp_keepalive_time    = 30
net.ipv4.tcp_max_syn_backlog   = 3240000
net.ipv4.tcp_max_tw_buckets    = 6000000
net.ipv4.tcp_no_metrics_save   = 1
net.ipv4.tcp_syn_retries       = 2
net.ipv4.tcp_tw_recycle        = 1
net.ipv4.tcp_tw_reuse          = 1
net.ipv4.tcp_slow_start_after_idle = 0

vm.min_free_kbytes             = 65536
vm.overcommit_memory           = 2
net.nf_conntrack_max           = 237680
net.netfilter.nf_conntrack_max = 237680

net.ipv4.ip_conntrack_max               = 131072
net.ipv4.netfilter.ip_conntrack_max     = 1048576

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

In nginx.conf, I set worker_connection to 1000000, worker_process 4, worker_rlimit_nofile 1000000

I set my openfile with ulimit -n 2000000, but I only get active connection about 65K, like this one. 

Active connections: 65318 
server accepts handled requests
 768234 768234 1549023 
Reading: 0 Writing: 61245 Waiting: 4073 

I try with wrk -t3 -c67000 -d40s http://my-ip:80

Is there any miss configuration that I make? Thank you for your reply.

Bayu Permadi,

Best Answer

Number of active connections is limited by number of socket's available.

socket is clien_ip + client_port + server_port + server_ip. Maximum port number is about 65k, so the only way to increase number of connections - use additional IP.

For example use virtual interfaces. Or Add virtual adapter to virtualbox machine.

Anyway, for good performance testing you should use distributed clients(load generators), because tests with single or several clients often becomes performance test of client, not server =)

Related Topic