I have a Linux server with an Erlang application which shows an odd limit of exactly 65536 max tcp incoming connections.
The Erlang application is written with the Cowboy framework.
I have tuned the kernel parameters in this way:
/etc/sysctl.conf:
# Increase system file descriptor limit
fs.file-max = 300000
# Discourage Linux from swapping idle processes to disk (default = 60)
vm.swappiness = 10
# Increase Linux autotuning TCP buffer limits
# Set max to 16MB for 1GE and 32M (33554432) or 54M (56623104) for 10GE
# Don't set tcp_mem itself! Let the kernel scale it based on RAM.
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.optmem_max = 40960
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
# Make room for more TIME_WAIT sockets due to more clients,
# and allow them to be reused if we run out of sockets
# Also increase the max packet backlog
net.core.netdev_max_backlog = 50000
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 10
# Disable TCP slow start on idle connections
net.ipv4.tcp_slow_start_after_idle = 0
# Disable source routing and redirects
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
# Log packets with impossible addresses for security
net.ipv4.conf.all.log_martians = 1
/etc/security/limits.conf:
* soft nofile 300000
* hard nofile 300000
I also checked the maximum number of Erlang processes, but it doesn't seem the limiting factor:
1> erlang:system_info(process_limit).
262144
The test connections are made by 4 client machines from 4 different IP addresses (so it isn't a problem with client limits) and after an initial authentication, the connections are left idle, so server CPU utilization is below 50% and memory utilization is also below 35%.
EDIT: I'm not sure if this is important: the Erlang server application is listening on port 8000 and I get it to listen on port 80 too with this iptables rule:
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000
What else should I look into to remove the 65536 connections limit?
Best Answer
Check how is coded your application. There is a limit of 65535 port, and those in that range: 0-1024 are reserved.
If your application use ephemeral's port; from RFC-6056
In last step ask the vendor if it's a hardcoded value. (an int_16)