I often temporarily and quickly (ad-hoc) want to tunnel http traffic (and other protocols also) from one server to another using an ssh tunnel. I use for this (at source server):
ssh -f -n -L *:80:target-server.com:80 target-server.com sleep 11555200
Problem is that after a while there is hundreds of connections that I'll see by
lsof -n -i:80
ssh 17076 root 5u IPv6 425353159 TCP <source-serverIP>:http->173.245.48.218:52116 (ESTABLISHED)
ssh 17076 root 8u IPv6 425352908 TCP <source-serverIP>:http->82.177.80.10:49936 (FIN_WAIT2)
ssh 17076 root 9u IPv6 425353148 TCP <source-serverIP>:http->173.245.48.218:31791 (ESTABLISHED)
ssh 17076 root 10u IPv6 425353029 TCP <source-serverIP>:http->80.125.175.214:49837 (FIN_WAIT2)
ssh 17076 root 11u IPv6 425353100 TCP <source-serverIP>:http->90.10.149.220:47749 (FIN_WAIT2)
ssh 17076 root 12u IPv6 425353160 TCP <source-serverIP>:http->79.22.138.109:60744 (ESTABLISHED)
ssh 17076 root 13u IPv6 425353105 TCP <source-serverIP>:http->90.10.149.220:53312 (FIN_WAIT2)
Those connections never seem to timeout. And when number of those reaches ~1024 tunnel does not accept new connections at all. What should I do for these connections timeout like it would without tunnel?
# lsof -n -i:80|grep 'FIN_WAIT2'|wc -l
1016
changing /proc/sys/net/ipv4/tcp_fin_timeout
to a small value does not fix anything.
Also this issue seems to happen only with tunneled http traffic (maybe https too). With tunneling pop3/imap I saw no such issues.
Also I wonder how to change max limit of open tunnel connection to > 1024. I tied
ulimit -n 99999 ssh -f -n -L *:80:target-server.com:80 target-server.com 'ulimit -n 99999 && sleep 11555200'
but it does not fix it. Also setting tcp_max_orphans
to a high value does not affect this.
Best Answer
There are three sets of commands that enable connection keepalives.
TCPKeepAlive
ServerAliveInterval
combined withServerAliveCountMax
ClientAliveInterval
combined withClientAliveCountMax
TCPKeepAlive
keeps a connection alive by sending a keepalive OUTSIDE of the standard ssh encryption, meaning any networking equipment can see that it's just a keepalive. This is spoofable. This method is not recommended.ServerAliveInterval
is initiated by the CLIENT.ServerAliveCountMax
will set a maximum number of keepalives to send, after which a disconnect occurs.ClientAliveInterval
is the same asServerAliveInterval
except it is initiated by the Server.Ensure that all three options are not being invoked. Check
$HOME/.ssh/config
and/etc/ssh/ssh_config
.