Windows – tcp source port is not randomly

linux-networkingtcpwindows

When we trying to get a file from a web-server, sometimes access is not possible. After checking the communication with tcpdump, we saw that the Source-machine is randomly choosing a source-port for communication. This should be fine, but the “randomly choosing” is not that random as supposed. So source ports are very often reused after a short time (sometimes only 2 seconds). On destination System the socket is in TIME_WAIT status and so communication is dropped by the destination. The question is, why is the server choosing the source port not randomly enough (like 3388, 3345,2345,3388).

We are able to recreate the exactly same behavior when using wget with the following parameters:

wget -t 1 "xxx/test.html" -O /dev/null -o /dev/null -d --bind-address=yyy.yyy.yyy.yyy

Then the system is choosing the port randomly, but not that random and if port is reused too shortly, then no communication is possible (resending SYN-Packets)

A liitle modification of the wget statement:

wget -t 1 "xxx/test.html" -O /dev/null -o /dev/null -d

and everything is working fine and ports are chosen one by one. Communication never hangs.

So our source system is working exactly like the first wget statement, but we need it to get working like the second one or choosing the ports more randomly. How to change the behavior of choosing-source-ports for TCP-Communications?

Best Answer

I have tested this locally and cannot replicate the behavior. Even when I set my linux machine to use only two local ports I am able to initiate multiple (20+) connections to the same domain (internal or external) without issue. I also ensure that the remote server is in TIME-WAIT state by setting the no-http-keep-alive header (and confirmed via netstat).

I discovered that when using the --bind-address flag for WGET, it uses Bind() to create the socket where the source IP is whatever is specified on the command line and the source port is chosen from /proc/sys/net/ipv4/ip_local_port_range. Whenever a new source port needs to be selected this will be the previous port + 2. After reaching the end of the source port range, it will wrap back around to the start and begin counting up in two's again.

When you omit the --bind-address flag, Connect() is used to establish the socket where the source IP is taken from the default interface and the source port is a random port in the /proc/sys/net/ipv4/ip_local_port_range range. New source ports will also be selected randomly from this range.

In either case, when a socket is reused, the Initial Sequence Number has been increased by the internal TCP clock, allowing servers to accept the new incarnation of the connection. This agrees with my testing of 20 connection requests using the same source port and increasing ISN values - all are accepted by the server in TIME-WAIT state.

If we exclude any weirdness being introduced by tcp_tw_recycle as I do not believe that option is present on a Windows server, the only other thing I can think of is a combination of:

  • a small ephemeral port range configured on your Linux server which results in regular re-use of local ports when using --bind-interface.
  • A firewall or some other network device between the client and server which is performing TCP Sequence Number randomisation. As a result, it cannot be relied upon that the Initial Sequence Numbers hitting the server will be greater than the previous connection and could result in dropped connections.