Server 2012 R2 ephemeral ports all slowly being bound

netstatnetworkingtcpwindows-server-2012-r2

One of our remote desktop session host servers has started to refuse connection every few days, and upon investigation, it seems it is exhausting available ephemeral ports.

This was identified by event ID 4231, which has error text of:

A request to allocate an ephemeral port number from the global TCP port space has failed due to all such ports being in use

When running netstat -anob, I don't see anything unusual. I then tried running "Log-EphemeralPortsStats.ps1" that I cam across on both a user blog post and I think a MS blog post.

After letting this run for some time, the number of ephemeral ports in use is, on average, around 10.

When I run netstat -aq, I see a list of all ephemeral ports in the range of 49186 – 55645 showing as state – BOUND. If I wait a little while and run it again, then then end port has increased by a few ports.

I don't know if this indicates a kernel-mode port leak, but I suspect this end port number will eventually increase all the way to 65535 and that is the point at which networking will fail.

How can I determine what has these ports bound, as they show:

TCP    0.0.0.0:55643    FLX-RDSH-V001:0    BOUND

Netstat doesn't seem to help me identify the culprit in this case.

Best Answer

TCPView is a very useful tool. After downloading and running, I could see a process using a high ephemeral port number. After a few seconds, the line for that process turned red, then green (I am guessing this indicates the opening of a new port), and then the port number had increased slightly.

Ending this process, unbound all of the ports it had in use, and stopped the exhaustion from happening.