Windows – Debugging the VMware virtual network adapter vmxnet3 — packets received by the adapter never make it to that port’s listener

virtualizationwindowswindows-server-2003

Background: I have a virtual instance of Windows Server 2003 running from a VMware environment (although I only have access to the actual guest OS right now). I have a custom .NET Windows service (full access to source) that is listening on a given port.

Problem: Client communication with this service began to degrade recently. Specifically clients' packets randomly stop making it to the listening service. Once a client's traffic fails to make it to the service it will never work again. The odd thing is that with Wireshark I can see every request for the client. But after that it's as though vmxnet3 drops that clients packets. But only that client's packets. Other clients are totally unaffected. Also the service logs every single byte it receives and is a mature server. Previously all servers and clients worked fine, the network adapter worked fine. Nothing has changed in anything or been updated.

Some people have conjectured that packet drops are occurring because the rx buffers are full. This suggestion was made because we were seeing "TCP ZeroWindowProbe" errors in Wireshark. I doubt the rx buffers are full because the network utilization at that time was only 0.1%, and many other clients had their communications completely unaffected.

Question: How can I debug this adapter to more closely inspect what happens to a packet after it hits the virtual adapter? How can I see which packets are dropped and why (whether it's a full rx buffer or not)?

Thanks in advance.

Best Answer

try enabling RSS on the vmxnet3 driver under the adapter properties.