Nat – VMWare NAT releasing SYN packets from guest but not admitting SYN/ACK packets to guest

nat;tcpvmware-server

VMWare Server 2.0.2
Host OS is Windows Server 2008 R2 Standard
Guest OS is Windows XP

I am using VMWare VMNet8, the NAT network. The VM guests can't see out to the network around the VM host.

Packet sniffing in the VM host shows that:

  • SYNs are sent to Internet hosts
  • SYN/ACKs are received from them
  • no ACK is sent back to the host
  • eventually the target Internet host times out the incomplete connection and issues RST.

Packet sniffing in the VM guest however shows only that:

  • SYNs are sent to Internet hosts
  • No SYN/ACKs are received
  • No RSTs are received either

so I get connection timeouts.

VMNet8 (and VMNet1) are configured in a completely standard way.

Following a suggestion elsewhere I replaced vmnat.exe in the VM Host with the version from VM Player. No change.

I have another server running an earlier version of VMWare Server, 1.0.6 where exactly the same setup works perfectly. The configuations are identical apart from the NAT subnet numbers. Until I did the packet sniffing I was putting this down to an undiscovered configuration difference or an undiscovered firewall setting in the host but the sniffing makes it clear it's a lower level problem with that, possibly still in VMWare's NAT.

Is this a known problem? With a workaround? Or am I on the wrong track and should be looking elsewhere?

Note: for reasons external to this problem I need to use NAT, not bridging.

Best Answer

It's a little difficult to prove that nobody knows whether or not this is a bug, but I did a quick search of the VMWare forums for you and found the following:

https://communities.vmware.com/message/1854012#1854012

I can see the SYN packet from the VM go over the vmnet8 adapter. This SYN is then sent over my phisical network adapter. I also receive the ACK,SYN back from the external host over the physical network adapter. I do NOT see the ACK,SYN go back over the vmnet8 to the VM.

https://communities.vmware.com/message/1672902#1672902

I see the second SYN sent from the HOST OS to the GUEST to initiate a new connection, and it get the appropriate SYN/ACK followed by an ACK, but the client never sees any of this from its end.

https://communities.vmware.com/message/2030172#2030172

I can ping any website and I get a response not problem. Howerver if I try and view the website with IE it comes back as can not be found. I've tried changing the DNS servers used by the VM's and it doesnt help.

Not sure if these are all a common misconfiguration or a bug. There's plenty of reports and network captures of goofy things happening with VMWare Server 2 and NAT.

I suppose you could try submitting a bug report...