Ubuntu vm stops responding to host intermittently – but still responding to other network computers

networkingUbuntuvmware-server

I'm running an ubuntu server (bitnami lampstack) vm in vmware-server on a windows host machine. The vm every 10-20 seconds stops responding to the host computer for no apparent reason. I can still access the vm and ping it on other network machines and over the remote console on the host, but it will periodically stop talking to the host in every other way — no ping, no response from the webserver, no dns — even though all are alive and kicking at the time.

Bridged networking, /etc/network/interfaces is pretty straightforward.

/etc/network/interfaces

ifconfig:

ifconfig

(sorry about the images — no SSH at the moment)

When I ping the machine from the windows host, it looks like this:

Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 192.168.1.7: bytes=32 time<1ms TTL=64
[repeat ad infinitum...]

Any ideas?

edit – upon further inspection, this is only happening on the host computer. Another network computer doesn't experience the problem — at least not at the same time, although there is the occasional lone timeout every 30 attempts or so. On the host computer, it normally stops responding like clockwork every 20-second or so. I'm very confused, but I'm starting to think my windows configuration is to blame.

edit2 – tried setting a different ip for the vm, no luck … only the host computer is having trouble.

Host computer talking to the router and other computers – no interruption

Other computers talking to the vm – no problem

Deferred host computer's dns to the router and told the router to connect to the vm's dns – host is getting uninterrupted dns now

Host trying to reach the vm 'directly' (ie. by network ip) – 15-seconds on, 15-seconds off. If it's an ip conflict, I have no idea the source.

Tried assigning the host computer a static network ip address. No difference.

Feels like someone's playing a cruel prank on me or something… anyone have a guess at all as to what I should test at this point?

UPDATE:

I added the host's mac address to the vm's arp cache manually:

  arp -s my.host.ip.address my:host:mac:addess

This fixed the problem ostensibly — but when I reboot the vm it starts flaking again.

I don't understand this at all. The mac address is still there on reboot if I run arp -n, but I have to run that command again to make it respond without interruption. I guess it would help if I knew what the hell I was doing.

Best Answer

This sounds to me like a IP address conflict.
Check if no other computer is also using that IP.

Related Solutions

Does a High Variation in Ping Response Times Indicate a Potential Network Issue

Variance in ping times can be indicative of several things, but the fact that you get the same behaviour regardless of which server in that data centre you ping seems to indicate that there may be an issue with a link or device along the path.

It's worth doing repeated traceroutes to the sever(s) in question, to see if you can identify a particular network device where latency is varying; or alternatively, it could be that there are multiple paths to this location, one of which experiences generally higher latency.

Are you seeing any packet loss when you perform extended pings? If so, this may be indicative of a congested link.

Could you provide a bit more detail about the connectivity between your office and your data centres? Are you using leased lines? Internet VPN? Have you spoken to your company's NOC/network administrators? They may be aware of this issue already.

Windows – Ping “replies” from same computer with ‘Destination host unreachable’ (no route to other computer)

There are plenty of cheap 802.11 APs and client cards out there with broken multicast* support, especially when encryption is enabled, and especially especially when WPA2 Mixed Mode (AES-CCMP simultaneously with TKIP) or 802.11i TSN (AES-CCMP and/or TKIP simultaneously with WEP) is enabled.

*Note: In 802.11, broadcast is a subset of multicast: it's a multicast that goes to everyone. So when I say "multicast", think "multicast/broadcast".

On 802.11, multicast is trickier in the AP -> client direction, because it has to be sent at a lowest-common-denominator multicast rate, it isn't ACKed or resent at the 802.11 layer, and if you have WPA or WPA2 encryption on, it has to be sent encrypted with a different key (the group key), and if you have WPA2 Mixed Mode or 802.11i TSN on, it has to be sent not only with a different key, but with a different cipher (a lowest-common-denominator cipher; TKIP in the case of WPA2 Mixed Mode, and WEP in the case of TSN).

One quick way to see if it's just multicast problem is to manually add static ARP entries on each machine, so they know the "IP address -> wireless MAC address" mappings of each other, and then see if you can ping. ARP uses broadcasts, so broken multicasts on an 802.11 network breaks the ability of a wireless client to receive ARP broadcasts when other clients are trying to look up the wireless client's MAC address. Without the ARP mapping, the ping frame can't be addressed at the 802.11 layer, so it can't be transmitted.

If static ARP mappings fixes it, try temporarily turning off all wireless encryption, then delete your static ARP mappings and try your test again. If ARP works this time, it's a sign that multicast is not completely broken on your 802.11 equipment, it's just broken when encryption is on.

About now, some might be asking, "But if broadcasts are broken, why did DHCP work? DHCP uses broadcasts!", and you're right that DHCP uses broadcasts, but in DHCP only the messages from the client to the server are broadcast. In the other direction, they're usually unicast. And it's only in the to the client direction that multicasts are tricky in 802.11.

Check with your AP and client card vendor for firmware and driver updates. Always buy Wi-Fi certified 802.11 gear, because the Wi-Fi certification testing specifically tests to ensure multicast works even when crypto is on. Please also "name and shame" the vendors of the AP and client card(s) in question.

When multicast is broken between an AP and a client, I can't think of an easy way to tell if the fault is with the AP or the client, other than process-of-elimination, by seeing if other brands of wireless clients have the same problem on that AP, and seeing if that wireless client has the same problem on other brands of APs.

The more advanced way to pin the blame on one end or the other is to use another machine that's not part of the test, but has an 802.11 card, to do an 802.11 monitor mode packet trace, starting from before the wireless client associates. Someone well-versed in 802.11 and WPA[2] key handshaking and the like can probably analyze it and find where to point fingers.

I do most of my networking work on Macs, so I can't walk you through getting 802.11 monitor mode packet traces on other platforms, but just in case you have a Snow Leopard (Mac OS X v10.6) box around, you can do:

sudo /usr/libexec/airportd en1 sniff 1  
[...run your test...]  
^C

...to do an 802.11 monitor mode capture, via en1, which is typically the built-in AirPort card on most Macs, on channel 1. Modify that if your AirPort card is not en1, or if your AP is not on channel 1.

Or you can do:

/System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport -z -c1  
sudo tcpdump -I -y IEEE802_11_radio -s0 -w monitorModeTrace.pcap  
[...run your test...]  
^C

(The -c1 argument to the airport tool puts in the interface on channel 1; modify that to the channel your AP is on.)

Or you can run Wireshark, tshark, etc., but on the Mac you'll still need to use the airport command to set the channel and to force a disassociation.

Best Answer

Related Solutions

Does a High Variation in Ping Response Times Indicate a Potential Network Issue

Windows – Ping “replies” from same computer with ‘Destination host unreachable’ (no route to other computer)

Related Topic