I have a virtual environment running vSphere 5.1. I have two ESXis with a vDS between them. There is, for the purposes of the discussion, just one portgroup on that vDS configured to use a VLAN. The uplink of each host is made of two teamed NICs, which are both connected to the same physical switch. On another port of the physical switch there is a firewall with one leg in the same VLAN as the portgroup.
I have observed that occasionally, when creating and starting a new VM on host1, the switch will not learn this VM's MAC address (even if I force a gratuitous ARP using arping). When this happens, the VM will not be able to communicate with the firewall, or even with other VMs (that use the same portgroup) on host2. However, it can still communicate with other VMs on host1 that are connected to the same portgroup.
Furthermore, if there is a VM on host1 that is working fine and can communicate externally, and I vMotion it to host2, then the switch will not learn that the MAC address has moved to a different port (again, even after I issue arping from the VM), and as a result this VM will lose connectivity to the firewall.
To add to the confusion, and this is just driving me crazy, if I run a "ping -t " from the firewall, while a VM is starting up, then the switch will learn the VM's MAC address and the VM will be able to communicate.
The "notify switches" option is turned on both at the portgroup and the uplink.
What could I be missing here? I am willing to consider crazy ideas. Tomorrow I will try rebooting the hosts just in case the vDS configuration is out of sync…
Best Answer
My advice to you is to upgrade your ESXi installations. Things like this sometimes fall into the bug category, and according to VMware, your build number (838463) is from October 2012. The current release is build 1312873.
It makes sense to patch if you can, since you're working with a very old build.
Also see: Are VMware ESXi 5 patches cumulative?