Cisco – VmwareESXi Vlans, Cisco devices and broadcast making the things odd

ciscovlanvmware-esxi

I'm having problem for some days trying to make a correct configuration

Right now I have a Cisco Catalyst switch, a Cisco Router 2811 and a VMware ESXi node with 4-8 VM (Linux,BSD, Windows), each one of those are configure to use 802.1q and they are group in 5 vlans .

Next, the Node connect to a Cisco Catalyst 2964 to the Interface port 0/1 with this configuration:

switchport trunk allowed vlan 1-11
switchport mode trunk
switchport nonegottiate

Plus "Interface vlan" for each vlan, at this point the Virtual hosts with ipv4 static address are able to ping to the Switch (Host of vlan 2 can ping Interface Vlan2 of the Switch , etc).

Next I configure the Interface of the switch which will be connect to a Cisco 2811 Router

switchport trunk allowed  vlan 1-11, 99 
switchport mode trunk 
switchport nonegottiate 
switchport mode trunk native 99

In the Cisco 2811 Router I created one dhcpv4 pool for each Vlan (Except 99) and begin to make the work on the interface 

interface FastEthernet0/0.1
 encapsulation dot1Q 1 
 ip address 192.168.1.1 255.255.255.0
 ipv6 address 2001:DB8:C0CA:1::1/64
!
interface FastEthernet0/0.2
 encapsulation dot1Q 2
 ip address 192.168.2.1 255.255.255.0
 ipv6 address 2001:DB8:C0CA:2::1/64
!
interface FastEthernet0/0.3
 encapsulation dot1Q 3
 ip address 192.168.3.1 255.255.255.0
 ipv6 address 2001:DB8:C0CA:3::1/64
!
interface FastEthernet0/0.4
 encapsulation dot1Q 4
 ip address 192.168.4.1 255.255.255.0
 ipv6 address 2001:DB8:C0CA:4::1/64
!
interface FastEthernet0/0.5
 encapsulation dot1Q 5
 ip address 192.168.5.1 255.255.255.0
 ipv6 address 2001:DB8:C0CA:5::1/64
!
interface FastEthernet0/0.99
encapsulation dot1Q 99 native
ip address 192.168.99.1 255.255.255.0

With this I removed the static IPv4 address and restart the virtual host. To my surprise they have IPv6 address well configured but not IPv4. My next big surprise come when I try to ping the Default Gateway with IPv6 of each host: I can't , but I can ping the remote hosts.

Next, I pick 2 different virtual hosts and put static IPv4 address on them, they can ping their local gateway without problem and even remote nodes, however the DHCP is still failing.

At some point I enable the "debug ip dhcp server packet" I find something very peculiar: All the broadcast messages are getting to the Router Interface fa 0/0.1 ** (and that interface is answering the DHCPOFFER) . With that I remembered another thing: **Before sending a local ping IPv6 check first with NDP using a Multicast message but is no the case when you try to send a remote packet.

At this point, I'm very sure the VMware ESXi has the Vlan well configured, the switch seem to be the same case because the router is able to deliver the "IPv6 Router advertisement" correctly to the virtual host. BUT the router is getting a wrong delivery with Broadcast (and seem) multicast and because that his answer are send in the wrong vlan (or never answer the NDP Neighbour Solicitation).

Suggestion?

— Added 25/03

Something I'm seeing is the next:

The CDP the VMWare ESXi node is getting from the Switch state that the Vlan is 1 and show the ip address range for that vlan. After some test I removed the Vlan1 from the port to the ESXi host and check again the CDP, the new range of IP address now was from the Vlan 2, but the VLAN is still mark as "1" and now the broadcast of the nodes aren't getting to the router.

— Added 25/03 (Second part)

With the suggestion of @JelmerS this is what VMWare display:

 esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         128         4           128               1500    vmnic0

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  VM Network            0        0           vmnic0
  Management Network    0        1           vmnic0

Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch1         128         13          128               1500    vmnic1

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  Vlan5                 5        2           vmnic1
  Vlan4                 4        3           vmnic1
  Vlan2                 2        3           vmnic1
  Vlan3                 3        2           vmnic1

The switch I'm working is the vSwitch1 which has all his nodes separated from the Internet (the other is used for remote connection to the nodes)

Best Answer

I would check your configuration in ESX.

A simple way to tell if your ESX config is broken would be to add vlan 99 onto Gi0/1 (towards your host) and set it as the native vlan (like you did towards your router). This will prevent untagged packets from ending up on a vlan that connects to Fa0/1.1 of your router and confirm that your vSwitch config needs examining.

Related Topic