Layer 2 bridged networking issue with QEmu


I am having a problem with a layer 2 bridged network configuration using Linux, QEmu, and VMware ESX. The configuration works fine on a small isolated network, but once introduced into our large corporate network, problems arise in the network layer.

I am using the server edition of Ubuntu 12.04.2 and QEmu 1.5.2 built from source emulating a SPARC host with qemu-system-sparc. The configuration I am trying to achieve is shown below with IPs shown where IPs are assigned and the last byte of the MAC addresses shown for the different interfaces.

    le0 (on QEmu hosted machine)
    tap0 (on Ubuntu machine)
    br0 (on Ubuntu machine)
    eth1        eth0    (eth1 and eth0 on Ubuntu machine)
    :19         :0f
    |           |
    Corporate Network

The eth0 and eth1 interfaces are both created through VMware, which bridges them through a single physical NIC.

The /etc/network/interfaces file for the Ubuntu machine is shown below.

# The loopback network interface
auto lo
iface lo inet loopback

auto eth1
iface eth1 inet manual

auto tap0
iface tap0 inet manual
    pre-up tunctl -u my-user -t tap0
    up ip link set tap0 up

auto br0
iface br0 inet manual
    bridge_ports tap0 eth1
    bridge_fd 15
    bridge_hello 2
    bridge_maxage 20
    bridge_stp off
    bridge_waitport 60
    bridge_pathcost eth1 32768
    bridge_maxwait 0

# The primary network interface
auto eth0
iface eth0 inet static

In both the isolated network and the corporate network ARP packets make it through the chain of bridges, and has correct ARP information for and vice versa. On the isolated network, the hosts can ping each other, and all seems well with application layer connections across the bridge.

However, on the corporate network, when pinging from the host to the ping packet makes it outward through the bridges to The host then attempts to send the ICMP reply, which never makes it back to the host.

The ARP cache on seems correct, with the IP address of mapping to the :56 MAC address. However, if I run tcpdump on the eth1 interface of the Ubuntu machine I see the ICMP requests coming out from the host, but don't see the replies coming back in (despite the fact that they are leaving when dumped from eth0 on that machine).

I have STP off, though it is running on the corporate network. The output of brctl showstp br0 shows the bridge to be forwarding even when on the corporate network.

Any idea why this isn't working on a larger and more sophisticated network, but does work in isolation? Could our switching gear on the corporate network somehow have corrupt MAC address tables?

Best Answer

The solution to this problem was to enable promiscuous mode on the VMware ESX virtual switch as described here. The physical switch and Ubuntu network interfaces were both setup correctly, but VMware was preventing the incoming traffic from ever making it to the eth1 interface.