Linux – Two interfaces on same network with bridge

bridgelinux

I have qemu hypervisor that has 2 physical interfaces. eth0 is used for the host's traffic (management) and eth1 has no IP and is part of br0 along with the VM's tap interfaces. Both physical nics are connected on the same unmanaged switch and everything is on the same network (192.168.1.0/24).

Everything works just fine. But traffic between VM's and the host (for example, the host runs a samba server) goes out of eth0 and back in eth1. Which makes sense. But I would rather have the host forward its traffic directly to br0 if it is destined for a VM. When a VM responds back to the host, traffic seems to be forwarded from br0 to the host directly (without egressing eth1). For this reason, in my ARP table, I get two entries for each VM IP. One that says that the VM can be reached through eth0, and another that says it can be reached through br0.

So my question is this: Is there a way to tell the host that if there are two ARP entries in its cache, for the same device, to prefer one over the other?

Or perhaps my design is very bad to begin with. I like to have one interface reserved for VM traffic (because it is GB interface) and the other for management (because it is a FE interface). I understand that to take advantage of the total throughput of those NIC, I should probably be bonding both NICs but that is not an option for me since the switch they are connected on is unmanaged.

I'd appreciate any suggestions.

Thank you.

Best Answer

I found a way to accomplish what I need. This would only work with openvswitch and not not plain linux bridges (luckily, I'm using OVS)

I now have eth0, eth1 and all tap interfaces in br0. I set the "internal" port of the bridge and eth0 as an access port with vlan 200

ovs-vsctl add-port br0 eth0 tag=200
ovs-vsctl set port br0 tag=200

Then I add eth1(the VM's physical nic through which I want VM traffic to go through) in the bridge but with vlan 200 as well as all tap interfaces

ovs-vsctl add-port br0 eth1 tag=100
ovs-vsctl add-port br0 tap1 tag=100 -- set interface tap1 ofport_request=1001
ovs-vsctl add-port br0 tap2 tag=100 -- set interface tap2 ofport_request=1002

At this point it's like having two bridges since we have 2 vlans. Now I want to be able to get traffic flowing between the host and each VMs without having to go out of eth0 and back in eth1. So for each tap interfaces I add 2 flows:

ovs-ofctl add-flow br0 "table=0,in_port=1001,priority=500,dl_type=0x800,nw_dst=192.168.1.33,actions=output=LOCAL"
ovs-ofctl add-flow br0 "table=0,in_port=LOCAL,priority=500,dl_dst=mac_of_vm,actions=output=1001"
Related Topic