Linux – KVM with public IPs using bridge – Guest has no connectivity

Preface Notes:

• The IP addresses shown are in RFC1918 space, but in reality are public IPs, I just altered the first octet
• Connectivity between the host and the internet is working fine
• The host is running Debian Jessie, uname -a below

Linux titan 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux

I have a server which has several addresses allocated within a /27. These details are:

Network: 10.9.114.64
Gateway: 10.9.114.65
Host:    10.9.114.80
Guest:   10.9.114.81
Bcast:   10.9.114.85

There are more IPs for further guests, but for now i'm just working with a single guest IP.

My plan is to bridge eth0 and the virtual network adapter, so that the guest can communicate directly with the host and the gateway.

On the host, I moved the addressing to a bridge interface, br0, shown below:

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
 address   10.9.114.80
 netmask   255.255.255.224
 gateway   10.9.114.65
 bridge_ports eth0
 bridge_stp off
 bridge_fd 0
 bridge_maxwait 0

This was verified with ifconfig:

br0       Link encap:Ethernet  HWaddr 54:04:a6:7e:e5:e2
          inet addr:10.9.114.80  Bcast:10.9.114.95  Mask:255.255.255.224
          inet6 addr: 2a01:4f8:151:6456::/64 Scope:Global
          inet6 addr: fe80::5604:a6ff:fe7e:e5e2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4349285 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3342075 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:631623791 (602.3 MiB)  TX bytes:4483323274 (4.1 GiB)

eth0      Link encap:Ethernet  HWaddr 54:04:a6:7e:e5:e2
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5130227 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5612825 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:763224208 (727.8 MiB)  TX bytes:4676628982 (4.3 GiB)

I then setup a debian VM using virt-install. It was originally set up to use NAT (as I can't get bridged networking working yet), but the current interface configuration is shown below:

(Taken from "virsh dumpxml jessie-amd64")

<interface type='bridge'>
<mac address='52:54:00:06:f1:d6'/>
<source bridge='br0'/>
<target dev='vnet0'/>
<model type='rtl8139'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</interface>


(Taken from "virsh edit jessie-amd64")

<interface type='bridge'>
<mac address='52:54:00:06:f1:d6'/>
<source bridge='br0'/>
<model type='rtl8139'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</interface>


After applying the bridge configuration, I rebooted the VM as per instructions on the libvirt site.

I can confirm that both the physical host interface and the virtual adapter are in the bridge together with "brctl show"

bridge name     bridge id               STP enabled     interfaces
br0             8000.5404a67ee5e2       no              eth0
                                                        vnet0

On the guest, I have the following configuration in /etc/network/interfaces:

# The primary network interface
auto eth0
iface eth0 inet static
  address  10.9.114.91
  netmask  255.255.255.224
  gateway  10.9.114.65

This was verified with ifconfig:

eth0      Link encap:Ethernet  HWaddr 52:54:00:06:f1:d6
          inet addr:10.9.114.91  Bcast:10.9.114.95  Mask:255.255.255.224
          inet6 addr: fe80::5054:ff:fec3:14e0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:32690 errors:0 dropped:0 overruns:0 frame:0
          TX packets:81 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2978342 (2.8 MiB)  TX bytes:4550 (4.4 KiB)

Based on my understanding, this should mean that the host and guest are essentially sharing a virtual switch connected to the gateway.

However, a ping from the host to the guest does not work, shown below:

pricetx@titan:/home/pricetx>ping 10.9.114.91
PING 10.9.114.91 (10.9.114.91) 56(84) bytes of data.
^C
--- 10.9.114.91 ping statistics ---
8 packets transmitted, 0 received, 100% packet loss, time 6999ms

I decided to check the ARP table to see if the host could at least arp across the bridge:

pricetx@titan:/home/pricetx>sudo arp -an
? (10.9.114.91) at 52:54:00:06:f1:d6 [ether] on br0
? (10.9.114.65) at f4:cc:55:4b:55:7b [ether] on br0

It can apparently see the guest (and the gateway).

However, checking the arp table on the guest, I don't appear to be able to see anything:

root@kvmtest:~# arp -an
? (10.9.114.65) at <incomplete> on eth0

My problem appears to be with the guest being unable to see other hosts on the bridge. Any suggestions are welcome!

I will continue to research this, and if I find a solution in the meantime, i'll update this post.