Linux – Ping does not work on TAP interfaces with bridge

bridgeiproute2linuxpolicy-routingrouting

I am trying to understand the linux bridging and local ip rules,

I have following topology on my linux laptop.

               br0 
     ___________|__________
     |                    |    
  |tap0                  tap1|
  |________Application_______|

The application above is creating 2 tap interfaces tap0 and tap1

I create the bridge and plug the tap interfaces to the bridge:

brctl addif br0 tap0
brctl addif br0 tap1

For ping to work, i need to add ip address to the interfaces,
So I add 192.168.13.1 to tap0 and 192.168.13.2 to tap1

The application reads from one interface and writes to another interface, for both the interfaces.

Now if I run "ping 192.168.13.2 -I tap0" 

PING 192.168.13.2 (192.168.13.2) from 192.168.13.1 tap0: 56(84) bytes of data.
From 192.168.13.1 icmp_seq=1 Destination Host Unreachable

tcpdump was showing arp could not be resolved, so I added static ARP entries:

arp -i tap0 -s 192.168.13.1 62:34:58:e7:8a:3a
arp -i tap1 -s 192.168.13.2 4a:6d:fa:51:7d:2d

brctl showmacs br0
port no mac addr        is local?   ageing timer
  2 4a:6d:fa:51:7d:2d   yes        0.00
  2 4a:6d:fa:51:7d:2d   yes        0.00
  1 62:34:58:e7:8a:3a   yes        0.00
  1 62:34:58:e7:8a:3a   yes        0.00

The bridge also seems to have learnt the MAC addresses.

However, the application and the tcpdump still receive 42 bytes packets
showing ARP packets not resolved yet, and ping give Host unreachable mesg.

My current routing table is:

ip route ls table main
169.254.0.0/16 dev virbr0  scope link  metric 1000 linkdown 
192.168.13.0/24 dev tap1  proto kernel  scope link  src 192.168.13.2 
192.168.13.0/24 dev tap0  proto kernel  scope link  src 192.168.13.1 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 linkdown

My current local routing table:

broadcast 192.168.13.0 dev tap1  proto kernel  scope link  src 192.168.13.2 
broadcast 192.168.13.0 dev tap0  proto kernel  scope link  src 192.168.13.1 
local 192.168.13.1 dev tap0  proto kernel  scope host  src 192.168.13.1 
local 192.168.13.2 dev tap1  proto kernel  scope host  src 192.168.13.2 
broadcast 192.168.13.255 dev tap1  proto kernel  scope link  src 192.168.13.2 
broadcast 192.168.13.255 dev tap0  proto kernel  scope link  src 192.168.13.1

I am having a feel that routing might not come into picture here: since this is a layer 2 broadcast domain. But since I am not well versed with linux bridging, I need some advice to proceed.

How can we make the ping work between tap0 and tap1.

Thanks
Nayan

Best Answer

I think you didn't grasp what's the role and usage of tun/tap interfaces, and that's the cause of your problems. Let me explain.

Consider the tap0 interface as the end of a virtual wire with two sides: the visible side: tap0, and the invisible side of the wire in the application: this invisible side has to be entirely handled by the application. The application will receive ethernet frames as data payload read on a file descriptor, if needed decode this data as ethernet (because it's in tap mode), then as ARP or IP and will then react to any needed event. In short your application requires a TCP/IP network stack, to be able to answer a ping.

Examples of such applications using tun/tap devices are openvpn or the combination of QEMU+VM+OS (QEMU is here to present this data payload to a network device driver of the VM's OS the same way this driver would see from an actual network device).

When you enslave tap0 and tap1 to the bridge, they become bridge ports: that's a hint they should not receive an IP (would it make sense to assign an IP to each port of your 24-ports switch?): those IPs are mostly ignored, except as an IP belonging to the host: a route with scope local is thus added to the local routing table. You could have added those IPs to the lo loopback interface with the same result. Beside this, there's no route that will work with those interfaces bridge ports, what matters now is their master br0.

What you should do is add an IP on the bridge itself (the route will then use the br0 interface) or instead you can create a veth pair, enslave one side to the bridge, and add an IP on the other side: the route will use the veth interface with the IP on it (and leave the bridge without IP which is cleaner imho).

Now for the remaining part it all depends on your application: it has to handle those 192.168.13.1 and 192.168.13.2 IPs itself, including first replying to ARP requests for them, before replying to ICMP echo requests.

If you're now discovering that the application's role is not just to create the tun/tap interfaces, then you should forget using an application and tun/tap interfaces: use network namespaces which duplicate as many network stacks as needed, create bridges where needed, and veth interfaces accross those namespaces. It can all be tested with ip netns and ip link add ... type veth ....

Here are some links to the first 3 parts of a tutorial making use of bridge and veth interfaces with network namespaces (alas they don't use ip netns which is easier to have the job done):

Fun with veth-devices, Linux bridges and VLANs in unnamed Linux network namespaces – I II III

Related Topic