Linux Bridge – multiple VLANs into default vlan

As far as I understand Debian network configuration infrastructure, using brX.YYY should not work as it would create a VLAN pseudo interface "above" the bridge, for which support has just begun being added to the kernel: https://lwn.net/Articles/513710/

Here is the kind of config I use:

# Bring up physical interface
iface eth0 inet manual

# Create VLAN interfaces
iface eth0.10 inet manual
         vlan_raw_device eth0

iface eth0.20 inet manual
...

# Create bridges
auto vmbr10
iface vmbr10 inet static
        address  10.10.10.81
        netmask  255.255.255.0
        #gateway  10.10.10.254
        bridge_ports eth0.10
        bridge_stp off
        bridge_fd 0

auto vmbr20
iface vmbr20 inet static
        address  10.10.20.81
...

If you want to use agregated/bonded interfaces:

• bond the physical interfaces
• create the VLAN interfaces on the bondX interface (bondX.YYY)
• use the bondX.YYY as bridge_ports

A lengthy solution ...

So I've been toying around with different concepts and I think I've made a workable solution. Basically, there is a main bridge trunk (bt) that eth0 belongs to. Then for each firewall VM has its own bridge.

bridge name     bridge id               STP enabled     interfaces
bt              8000.002618895a72       no              eth0
bt-fw1          8000.000000000000       no              tap100i0
bt-fw2          8000.000000000000       no              tap101i0

At this point the tagged traffic from the firewall VMs will enter their own bridge for traffic to enter.

I then created VLANs for each interface that needs tagged traffic - and the respective VLAN on the main bridge for untagged traffic.

Eg. VLANs 1, 4000-4005

bt-fw1.1       | 1  | bt-fw1
bt-fw1.4000    | 4000  | bt-fw1
bt-fw1.4001    | 4001  | bt-fw1
bt-fw1.4002    | 4002  | bt-fw1
bt-fw1.4003    | 4003  | bt-fw1
bt-fw1.4004    | 4004  | bt-fw1
bt-fw1.4005    | 4005  | bt-fw1
bt-fw2.1       | 1  | bt-fw2
bt-fw2.4000    | 4000  | bt-fw2
bt-fw2.4001    | 4001  | bt-fw2
bt-fw2.4002    | 4002  | bt-fw2
bt-fw2.4003    | 4003  | bt-fw2
bt-fw2.4004    | 4004  | bt-fw2
bt-fw2.4005    | 4005  | bt-fw2
bt.1           | 1  | bt
bt.4000        | 4000  | bt
bt.4001        | 4001  | bt
bt.4002        | 4002  | bt
bt.4003        | 4003  | bt
bt.4004        | 4004  | bt
bt.4005        | 4005  | bt

Then for each VLAN, a bridge is created, that combines all respective VLANs from each interface to allow untagged communication between the main bridge and VM bridges.

bt.v1                   8000.2a8c73ad057d       no      bt-fw1.1
                                                        bt-fw2.1
                                                        bt.1
bt.v4000                8000.2a8c73ad057d       no      bt-fw1.4000
                                                        bt-fw2.4000
                                                        bt.4000
bt.v4001                8000.2a8c73ad057d       no      bt-fw1.4001
                                                        bt-fw2.4001
                                                        bt.4001
bt.v4002                8000.2a8c73ad057d       no      bt-fw1.4002
                                                        bt-fw2.4002
                                                        bt.4002
bt.v4003                8000.2a8c73ad057d       no      bt-fw1.4003
                                                        bt-fw2.4003
                                                        bt.4003
bt.v4004                8000.2a8c73ad057d       no      bt-fw1.4004
                                                        bt-fw2.4004
                                                        bt.4004
bt.v4005                8000.2a8c73ad057d       no      bt-fw1.4005
                                                        bt-fw2.4005
                                                        bt.4005

This now allows trunked interfaces to the firewall VMs, with only the VLANs I want on them.

Any future guests that need a single untagged VLAN only, can merely be added to the respective bt.X bridge.

Adding host IP interfaces is as easy as adding the IP to the respective VLAN bridge

Eg.

ip addr add 192.168.100.1/24 dev bt.v4005

A helper script

As my /etc/network/interfaces file could rapidly end up being massive, I wrote a small script that allows for minimal configuration - but with the same desired end result.

In the /etc/network/interfaces file it contains

# bridge bt-c0-fw1 vlan 1 4000-4005 interfaces tap100i0
# bridge bt-c0-fw2 vlan 1 4000-4005 interfaces tap101i0
# bridge bt vlan 1 4000-4005 interfaces eth0
# bridge-vlan bt vlan 1 4000-4005 interfaces bt-fw1 bt-fw2 bt

auto eth0
iface eth0 inet manual
  post-up /scripts/build-bridges.sh || /bin/true

Then in the post-up script, /scripts/build-bridges.sh

#!/bin/bash

CONFIG="/etc/network/interfaces"
DEFINERS="vlan|interfaces"
DEBUG=0

# Tear down all interfaces
modprobe -r 8021q
modprobe 8021q

brctl show | awk 'NR>1{print $1}' | while read BRIDGE; do
  ifconfig "$BRIDGE" down
  brctl delbr "$BRIDGE"
done

function run()
{
  if [ $DEBUG -eq 1 ]; then
    echo "$@"
  else
    eval "$@"
  fi
}

function get_vars()
{
  DATA=( $(echo "$2" | grep -Eoh "$1.+?" | sed -E "s/^$1 //g;s/($DEFINERS).+//g") )
  for VAL in ${DATA[@]}; do
    echo $VAL | grep -qE "[0-9]+-[0-9]+"
    if [ $? -eq 0 ]; then
      LOWER=$(echo $VAL | cut -f1 -d"-")
      UPPER=$(echo $VAL | cut -f2 -d"-")
      for i in $(seq $LOWER $UPPER); do
        echo $i
      done
    else
      echo $VAL
    fi
  done
}

# Build bridges
while read LINE; do
  BRIDGE=$(get_vars bridge "$LINE")
  if [[ ! "$BRIDGE" == "" ]]; then
    run brctl addbr $BRIDGE
    run ifconfig $BRIDGE up
    for INTERFACE in $(get_vars interfaces "$LINE"); do
      ifconfig $INTERFACE >/dev/null 2>&1
      if [ $? -eq 0 ]; then
        run brctl addif $BRIDGE $INTERFACE
      fi
    done
    run ifconfig $BRIDGE up
    for VLAN in $(get_vars vlan "$LINE"); do
      run vconfig add $BRIDGE $VLAN 2>&1 | grep -vE "(VLAN 1 does not work|consider another number)"
      run ifconfig $BRIDGE.$VLAN up
    done
  fi
done < <(grep -E "^# bridge " $CONFIG)

# Build vlan bridges
while read LINE; do
  BRIDGE=$(get_vars "bridge-vlan" "$LINE")
  for VLAN in $(get_vars " vlan" "$LINE"); do
    run brctl addbr $BRIDGE.v$VLAN
    run ifconfig $BRIDGE.v$VLAN up
    for INTERFACE in $(get_vars interfaces "$LINE"); do
      ifconfig $INTERFACE.$VLAN >/dev/null 2>&1
      if [ $? -eq 0 ]; then
        run brctl addif $BRIDGE.v$VLAN $INTERFACE.$VLAN
      fi
    done
  done
done < <(grep -E "^# bridge-vlan " $CONFIG)

exit 0

I hope this helps someone else, I've spent DAYS reading and testing and this seems to be the most elegant, easy to manage solution.

There's some good reading here too http://blog.davidvassallo.me/2012/05/05/kvm-brctl-in-linux-bringing-vlans-to-the-guests/