A lengthy solution ...
So I've been toying around with different concepts and I think I've made a workable solution. Basically, there is a main bridge trunk (bt) that eth0 belongs to. Then for each firewall VM has its own bridge.
bridge name bridge id STP enabled interfaces
bt 8000.002618895a72 no eth0
bt-fw1 8000.000000000000 no tap100i0
bt-fw2 8000.000000000000 no tap101i0
At this point the tagged traffic from the firewall VMs will enter their own bridge for traffic to enter.
I then created VLANs for each interface that needs tagged traffic - and the respective VLAN on the main bridge for untagged traffic.
Eg. VLANs 1, 4000-4005
bt-fw1.1 | 1 | bt-fw1
bt-fw1.4000 | 4000 | bt-fw1
bt-fw1.4001 | 4001 | bt-fw1
bt-fw1.4002 | 4002 | bt-fw1
bt-fw1.4003 | 4003 | bt-fw1
bt-fw1.4004 | 4004 | bt-fw1
bt-fw1.4005 | 4005 | bt-fw1
bt-fw2.1 | 1 | bt-fw2
bt-fw2.4000 | 4000 | bt-fw2
bt-fw2.4001 | 4001 | bt-fw2
bt-fw2.4002 | 4002 | bt-fw2
bt-fw2.4003 | 4003 | bt-fw2
bt-fw2.4004 | 4004 | bt-fw2
bt-fw2.4005 | 4005 | bt-fw2
bt.1 | 1 | bt
bt.4000 | 4000 | bt
bt.4001 | 4001 | bt
bt.4002 | 4002 | bt
bt.4003 | 4003 | bt
bt.4004 | 4004 | bt
bt.4005 | 4005 | bt
Then for each VLAN, a bridge is created, that combines all respective VLANs from each interface to allow untagged communication between the main bridge and VM bridges.
bt.v1 8000.2a8c73ad057d no bt-fw1.1
bt-fw2.1
bt.1
bt.v4000 8000.2a8c73ad057d no bt-fw1.4000
bt-fw2.4000
bt.4000
bt.v4001 8000.2a8c73ad057d no bt-fw1.4001
bt-fw2.4001
bt.4001
bt.v4002 8000.2a8c73ad057d no bt-fw1.4002
bt-fw2.4002
bt.4002
bt.v4003 8000.2a8c73ad057d no bt-fw1.4003
bt-fw2.4003
bt.4003
bt.v4004 8000.2a8c73ad057d no bt-fw1.4004
bt-fw2.4004
bt.4004
bt.v4005 8000.2a8c73ad057d no bt-fw1.4005
bt-fw2.4005
bt.4005
This now allows trunked interfaces to the firewall VMs, with only the VLANs I want on them.
Any future guests that need a single untagged VLAN only, can merely be added to the respective bt.X bridge.
Adding host IP interfaces is as easy as adding the IP to the respective VLAN bridge
Eg.
ip addr add 192.168.100.1/24 dev bt.v4005
A helper script
As my /etc/network/interfaces file could rapidly end up being massive, I wrote a small script that allows for minimal configuration - but with the same desired end result.
In the /etc/network/interfaces file it contains
# bridge bt-c0-fw1 vlan 1 4000-4005 interfaces tap100i0
# bridge bt-c0-fw2 vlan 1 4000-4005 interfaces tap101i0
# bridge bt vlan 1 4000-4005 interfaces eth0
# bridge-vlan bt vlan 1 4000-4005 interfaces bt-fw1 bt-fw2 bt
auto eth0
iface eth0 inet manual
post-up /scripts/build-bridges.sh || /bin/true
Then in the post-up script, /scripts/build-bridges.sh
#!/bin/bash
CONFIG="/etc/network/interfaces"
DEFINERS="vlan|interfaces"
DEBUG=0
# Tear down all interfaces
modprobe -r 8021q
modprobe 8021q
brctl show | awk 'NR>1{print $1}' | while read BRIDGE; do
ifconfig "$BRIDGE" down
brctl delbr "$BRIDGE"
done
function run()
{
if [ $DEBUG -eq 1 ]; then
echo "$@"
else
eval "$@"
fi
}
function get_vars()
{
DATA=( $(echo "$2" | grep -Eoh "$1.+?" | sed -E "s/^$1 //g;s/($DEFINERS).+//g") )
for VAL in ${DATA[@]}; do
echo $VAL | grep -qE "[0-9]+-[0-9]+"
if [ $? -eq 0 ]; then
LOWER=$(echo $VAL | cut -f1 -d"-")
UPPER=$(echo $VAL | cut -f2 -d"-")
for i in $(seq $LOWER $UPPER); do
echo $i
done
else
echo $VAL
fi
done
}
# Build bridges
while read LINE; do
BRIDGE=$(get_vars bridge "$LINE")
if [[ ! "$BRIDGE" == "" ]]; then
run brctl addbr $BRIDGE
run ifconfig $BRIDGE up
for INTERFACE in $(get_vars interfaces "$LINE"); do
ifconfig $INTERFACE >/dev/null 2>&1
if [ $? -eq 0 ]; then
run brctl addif $BRIDGE $INTERFACE
fi
done
run ifconfig $BRIDGE up
for VLAN in $(get_vars vlan "$LINE"); do
run vconfig add $BRIDGE $VLAN 2>&1 | grep -vE "(VLAN 1 does not work|consider another number)"
run ifconfig $BRIDGE.$VLAN up
done
fi
done < <(grep -E "^# bridge " $CONFIG)
# Build vlan bridges
while read LINE; do
BRIDGE=$(get_vars "bridge-vlan" "$LINE")
for VLAN in $(get_vars " vlan" "$LINE"); do
run brctl addbr $BRIDGE.v$VLAN
run ifconfig $BRIDGE.v$VLAN up
for INTERFACE in $(get_vars interfaces "$LINE"); do
ifconfig $INTERFACE.$VLAN >/dev/null 2>&1
if [ $? -eq 0 ]; then
run brctl addif $BRIDGE.v$VLAN $INTERFACE.$VLAN
fi
done
done
done < <(grep -E "^# bridge-vlan " $CONFIG)
exit 0
I hope this helps someone else, I've spent DAYS reading and testing and this seems to be the most elegant, easy to manage solution.
There's some good reading here too http://blog.davidvassallo.me/2012/05/05/kvm-brctl-in-linux-bringing-vlans-to-the-guests/
You can't see VLAN tag from tcpdump -i eth0 output on i686/x86_64 architecture because of VLAN acceleration. the VLAN layer will be filtered by kernel so it always looks untagged. Please refer to Bug 498981 - tcpdump cannot deal with 802.1q vlan tag
According to your case, you can get VLAN tags via:
tcpdump -i eth0 -Uw - | tcpdump -en -r - vlan 20
You should see the following output:
<timestamp> <mac-addr-of-eth0> > Broadcast, ethertype 802.1Q (0x8100), length 346: vlan 20, p 0, ethertype IPv4, 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP
Best Answer
Of course, bridge would forward any VLAN anyway so no need for multiple VLAN bridges :-)
So, I have a br0 with interfaces eth0 and eth1 that has no IP set and is used to forward any traffic. To access the unit using a management VLAN 200, I create a br0.200 and set its IP.
So simple :-)