I don't know if this will help but I just ran into a similar problem while troubleshooting keepalived on two Linux KVM guests each running Ubuntu 10.04. I found that while running tcpdump -i eth1 would see the multicast addresses (which is a different subnet then the IP assigned to the NIC) but if I ran tcpdump -i any it wouldn't. I did some further testing of tcpdump while monitoring dmesg and found when I used the eth1 device on tcpdump it would put the NIC into promiscuous but when using the "any" device, neither eth0 or eth1 entered promiscuous. This contrary to how it's handled on a physical host where "any" puts all NIC's into promiscuous or at least on the hosts I tested with.
I ran the command ip link set eth1 promisc on
and then when I used the "any" device, it was now able to see the traffic. This applied equally to eth0 however I know the traffic I wanted wasn't coming there so I only did this to test. You can save this for the host by editing /etc/network/interfaces and adding a line beginning with "post-up " followed by the command you just used and this ensures the device enters promiscuous when the best is booted.
I don't believe a NIC would normally need to in promiscuous to see multicast traffic but in this case with the KVM guest it seems that was the case and it appears if the device is not set to promiscuous then it only sees IP packets on a subnet with the same IP as the NIC if not in promiscuous. Snort uses libpcap, the same library as tcpdump IIRC and if it's trying to set promiscuous via the any interface then it seems it's not succeeding where it should. I don't believe keepalive requires promiscuous under normal circumstances but in this case it seems to be the only way to see the multicast traffic.
Hope this helps.
A lengthy solution ...
So I've been toying around with different concepts and I think I've made a workable solution. Basically, there is a main bridge trunk (bt
) that eth0
belongs to. Then for each firewall VM has its own bridge.
bridge name bridge id STP enabled interfaces
bt 8000.002618895a72 no eth0
bt-fw1 8000.000000000000 no tap100i0
bt-fw2 8000.000000000000 no tap101i0
At this point the tagged traffic from the firewall VMs will enter their own bridge for traffic to enter.
I then created VLANs for each interface that needs tagged traffic - and the respective VLAN on the main bridge for untagged traffic.
Eg. VLANs 1, 4000-4005
bt-fw1.1 | 1 | bt-fw1
bt-fw1.4000 | 4000 | bt-fw1
bt-fw1.4001 | 4001 | bt-fw1
bt-fw1.4002 | 4002 | bt-fw1
bt-fw1.4003 | 4003 | bt-fw1
bt-fw1.4004 | 4004 | bt-fw1
bt-fw1.4005 | 4005 | bt-fw1
bt-fw2.1 | 1 | bt-fw2
bt-fw2.4000 | 4000 | bt-fw2
bt-fw2.4001 | 4001 | bt-fw2
bt-fw2.4002 | 4002 | bt-fw2
bt-fw2.4003 | 4003 | bt-fw2
bt-fw2.4004 | 4004 | bt-fw2
bt-fw2.4005 | 4005 | bt-fw2
bt.1 | 1 | bt
bt.4000 | 4000 | bt
bt.4001 | 4001 | bt
bt.4002 | 4002 | bt
bt.4003 | 4003 | bt
bt.4004 | 4004 | bt
bt.4005 | 4005 | bt
Then for each VLAN, a bridge is created, that combines all respective VLANs from each interface to allow untagged communication between the main bridge and VM bridges.
bt.v1 8000.2a8c73ad057d no bt-fw1.1
bt-fw2.1
bt.1
bt.v4000 8000.2a8c73ad057d no bt-fw1.4000
bt-fw2.4000
bt.4000
bt.v4001 8000.2a8c73ad057d no bt-fw1.4001
bt-fw2.4001
bt.4001
bt.v4002 8000.2a8c73ad057d no bt-fw1.4002
bt-fw2.4002
bt.4002
bt.v4003 8000.2a8c73ad057d no bt-fw1.4003
bt-fw2.4003
bt.4003
bt.v4004 8000.2a8c73ad057d no bt-fw1.4004
bt-fw2.4004
bt.4004
bt.v4005 8000.2a8c73ad057d no bt-fw1.4005
bt-fw2.4005
bt.4005
This now allows trunked interfaces to the firewall VMs, with only the VLANs I want on them.
Any future guests that need a single untagged VLAN only, can merely be added to the respective bt.X
bridge.
Adding host IP interfaces is as easy as adding the IP to the respective VLAN bridge
Eg.
ip addr add 192.168.100.1/24 dev bt.v4005
A helper script
As my /etc/network/interfaces
file could rapidly end up being massive, I wrote a small script that allows for minimal configuration - but with the same desired end result.
In the /etc/network/interfaces
file it contains
# bridge bt-c0-fw1 vlan 1 4000-4005 interfaces tap100i0
# bridge bt-c0-fw2 vlan 1 4000-4005 interfaces tap101i0
# bridge bt vlan 1 4000-4005 interfaces eth0
# bridge-vlan bt vlan 1 4000-4005 interfaces bt-fw1 bt-fw2 bt
auto eth0
iface eth0 inet manual
post-up /scripts/build-bridges.sh || /bin/true
Then in the post-up script, /scripts/build-bridges.sh
#!/bin/bash
CONFIG="/etc/network/interfaces"
DEFINERS="vlan|interfaces"
DEBUG=0
# Tear down all interfaces
modprobe -r 8021q
modprobe 8021q
brctl show | awk 'NR>1{print $1}' | while read BRIDGE; do
ifconfig "$BRIDGE" down
brctl delbr "$BRIDGE"
done
function run()
{
if [ $DEBUG -eq 1 ]; then
echo "$@"
else
eval "$@"
fi
}
function get_vars()
{
DATA=( $(echo "$2" | grep -Eoh "$1.+?" | sed -E "s/^$1 //g;s/($DEFINERS).+//g") )
for VAL in ${DATA[@]}; do
echo $VAL | grep -qE "[0-9]+-[0-9]+"
if [ $? -eq 0 ]; then
LOWER=$(echo $VAL | cut -f1 -d"-")
UPPER=$(echo $VAL | cut -f2 -d"-")
for i in $(seq $LOWER $UPPER); do
echo $i
done
else
echo $VAL
fi
done
}
# Build bridges
while read LINE; do
BRIDGE=$(get_vars bridge "$LINE")
if [[ ! "$BRIDGE" == "" ]]; then
run brctl addbr $BRIDGE
run ifconfig $BRIDGE up
for INTERFACE in $(get_vars interfaces "$LINE"); do
ifconfig $INTERFACE >/dev/null 2>&1
if [ $? -eq 0 ]; then
run brctl addif $BRIDGE $INTERFACE
fi
done
run ifconfig $BRIDGE up
for VLAN in $(get_vars vlan "$LINE"); do
run vconfig add $BRIDGE $VLAN 2>&1 | grep -vE "(VLAN 1 does not work|consider another number)"
run ifconfig $BRIDGE.$VLAN up
done
fi
done < <(grep -E "^# bridge " $CONFIG)
# Build vlan bridges
while read LINE; do
BRIDGE=$(get_vars "bridge-vlan" "$LINE")
for VLAN in $(get_vars " vlan" "$LINE"); do
run brctl addbr $BRIDGE.v$VLAN
run ifconfig $BRIDGE.v$VLAN up
for INTERFACE in $(get_vars interfaces "$LINE"); do
ifconfig $INTERFACE.$VLAN >/dev/null 2>&1
if [ $? -eq 0 ]; then
run brctl addif $BRIDGE.v$VLAN $INTERFACE.$VLAN
fi
done
done
done < <(grep -E "^# bridge-vlan " $CONFIG)
exit 0
I hope this helps someone else, I've spent DAYS reading and testing and this seems to be the most elegant, easy to manage solution.
There's some good reading here too http://blog.davidvassallo.me/2012/05/05/kvm-brctl-in-linux-bringing-vlans-to-the-guests/
Best Answer
What you want, while usefull, is not very straighfoward to do. Basically, you have 3 possibilities:
Solution n.1 is (probably) the most complete, but the most difficult to setup.
Solution n.2 is what I adopted for my lab test. For example, for VLAN 10 and 20, you had do create a similar setup:
Please note that if you need to have untagged traffic also on the tagged bridge, you must issue the following ebtables rule:
ebtables -t broute -A BROUTING -i eth0 -p 802.1Q -j DROP
This rule is needed due to how the Linux networking stack manage incoming packets. For more information, see here, here and hereSolution n.3 should be the clever one, but you need kernel support for it (and libvirt also). CentOS7 should have a sufficient recent kernel, but I don't know if all other software pieces are ready.
Solution n.4 (bonus one): if you don't need guest/host communication, give a look at the macvtap driver. While I don't use it, you should be able to setup a similar solution:
For me, the impossibility to communicate from/to guest/host is too much of a limitation, but your requirement may be different.
Finally, have a look here, it's worth reading.