Freebsd – Using two network interfaces for FreeBSD VM

How do I turn on/off one network interface without affecting the other in FreeBSD?

I have a FreeBSD 8.2 virtual machine (Virtual Box 4.1.4), with two network interfaces, one bridged to my Ethernet card and another to WiFi on the host OS (Mac OSX Lion 10.7.2). In the VM, I assigned the first two network interfaces (em0 and em1) to static IP addresses, and set the default router IP to my router's IP, as shown in the config here:

/etc/rc.conf :

inetd_enable="NO"
sshd_enable="YES"

ifconfig_em0="inet 192.168.1.109 netmask 255.255.255.0"
ifconfig_em1="inet 192.168.1.129 netmask 255.255.255.0"
defaultrouter="192.168.1.1"

netstat -r :

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.1.1        UGS         2       71    em0
127.0.0.1          link#3             UH          0        0    lo0
192.168.1.0/24     link#1             U           2       78    em0
192.168.1.109      link#1             UHS         0        0    lo0
192.168.1.129      link#2             UHS         0        0    lo0

With the configuration above, I can ssh to em0 or em1 from the host OS, but if I take down em0 with:

sudo ifconfig em0 down

…em1 loses network access, as if it were also taken down. That is, the host OS can't ssh into em1; and the guest OS can't ping google.com or ping 192.168.1.130 (IP of host OS).

I don't have this problem in Ubuntu Server 11.10: I can setup a static IP for my WiFi bridge and a dynamic IP for my Ethernet bridge; and take either of them down without affecting the other.

BEFORE em0 down

$ ifconfig -a
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
    ether 08:00:27:16:4d:32
    inet 192.168.1.109 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
    ether 08:00:27:3f:a7:44
    inet 192.168.1.129 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=3<RXCSUM,TXCSUM>
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
    inet6 ::1 prefixlen 128 
    inet 127.0.0.1 netmask 0xff000000 
    nd6 options=3<PERFORMNUD,ACCEPT_RTADV>

$ arp -an
? (192.168.1.129) at 08:00:27:3f:a7:44 on em1 permanent [ethernet]
? (192.168.1.130) at c4:2c:03:2b:73:d1 on em0 expires in 314 seconds [ethernet]

AFTER em0 down

$ ifconfig -a
em0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 08:00:27:16:4d:32
        inet 192.168.1.109 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 08:00:27:3f:a7:44
        inet 192.168.1.129 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet6 ::1 prefixlen 128
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        
$ arp -an
? (192.168.1.129) at 08:00:27:3f:a7:44 on em1 permanent [ethernet]
? (192.168.1.130) at c4:2c:03:2b:73:d1 on em0 expires in 190 seconds [ethernet]

Output (as requested):

@host : ~/tmp
$ ifconfig -a
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
    options=3<RXCSUM,TXCSUM>
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 
    inet 127.0.0.1 netmask 0xff000000 
    inet6 ::1 prefixlen 128 
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
stf0: flags=0<> mtu 1280
en0: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
    options=2b<RXCSUM,TXCSUM,VLAN_HWTAGGING,TSO4>
    ether c4:2c:03:2b:73:d1 
    inet6 fe80::c62c:3ff:fe2b:73d1%en0 prefixlen 64 scopeid 0x4 
    inet 192.168.1.130 netmask 0xffffff00 broadcast 192.168.1.255
    media: autoselect (100baseTX <full-duplex,flow-control>)
    status: active
fw0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 4078
    lladdr d8:a2:5e:ff:fe:f2:f9:d4 
    media: autoselect <full-duplex>
    status: inactive
en1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    ether d8:30:62:58:93:7d 
    inet6 fe80::da30:62ff:fe58:937d%en1 prefixlen 64 scopeid 0x6 
    inet 192.168.1.146 netmask 0xffffff00 broadcast 192.168.1.255
    media: autoselect
    status: active
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
    ether 0a:30:62:58:93:7d 
    media: autoselect
    status: active
vboxnet0: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    ether 0a:00:27:00:00:00 

@host : ~/tmp
$ sudo arp -ad 
192.168.1.1 (192.168.1.1) deleted
192.168.1.109 (192.168.1.109) deleted

@host : ~/tmp
$ arp -an
? (192.168.1.1) at 0:1d:7e:f8:1c:dd on en0 ifscope [ethernet]

@host : ~/tmp
$ ping -c3 192.168.1.109
PING 192.168.1.109 (192.168.1.109): 56 data bytes
64 bytes from 192.168.1.109: icmp_seq=0 ttl=64 time=0.418 ms
64 bytes from 192.168.1.109: icmp_seq=1 ttl=64 time=0.457 ms
64 bytes from 192.168.1.109: icmp_seq=2 ttl=64 time=0.468 ms

--- 192.168.1.109 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.418/0.448/0.468/0.021 ms

@host : ~/tmp
$ ping -c3 192.168.1.129
PING 192.168.1.129 (192.168.1.129): 56 data bytes
64 bytes from 192.168.1.129: icmp_seq=0 ttl=64 time=104.138 ms
64 bytes from 192.168.1.129: icmp_seq=1 ttl=64 time=22.004 ms
64 bytes from 192.168.1.129: icmp_seq=2 ttl=64 time=44.780 ms

--- 192.168.1.129 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 22.004/56.974/104.138/34.622 ms

@host : ~/tmp
$ arp -an
? (192.168.1.1) at 0:1d:7e:f8:1c:dd on en0 ifscope [ethernet]
? (192.168.1.109) at 8:0:27:16:4d:32 on en0 ifscope [ethernet]
? (192.168.1.129) at d8:30:62:58:93:7d on en0 ifscope [ethernet]

#############################
# GUEST em0 down here
#############################

@host : ~/tmp
$ arp -an
? (192.168.1.1) at 0:1d:7e:f8:1c:dd on en0 ifscope [ethernet]
? (192.168.1.109) at 8:0:27:16:4d:32 on en0 ifscope [ethernet]
? (192.168.1.129) at d8:30:62:58:93:7d on en0 ifscope [ethernet]

@host : ~/tmp
$ sudo arp -ad
192.168.1.1 (192.168.1.1) deleted
192.168.1.109 (192.168.1.109) deleted
192.168.1.129 (192.168.1.129) deleted

@host : ~/tmp
$ ping -c3 192.168.1.129
PING 192.168.1.129 (192.168.1.129): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1

--- 192.168.1.129 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

@host : ~/tmp
$ arp -an
? (192.168.1.1) at 0:1d:7e:f8:1c:dd on en0 ifscope [ethernet]
? (192.168.1.129) at d8:30:62:58:93:7d on en0 ifscope [ethernet]





###################
GUEST OS

@guest : ~/tmp
$ sudo arp -ad
192.168.1.129 (192.168.1.129) deleted

@guest : ~/tmp
$ arp -an
? (192.168.1.129) at 08:00:27:3f:a7:44 on em1 permanent [ethernet]

#############################
# HOST pings .109 and .129 here
#############################

@guest : ~/tmp
$ arp -an
? (192.168.1.129) at 08:00:27:3f:a7:44 on em1 permanent [ethernet]
? (192.168.1.130) at c4:2c:03:2b:73:d1 on em0 expires in 1188 seconds [ethernet]

@guest : ~/tmp
$ sudo ifconfig em0 down

@guest : ~/tmp
$ arp -an
? (192.168.1.129) at 08:00:27:3f:a7:44 on em1 permanent [ethernet]
? (192.168.1.130) at c4:2c:03:2b:73:d1 on em0 expires in 1170 seconds [ethernet]

@guest : ~/tmp
$ sudo arp -ad
192.168.1.129 (192.168.1.129) deleted
delete: cannot locate 192.168.1.130

#############################
# HOST pings .129 here
#############################

@guest : ~/tmp
$ arp -an
? (192.168.1.129) at 08:00:27:3f:a7:44 on em1 permanent [ethernet]
? (192.168.1.130) at c4:2c:03:2b:73:d1 on em0 expires in 1095 seconds [ethernet]

Best Answer

What you are doing is pretty strange:
1) You have same subnet reachable from different interfaces which will make your life a bit difficult for some types of server software
2) When you put 192.168.1.0/24's interface down you should mangle routes:

sudo route add -net 192.168.1.0/24 -iface em1

One of pretty straightforward solutions for your problem is to use link aggregation technique.

There is lagg(4) device with failover mode for that. There is config from my notebook which always stays on fastest available link:

ifconfig_em0="ether 00:26:c7:xx:xx:xx"
ifconfig_wlan0="WPA"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto failover laggport em0 laggport wlan0 DHCP"

in your case you should have lagg with em0 and em1. Also note that I set MAC of em0 same as my wifi card.

As another solution you can set up an IGP between host and guest OS (e.g., OSPF).

How do I turn on/off one network interface without affecting the other in FreeBSD?

Best Answer

Related Solutions

Router – Add route on OpenBSD for DSL modem on external network

Linux Networking – Two Network Interfaces and Two IP Addresses on the Same Subnet in Linux

Related Topic