Ubuntu 12.04 Server – eth0 1Gbps NIC eth1 10Gbps NIC – all traffic using eth0

10gbethernetmac-osxnetworkingUbuntu

UPDATE: (per request below)

serveradmin@FILESERVER:~$ cat /etc/hosts
127.0.0.1   localhost
127.0.1.1   FILESERVER

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Ubuntu Server 12.04.1 x64

Primary role is an NFS fileserver, for Mac OSX Clients.

Hardware:

Eth0: 00:19.0 Ethernet controller: Intel Corporation 82579V Gigabit Network Connection (rev 04)

Eth1: 07:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC

Config:

ifconfig
eth0      Link encap:Ethernet  HWaddr <MACADDRESS>  
      inet addr:192.168.0.150  Bcast:192.168.0.255  Mask:255.255.255.0
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:460042020 errors:0 dropped:148 overruns:0 frame:0
      TX packets:231906707 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:581431978417 (581.4 GB)  TX bytes:259057368617 (259.0 GB)
      Interrupt:20 Memory:f7d00000-f7d20000 

eth1      Link encap:Ethernet  HWaddr <MACADDRESS>  
      inet addr:192.168.0.100  Bcast:192.168.0.255  Mask:255.255.255.0
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:6832208 errors:0 dropped:2 overruns:0 frame:0
      TX packets:376 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:513826442 (513.8 MB)  TX bytes:33688 (33.6 KB)
      Interrupt:59 

lo        Link encap:Local Loopback  
      inet addr:127.0.0.1  Mask:255.0.0.0
      UP LOOPBACK RUNNING  MTU:16436  Metric:1
      RX packets:507 errors:0 dropped:0 overruns:0 frame:0
      TX packets:507 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:45057 (45.0 KB)  TX bytes:45057 (45.0 KB)

nano /etc/network/interfaces

#The loopback network interface
auto lo
iface lo inet loopback

#The primary network interface
auto eth0
iface eth0 inet static
    address 192.168.0.150
    netmask 255.255.255.0
    network 192.168.0.0
    broadcast       192.168.0.255
    gateway 192.168.0.1
    dns-nameservers 192.168.0.1 8.8.8.8

#second network interface
auto eth1
iface eth1 inet static
    address 192.168.0.100
    netmask 255.255.255.0
    network 192.168.0.0
    broadcast       192.168.0.255
    gateway 192.168.0.1
    dns-nameservers 192.168.0.1 8.8.8.8

Currently I am using on the OSX clients: nfs://192.168.0.100/Volumes/Storage to mount the NFS share.

My problem is why would all the data (and I have checked using various monitoring tools bmon, iftop, glances, etc) be going over the slower connection??

Also, after configuring /etc/network/interfaces with the above setup I always get an error message at bootup something about waiting for network configuration. Are these connected?

Best Answer

You can restrict interfaces to only respond to ARP broadcasts for addresses configured on the interface itself (rather than any on the system) via the following:

sysctl net.ipv4.conf.eth[num].arp_filter=1

eg, in this case:

sysctl net.ipv4.conf.eth0.arp_filter=1
sysctl net.ipv4.conf.eth1.arp_filter=1

Make the change permanent by adding the lines (minus the sysctl command) to /etc/sysctl.conf