MAC addresses on dual-NIC mainboards

dhcpmac addressnicpxe-boot

Here's a weird problem.

We've got a number of devices with dual-NIC mainboards. Some are Realtek NICs, which suck. Some are Intel e1000s, which don't.

I've just noticed on 2 machines, one is an Intel NIC, one is a Realtek, that when
I put the MAC address of one machine into the dhcpd.conf file on our DHCP server to get it to PXE boot the machine into a rebuild environment, initially everything is fine.

The server gets a DHCP allocation, and PXE boots into the Ubuntu preseed enviroment.

On one or two machines, it gets as far as Ubuntu's DHCP network configuration, and fails. If i pull up a busybox shell (on tty2 on the installing machine), and run ip link, I can see that the UP flag is set on the other NIC.

Here's some stuff.

  host xeon16-ghz240-gb48-node1 {
        hardware ethernet BC:AE:C5:07:1F:18;
        filename "pxelinux.0";
        next-server 192.168.123.80;
  }

That's what's in dhcpd.conf

This is what ip link on the evil machine looks like.
ip link output

Only one NIC is actually connected (deliberately).

As you can see, the NIC that's in the dhcpd config, is not marked as UP, and the link that is UP, isn't the one in DHCP.

So far I've seen this on two brands of dual-NIC configuration.

Does anyone know 1) what's causing it, and b) What we can do about it?

Best Answer

There's always more than one way to do anything :)

Solution 1

Motherboards with one of each?

Blacklist whichever module (ethtool -i eth0) is supporting the Realtek card.

Ubuntu supports module_name.blacklist=yes to blacklist it at boot and you should be able to change the modprobe options in the preseed environment so that it doesn't get probed later.

Solution 2

Let me rephrase the problem:

We have motherboards with two NICs and we want them to work consistently no matter which interface is plugged in. We can't always determine which interface (from the OS point of view) will be plugged in.

Set up bonding! Use an active-passive configuration (mode=active-backup miimon=100) with both interfaces as slaves. This way, it will always work no matter which interface is plugged in.

Solution 3

Are the motherboards consistent enough that the NICs always show up on the same PCI ID? Use udev rules to always assign the card on a particular PCI address to eth0 and the card on the other address to eth1.

Note that you can have two different udev rules that assign a device to eth0 - this allows you to handle the Realtek and e1000 case at the same time.

Related Topic