Linux – irqbalance on linux and dropped packets

linuxpackets

I am investigating dropped packets on a dual core, quad XEON box running Linux. One thing I see is irqbalance running on the system. I have a couple of questions. Reading the docs here I think I understand how it is supposed to work, but one thing that seems confusing is this line – "The current Linux irqbalance program is several years old in design, and is blissfully unaware of the ideas of Quad (or even Dual) core or even power usage. The program is conceptually closer to the naive balancing than to the smart interrupt balancer." This seems to indicate that there is an old and a new version of irqbalance. Is this the case? How can you tell which is running on the machine.

Also, if my goal is to optimize packet processing during bursts, do I want to run irqbalance, or should I manually bind the network card to a set of CPUs?

Best Answer

One solution to this problem is to use a combination of disabling irqbalance, cpu isolation and pinning the NIC interrupts to an isolated CPU.

  • Disable irqbalance. This will cause all interrupts to be on CPU0
  • Isolate CPUs 1-3 via a kernel boot parameter "isolcpus=1-3"
  • Manually pin the interrupts for the NIC to one of the isolated CPUs (echo 'cpumask' . /proc/irq//smp_affinity).

Doing this will allow the NIC to have its IRQs run on a CPU which is not bothered by random kernel threads and IRQs from other hardware devices.

Here's a good explanation of how to pin an IRQ to a specific CPU.

https://cs.uwaterloo.ca/~brecht/servers/apic/SMP-affinity.txt

Related Topic