Network Performance – Why Iperf Still Reports 1Gbps Performance When Using Bonding Over Two 1Gbps Adapters?

bondingnetworkingperformanceUbuntu

As I understand it, bonding brings among other benefits the ability to increase the network speed between two machines in a LAN.

Bonding […] means combining several network interfaces (NICs) to a single link, providing either high-availability, load-balancing, maximum throughput, or a combination of these.

Source: Ubuntu documentation, emphasis mine.

I have bonding configured on two servers; both have two 1Gbps NIC adapters. When testing speed between those servers using iperf, the report indicates:

  • 930 to 945 Mbits/sec when using balance-rr bonding mode.
  • 520 to 530 Mbits/sec from machine A to B when using 802.3ad,
  • 930 to 945 Mbits/sec from machine B to A when using 802.3ad.

An interesting thing is that when using 802.3ad, ifconfig indicates that practically all RX is on eth0 (2.5 GB vs. a few KB/MB) and all TX on eth1 on machine A, and the inverse on machine B.

When asking iperf to use multiple connections (iperf -c 192.168.1.2 -P 10), the obtained sum is very close to the results displayed when using a single connection.

Two machines are connected to a Netgear GS728TS which has LACP configured properly (I hope), with two LAGs covering two ports each. IEEE 802.3x mode is enabled.

Is iperf suited well for this sort of tests? If yes, is there something I'm missing?

Best Answer

Bonded interfaces do not grant additional bandwidth to individual network flows. So if you're only running one copy of iperf then you will only be able to use one network interface at a time. If you have two NIC in a lagg then you'll need at least two completely independent copies of iperf running on the computer to see any simultaneous utilization. This will apply to actual loads as well - eg a Samba client will still only see 1Gb throughput, but two clients could each see 1Gb if your lagg has two NICs. This all assumes you have the lagg configured to use both NICs (The 802.3ad option will do this).