Windows – Why MPIO instead of 802.3ad Team for iSCSI

When you say you can't "get the traffic to utilize more than 1 network connection", there's two parts to it:

First, how many luns are you using Load balancing a single lun (one drive letter, so to speak) is much, much harder than load balancing multiple luns. You can get a huge head start by breaking the iSCSI share into two separate luns, then configuring iSCSI on the Windows box so that:

• Windows network card #1 is the primary path for lun #1, and it points to Linux network card A
• Windows network card #2 is the primary path for lun #2, and it points to Linux network card B (with a separate IP address)

This lets you bypass the one-connection limit in a duct-tape way, but you won't be able to get more than 1 network connection of throughput to a single lun at a given time.

Next, are you measuring send speeds or receive speeds?

To be able to send more than one network connection of traffic to an iSCSI device on the other end, you need to use MPIO on the Windows box and that's pretty much it. Configure MPIO with both IP addresses of the Linux host and you should be able to saturate two or more network cards of outgoing traffic.

However, to be able to receive more than one network connection of traffic on the Windows box, you're going to be pretty much out of luck unless you do something like channel bonding with your network cards. The Linux machine will only send traffic to one MAC address for one lun at a time. You can get around this by doing channel bonding so that the switch routes traffic for one MAC address through multiple network cards.

Here's a couple of my blog posts talking about the myth of active-active multipathing:

• http://www.brentozar.com/archive/2009/05/san-multipathing-part-1-what-are-paths/
• http://www.brentozar.com/archive/2009/05/san-multipathing-part-2-what-multipathing-does/

MPIO has various policies available to it. As the Coding Gorilla points out, most of those policies allow for load balancing across multiple connections to aggregate bandwidth. Both your initiator and target have to have multiple connections for it to actually be faster than single link speed however. Round Robin is a poor choice of policy; you should be using either Weighted Distribution or Least Queue Depth.

The iSCSI SAN and Server I have here have 4 ports each and I can actually get ~3.2Gbps under fairly ideal circumstances. If you need something faster than that, you'd be looking at FC or IB.

Also, do not use Trunking/Link Aggregation/etc on iSCSI links. When one links fails the connection will fail. You must use MPIO to accomplish link redundancy.