LACP vs 802.3ad

lacpsolaris

I’m looking to create an aggregation on a Solaris box using dladm. I understand that once the aggregation is created, 802.3ad will be used to balance the load depending on the policy (L2, L3 or L4). The only requirements are that the interfaces are connected to a single switch that supports 802.3ad and the interfaces are running at the same speed / full duplex. There are a few questions I’m hoping someone will comment on:

  1. By default LACP is disabled on each aggregation. What’s the benefit of enabling LACP? Wouldn’t I already be load balancing with 802.3ad and the default L4 policy, which, as I understand, selects the outbound interface based on a hash of the source and destination ports. Reading wikipedia, there appears to only be two benefits of LACP (1) failover and (2) automatic configuration. Doesn’t 802.3ad already support failover? If a link goes down, the switch will still try to transmit packets to that interface? It’s hard to believe that’s true. And in terms of automatic configuration, I’m not certain what needs to be configured on the switch. For 802.3ad, I assume the switch just needs to know which load balancing policy (L2, L3 or L4) to use for sending packets to the aggregation. Am I missing something? What’s the advantage of LACP over 802.3ad?

  2. I was reading online that NFS uses two connections between server/client: one for data and one for metadata, and that the typical transmission for packets in an aggregation is round-robin resulting in all the data traffic going over one interface with metadata on the other interface (assuming a two port aggregation). This seems to go against what I read about 802.3ad’s load balancing policy. If L4 is being used (Solaris dladm default), the outgoing interface is going to be based on the source and destination port, and assuming the switch is also using L4 the incoming interface will also be based on src/dst port. Am I wrong? BTW, does a layer 2 switch really look at the src/dst port? It seems resource intensive for a switch to pull the packet apart to calculate the hash and then reassemble. I also wouldn’t expect the outgoing and incoming interface to be used for the same src/dst hash i.e. perhaps the hash algorithm used by the host is different from the switch or they count ports from different ends. For this reason, I’m confused why a single stream would be limited to the max throughput of a single interface – if incoming and outgoing transmission may be on different interfaces.

I apologize for the fragmented post. I’m trying to get an understanding of the technologies and I haven’t been able to find a good tutorial or article on how these protocols are actually implemented. I see a lot of articles grouping 802.3ad and LACP as one and the same. Any comments will be appreciated.

Thanks!

Best Answer

IEEE 802.3ad is the standard for link aggregation, not withstanding the move of link aggregation standards to the 802.1 group, as 802.1ax.

The real advantage of LACP is the LACPDUs that transit the link from the switch to the host. These ensure that both sides of the link are capable of LACP. A secondary advantage is that with LACP, both the host and the switch think of all aggregated ports as a single port, allowing full use of all paths, as opposed to host-side LAG, where the switch still sees multiple ports, and all packets to the host traverse a single path, and only outbound packets from the host are load balanced across links.

If you're using a switch vendor that supports MLAG, or multi-chassis link aggregation, then you can use LACP to bond multiple links connected to multiple switches. This permits a great deal of resiliencey, while easing manageability and optimizing throughput.

But basically, if your switch supports LACP, use LACP. If your switch doesn't support LACP, then use non-LACP aggregation.

Related Topic