I'm sending data on multiple ports evenly, but only receiving data on two ports at my receiver. Both servers (on the same tcp subnet) are connected to pair of Nexus 7K switches (I believe they are setup as peer switches), and each are connected to 4 port LACP port channel (My understand is these are virtual port channels, Layer 2 Port-Channels) with the default load balance method. Would changing the Nexus load balance method to src-dst-port help me spread data? Even if the port channel is a layer 2 port channel, for IP traffic, could it use the tcp port ? I'm not a network engineer, but need work with network engineers, so I want to try and understand as much as possible so I don't waste their time. Thanks for any thoughts 😉
Will Nexus load-balance method “src-dst-port” help balance link usage between two servers on same subnet
cisco-nexus-7kload balancingport-channel
Related Solutions
We don't have router, all we have L2 switch network. We have many servers behind trunk, so should i use src-dsp-ip? or src-dst-mac ?
You need to weigh several factors to determine which method to use in different places on your network. Different network engineers will come up with different recommendations. This is really leading to opinion-based answers, and it is probably better suited for a discussion on Network Engineering Chat.
Cisco has some guidance that you should understand:
Chapter: Configuring EtherChannels
Load-Balancing and Forwarding Methods
EtherChannel balances the traffic load across the links in a channel by reducing part of the binary pattern formed from the addresses in the frame to a numerical value that selects one of the links in the channel. EtherChannel load-balancing can use MAC addresses or IP addresses, source or destination addresses, or both source and destination addresses. The selected mode applies to all EtherChannels configured on the switch. You configure the load-balancing and forwarding method by using the port-channel load-balance global configuration command.
With source-MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on the source-MAC address of the incoming packet. Therefore, to provide load-balancing, packets from different hosts use different ports in the channel, but packets from the same host use the same port in the channel.
With destination-MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on the destination host's MAC address of the incoming packet. Therefore, packets to the same destination are forwarded over the same port, and packets to a different destination are sent on a different port in the channel.
With source-and-destination MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on both the source and destination MAC addresses. This forwarding method, a combination source-MAC and destination-MAC address forwarding methods of load distribution, can be used if it is not clear whether source-MAC or destination-MAC address forwarding is better suited on a particular switch. With source-and-destination MAC-address forwarding, packets sent from host A to host B, host A to host C, and host C to host B could all use different ports in the channel.
With source-IP address-based forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the EtherChannel based on the source-IP address of the incoming packet. Therefore, to provide load-balancing, packets from different IP addresses use different ports in the channel, but packets from the same IP address use the same port in the channel.
With destination-IP address-based forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the EtherChannel based on the destination-IP address of the incoming packet. Therefore, to provide load-balancing, packets from the same IP source address sent to different IP destination addresses could be sent on different ports in the channel. But packets sent from different source IP addresses to the same destination IP address are always sent on the same port in the channel.
With source-and-destination IP address-based forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the EtherChannel based on both the source and destination IP addresses of the incoming packet. This forwarding method, a combination of source-IP and destination-IP address-based forwarding, can be used if it is not clear whether source-IP or destination-IP address-based forwarding is better suited on a particular switch. In this method, packets sent from the IP address A to IP address B, from IP address A to IP address C, and from IP address C to IP address B could all use different ports in the channel.
Different load-balancing methods have different advantages, and the choice of a particular load-balancing method should be based on the position of the switch in the network and the kind of traffic that needs to be load-distributed. In Figure 40-5, an EtherChannel of four workstations communicates with a router. Because the router is a single-MAC-address device, source-based forwarding on the switch EtherChannel ensures that the switch uses all available bandwidth to the router. The router is configured for destination-based forwarding because the large number of workstations ensures that the traffic is evenly distributed from the router EtherChannel.
Use the option that provides the greatest variety in your configuration. For example, if the traffic on a channel is going only to a single MAC address, using the destination-MAC address always chooses the same link in the channel. Using source addresses or IP addresses might result in better load-balancing.
Figure 40-5 Load Distribution and Forwarding Methods
The answer is:
The setting
forwarding-options {
hash-key {
family inet {
layer-4;
}
}
}
only works in PACKET MODE
The default mode for an SRX is flow mode. You can set configure packet mode by deleting ALL of the configuration under the security stanza and then running
set security forwarding-options family mpls mode packet-based
once you have committed that you will need to reboot your device.
You can check and see which mode it's running in by running the operational mode command
show security flow status
This gives the following output
Flow forwarding mode:
Inet forwarding mode: packet based
Inet6 forwarding mode: flow based
MPLS forwarding mode: packet based
ISO forwarding mode: drop
Flow trace status
Flow tracing status: off
Flow session distribution
Distribution mode: RR-based
GTP-U distribution: Disabled
Flow ipsec performance acceleration: off
Flow packet ordering
Ordering mode: Hardware
You are interested in lines 2 & 3. Below, mine show that the device is in packet mode for IPV4 traffic and flow mode for IPV6.
Inet forwarding mode: packet based
Inet6 forwarding mode: flow based
For more details see here: https://kb.juniper.net/InfoCenter/index?page=content&id=KB30461
Note: It took a JTAC engineer a few weeks to work this out after building it in a lab and testing it. He is going to try and get the documentation updated to note it only works in packet mode!
Best Answer
If your goal is to do round-robin load balancing, then I would highly discourage that because it leads to out-of-order data delivery that can actually slow things down.
Your switches will balance flows across a channel not spread a single flow across multiple channel links. A single flow will use a single link. The hash algorithm used will determine what is considered a flow. For example, if you use the
src-dst-port
hash, and the data you are sending across is only using the same source and destination ports, then you will have a single flow that uses a single link of the channel.The channel can balance with multiple flows in aggregate, allowing you to use the full channel bandwidth when you have multiple flows, but a single flow will only use a single link in the channel. This avoids problems created when spreading a single flow across multiple links.
If you want greater bandwidth for a single flow, then you need to upgrade the speed of the links.