Linux – Multipath routing in post-3.6 kernels

linuxrouting

As you all probably know, ipv4 route cache has been removed in 3.6 Linux kernel series, which had serious impact on multipath routing. IPv4 routing code (unlike IPv6 one) selects next hop in a round-robin fashion, so packets from given source IP to given destination IP don't always go via the same next hop. Before 3.6 the routing cache was correcting that situation, as next hop, once selected, was staying in the cache, and all further packets from the same source to the same destination were going through that next hop. Now next hop is re-selected for each packet, which leads to strange things: with 2 equal cost default routes in the routing table, each pointing to one internet provider, I can't even establish TCP connection, because initial SYN and final ACK go via different routes, and because of NAT on each path they arrive to destination as packets from different sources.

Is there any relatively easy way to restore normal behaviour of multipath routing, so that next hop is selected per-flow rather than per-packet? Are there patches around to make IPv4 next hop selection hash-based, like it is for IPv6? Or how do you all deal with it?

Best Answer

If possible, upgrade to Linux Kernel >= 4.4 ....

Hash-based multipath routing has been introduced, which in many ways is better than the pre 3.6 behaviour. It is flow-based, taking a hash of the source and destination IPs (ports are ignored) to keep the path steady for individual connections. One downside is that I believe there were various algorithms/config modes available pre 3.6, but now you get what you're given!. You can use affect the choice of path by weight though.

If you are in my situation then you actually want the 3.6 >= behaviour < 4.4 but it is no longer supported.

If you do upgrade to >= 4.4 then this should do the trick, without all the other commands:

ip route add default  proto static scope global \
nexthop  via <gw_1> weight 1 \
nexthop  via <gw_2> weight 1

Alternatively by device:

ip route add default  proto static scope global \
 nexthop  dev <if_1> weight 1 \
 nexthop  dev <if_2> weight 1
Related Topic