Linux – How to Add a Route to a Prefix via Specific Device with Default Route for Certain Destinations

iproute2ipsetlinuxroutingvpn

I'm currently dealing with a VPN which connection endpoint lies within the subnet which prefix shall be tunneled via that specific VPN.

Essentially the problem thus boils down to match against a (larger) set of destination addresses (/16 mask), except for a wholly contained (small) subset of specific destinations that shall not routed that way (and instead go via the default route).

If the problem was about filtering, in Linux using this could be implemented using a ipset set up like

ipset create MyVPN hash:net
ipset add MyVPN $MYVPNNET/$MYVPNMASK
ipset add MyVPN $MYVPNENDPOINT nomatch

However such ipsets can only be used inside netfilter.

Now my question is, how I could set up something equivalent using Linux Advanced IP Routing, i.e. the ip route family of commands?

Best Answer

As it turns out, in Linux one can use ipset matching to select routing tables. The extra ingredient is using a netfilter rule that will match the outgoing packets and apply a fwmark to them, which then can be used to select a dedicated routing table. This dedicated routing table then will contain just a single default route through the VPN.

Here's in script form the general idea, how to set this up.

# Subnet(s) to route through the TUN

DSTNETS=.../.. .../..

# Exclude these destinations from routing through the TUN

EXCLUDE=... ...

# TUN device to use for this connection, and its parameters Those usually are supplied by the VPN daemon

TUNDEV=...
TUNADDR=.../..
TUNPEER=...

# Name for the ipset used to match destinations. Base on the TUN name

IPSET=${TUNDEV}ipset

# We need a netfilter fwmark and a iproute2 table. fwmarks and tables are 32 bit values, limited to the signed integer range, i.e. [0, 2³¹-1] fwmarks may be used as bitmasks signalling multiple flags, so depending on our needs either set a single particular bit (or few), or a very specific value that is then compared for equality.

FWMARK=0x...
TABLE=0x...

# Create the ipset and populate it with the IP address ranges and subnets to match to and not to.

ipset create $IPSET hash:net
for d in $DSTNETS ; do ipset add $IPSET $d ; done
for x in $EXCLUDE ; do ipset add $IPSET $x nomatch ; done

# Create a new iproute2 ruletable that will be used for route lookup for all packets that have our fwmark of choice set

ip rule add fwmark $FWMARK table $TABLE

# This is where the magic happens: Create a netfilter rule that will match packets originating on this host, for destinations that match the ipset we just created and mark them with our chosen fwmark. Due to the rule we just created before, those packets will then be routed using that specific table, instead the global ones.

iptables -t mangle -A OUTPUT -m set --match-set $IPSET dst -j MARK --set-mark $FWMARK

# Also add a netfilter rule to overwrite the source address for these packets since the destination network will likely reject them, if they don't match the address range used for the VPN

iptables -t nat -A POSTROUTING -m set --match-set $IPSET dst -j SNAT --to-source $INTERNAL_IP4_ADDRESS

# Now we can set up the actual TUN device. Strictly speaking those steps could have been done before, but then for a short time between the TUN coming up and establishing the routing rules some packets might have ended up in limbo

ip link set dev $TUNDEV up
ip addr add $TUNADDR peer $TUNPEER dev $TUNDEV

# Finally establish a default route going through the TUN in our dedicated routing table which due to the fwmark rule will be hit only by packets matching to our ipset

ip route add default dev $TUNDEV table $TABLE

To tear everthing down, just follow the script in reverse, deleting stuff; the ipset can be destroyed with a single command.

Related Topic