GRE Tunnel – Using as Default Route on OpenWrt

It's kind of late, but if you didn't solve it, here is how:

Given your example just replace:

• publicip: 8.8.8.8 with 50.2.2.2
• publicip: 7.7.7.7 with 70.2.2.3
• private ip, could be any valid private address.
• privateip: 10.0.201.1 and 10.0.201.2

Public ips:

• A = 50.2.2.2-22
• B = 70.2.2.3

tunneling with gre:

On A:

# adding the interface for the tunnel
$ ip tunnel add tun2 mode gre remote 70.2.2.3 ttl 64

# setting the private ip address 
$ ifconfig tun2 10.0.201.1/24
$ ifconfig tun2 up

# A point to point 
$ ifconfig tun2 pointopoint 10.0.201.2

# enabling multicast (it's not necessary for this)
$ ifconfig tun2 multicast

$ ifconfig tun2 arp
$ ifconfig tun2 broadcast

# default route for the tunnel
$ ip route add 10.0.201.2 dev tun2 

# enable ip forward
$ echo 1 > /proc/sys/net/ipv4/ip_forward

# add the permanent entries to the arp table in order to get the complete loop. (without this doesn't work)
# replace the public ips for your ips, and the mac for your real mac for your interface
# the word pub it's the most important here, if it's not there the arps will never go outside
$ arp -s 50.2.2.20 00:00:00:00:00:00 -i eth0 pub
$ arp -s 50.2.2.21 00:00:00:00:00:00 -i eth0 pub
$ arp -s 50.2.2.22 00:00:00:00:00:00 -i eth0 pub

On B:

# adding the interface for the tunnel
$ ip tunnel add tun2 mode gre remote 50.2.2.2 ttl 64

# setting the private ip address 
$ ifconfig tun2 10.0.201.2/24
$ ifconfig tun2 up

# point to point B
$ ifconfig tun2 pointopoint 10.0.201.1

# enabling multicast (it's not necessary for this)
$ ifconfig tun2 multicast

$ ifconfig tun2 arp
$ ifconfig tun2 broadcast

# default route for the tunnel
$ ip route add 10.0.201.1 dev tun2 

$ echo 1 > /proc/sys/net/ipv4/ip_forward

# putting the ips to listen in the eth0 as secondary ips
$ ip ad add 50.2.2.20/32 dev eth0
$ ip ad add 50.2.2.21/32 dev eth0
$ ip ad add 50.2.2.22/32 dev eth0

And that's it, you should have a fully functional tunnel and the ability to route ips that are far away from were you want to use them, so you can now start to bind some daemons to those IPs.

Another thing to have in mind is that if you have so many IPs, you've to be careful with your broadcast domain on point A, and if you're planning to tunnel more than 500 IPs, then you've to change the default values of Linux for the arp table in order to keep all entries:

$ echo 1024 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
$ echo 4096 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
$ echo 16384 > /proc/sys/net/ipv4/neigh/default/gc_thresh3

Sources:

• http://www.lartc.org/lartc.html
• http://linux-ip.net/gl/ip-tunnels/node9.html
• http://yurisk.info/2009/12/15/arp-table-overflow-in-checkpoint-nad-linux-in-general/
• http://kainlite.blogspot.com.ar/2012/05/ip-tunneling-on-linux-lets-suppose-that.html

I was looking for the same a long time ago and found your post.

Routes are processed from the most specific route to the least specific (aka default) route.

default via 10.31.96.1 dev em3  proto static 
10.0.0.0/8 dev em1  proto kernel  scope link  src 10.0.0.100 
10.31.96.0/22 dev em3  proto kernel  scope link  src 10.31.97.100 
10.31.96.0/22 dev em4  proto kernel  scope link  src 10.31.96.61

You said you want should be going out through em3 unless its destined for something else on the local 10.0.0.1/8 subnet. This is exactly what is happening. The IP address 10.31.45.1 is within 10.0.0.0/8 and so it is leaving via em1. The 10.0.0.0/8 route matches that address is more specific then the default route. The address doesn't match the the 10.31.96.0/22 route. Therefore the em1 route is selected.

Your real problem is that you have a subnetmask on that em1 interface that is far too large for what you probably need, and it conflicts with the other networks. Anything destined for a IP address in the 10.0.0.1-10.255.255.254 range will attempt to use em1 as if it was local, which the exception of addresses in the 10.31.96.0/22 which will leave via em3/em4.

Your solution is to either fix the em1 subnet/network so that it doesn't conflict with your other networks, or to add lots of routes.

Something like ip route add 10.31.45.0/24 via 10.31.96.1 might do what you want.