Cisco – Configuring Static IP Block On Meraki Security Appliance

As you mentioned, your static routes break, because there is no existing way to force failover to the WAN link at both sites when one site looses the uplink to the internet.

The best solution to this problem is to set up some form of routed tunnel (sourced from the ISP-link addresses on R1 and R2 in the diagram) through the internet, run a dynamic routing protocol (OSPF / EIGRP) through your tunnel and backup WAN, and set the routing metrics for the WAN link to be much higher.

Since you source the tunnel from the internet links on both sides, if either side looses internet connectivity the tunnel breaks (and the IGP recoverges to your WAN link). If you do this, be sure that PMTUD works through the tunnel so the tunnel overhead is accounted for. This kind of inter-office encrypted VPN is quite common, so it's a familiar topology for anyone who has to support this network in the future.

The issue you face with your existing design is that all your inter-office traffic traffic seems to be unencrypted (at least if I read your question at face-value). If you are not already doing so, I would strongly recommend that you encrypt all traffic between your corporate sites, if it's going through the internet.

You're right, if you don't take any measures this could happen. It's a violation of the acceptable use policy of most IXP's I know, but still you want to prevent it from happening.

Your first solution is a good thing to do and will solve your problem. Just make sure you don't keep track of session state in iptables, that will probably kill performance or even your router.

You could consider to do outbound filtering as well in a similar way: do not allow packets to leave your network originating from unknown sources. This will prevent prevent hosts in your network from sending spoofed IP packets, which are commonly used in DDoS attacks.

I wouldn't implement the second solution. It's complicated and doesn't scale well if you have multiple routers handling your transits and peerings or if you have a large number of peering sessions (a couple of hunderds on an IXP isn't that uncommon).

On all the hardware router platforms I know this problem is solved in configuration by configuring RPF on the outbound interface and/or by writing filters.