I'm setting up a second site for redundancy and DR.
The goals are as follows:
- ISP fault-tolerance from either site: HostA, for example, can reach the internet in the event that ISP-A(primary) and ISP-B go down.
- Similarly, servers will continue to be accessible from the internet in the event that 1 or 2 connections are lost.
Notes:
- We already have Router-A connected via BGP among two ISPs. Either ISP can go down and we can continue to function, so assume that the initial BGP/ARIN steps are completed.
- EDIT: sites interconnected via 1 or more 1gbps private lines
- EDIT: Routers are also firewalls in this case
My questions:
- How do I handle the default gateway situation?
- Do I point clients in their respective sites at their respective router? What if that router loses connectivity?
- What about asynchronous routing? Say ISP-A is our primary and traffic comes in destined for a server on Site B (who is pointed at Router-B for default GW), won't that cause issues? Ideally we wouldn't have such a situation even if it means designating one router as default GW for all and have it route to the second router in the event it loses it's ISP connectivity.
- I was thinking I needed something like OSPF to propagate the awareness of ISP link failures, but I'm not clear on what it would help.
Thanks.
Best Answer
This might be a bit long winded, but I'll try to address most of your issues in one go. Wish me luck!
You can have your upstream service providers advertise a default route (0.0.0.0) through BGP to your routers. When you configure OSPF on your BGP routers, you can use the command "default-information originate." As long as you're getting the default route from your ISP (which only happens when you have connectivity to that ISP), you'll advertise the default route into OSPF. As soon as you lose that default route from your ISP, you'll lose the OSPF default route.
You'll need to establish BGP peering between all of your internet routers (the ones connected to your ISPs). This is an internal BGP or iBGP relationship and needs to be a full mesh. Per your diagram, you only have two internet routers, so a full mesh is simple. There are no tricky steps to peer on iBGP versus eBGP; just a simple neighbor statement. When BGP sees that your neighbor has the same AS number, it forms an iBGP relationship instead of an eBGP one.
Preferring one route over another is a bit tricky, particularly where BGP is concerned. Load balancing is done using route maps and traditionally with as-path prepending. There are a few ways to do this. I'd like to add a caveat that if you aren't filtering outgoing route advertisements, you'll end up advertising ISP A's routes to ISP B and C and so forth which will turn you into a transit AS and you'll end up piping some A <-> B, A <-> C, and B <-> C traffic through your network which is probably not what you want.
Here's one go at your load balancing:
If you have more than one subnet, you can even things out a little more and use router B for site B and router A for site A by selectively prepending your AS path to individual subnets. Here's an example:
Router A:
Router B:
What you've effectively done is make site B's subnets look less appealing coming from Router A than they do from ISP C and make site A's subnets look less appealing coming from Router B than they do from ISP A/B. You might have to play around with your AS path prepending some to get the right amount of prepending in.
I hope this helps! BGP is a bit of a monster, but once you understand the parts, it's fun to play with. I highly recommend the CBT Nuggets series on BGP if you feel a bit shaky on the subject and I always use GNS3 as a testbed to verify big network changes before I implement them.