Cisco – Network Layout and Design

You have an elegant physical setup, but logically, it’s deficient in a lot of ways. You’re relying on layer 2 for all of your connections, which doesn’t provide any real failover your setup should provide. Luckily, you aren’t wasting any bandwidth with STP because you don’t have any redundant links; VLAN 100 and VLAN 200 only have a single way out, if I’m reading this correctly.

I can't implement multiple area OSPF because the CPE public subnets are too difficult to summarize and the public range is split over several POPs without hierarchy, so this would be inefficient.

I don’t even think you need to go as far as setting up multiple areas, the segment your concerned with is fairly small. Cisco recommends you stay under 50 routers per area, though, most will agree that you can exceed that amount by quite a significant margin and still be considered healthy. I have about 80 routers per area with no issues; Ron has even more than that without any hiccups.

• Implementation of EtherChannel between the access switches and CMTS.

This all hinges off what happens higher up in your network. If you’re significantly oversubscribing your distribution to core links, then you may not see any real benefit in increasing your bandwidth at the access layer. The same holds true if you’re doing this with your access to distribution.

Again, as most have mentioned in the comments, implementing OSPF would be pretty easy. If you were utilizing a layer 3 protocol, you would share that load across equal/unequal cost links, too. Perhaps a redundant link between your distribution switches would allow each access switch a redundant path out. As it stands, if one of your access links goes down, you lose an entire subnet (VLAN 100 or VLAN 200).

If you are running EIGRP over the leased line, then the routes over that link will have an AD of 90. If the routes from the PE come to the CE via iBGP, then the AD will be 200 for those. The routes received from EIGRP will naturally be preferred over those received from iBGP. That means that your scenario will work out without any changes or routing policies, and the network should fail over from the leased line to the MPLS connection automatically, and go back when EIGRP is reestablished once the leased line comes back up.

The same holds true for OSPF (AD 110) and iBGP (AD 200). The only hitch would be with eBGP (different ASes) which has an AD of 20 (lower than other routing protocols).

If, on the other hand, you are running EIGRP between the CE and PE as is shown in the drawing, then you can decrease the bandwidth or increase the delay (really only labels) on the CE interface toward the PE, and you will then prefer the routes received from the leased line.