Cisco – Calculating Cisco router memory requirements for full BGP feeds

Yes, you did hit the nail on the head.

You will get asymmetry in the improved design, but asymmetry is a fact of life on the Internet, and there's really no good reason to expect symmetric routing of traffic to/from. Shoot, the whole concept of packet routing is that separate packets are routed independently of each other and may take different paths, even packets going in the same direction.

Personally, I loath PBR. Its one of those technologies that when I decide that its the best solution to the problem, I stop and take a step back to see if I really understand the real nature of the problem, even back to figuring out what the business problem to be solved is. When I do so, I almost always find that there is a way to solve the problem without using a technology like that.

Having full Internet routes in your routers will take some getting used to, but once you get used to it, it is indeed very easy to understand and troubleshoot. Certainly there are fewer "moving parts" of different protocols to worry about.

You don't want to have full Internet routes in your OSPF database, so you'll want to advertise a default via OSPF into the interior of your network (or perhaps static default...personally I prefer default in OSPF). That will move traffic towards the BGP speaking Internet routers which can make the more fully informed decision of having the full Internet routes.

That will give you close to "destination based best path". There will still be cases where the traffic will do things you don't quite expect, so you'll want to get familiar with the BGP route selection process.

Just like BGP attributes are packed when sending BGP UPDATEs, they are stored in memory in a rather compact format where each prefixes only holds references to the attributes that apply to that route. The AS path and the communities applied to a route are typically the attributes that are largest in size and as these are often the same over a large amount of routes, a great deal of memory can be saved by just storing an AS path once and then keeping references to all routes for which this AS path applies. The exact in-memory format changes with different versions of IOS/JUNOS.

A route for the same prefix received via different eBGP sessions will likely share many attributes (especially if the eBGP sessions are to the same upstream) and so they can be "packed" quite efficiently in memory.

Your explanation of BGP behaviour is mostly correct. R1 has sent the content of its entire BGP RIB to R2. R2s eBGP session is enablde and for every route received, R2 will compare it to the one in its BGP RIB (which mostly contains routes from R1). When R2 finds a prefix from its new eBGP peer that is better than the one in BGP RIB, it will use the new eBGP route and announce it to R1. If R1 finds the route received from R2 to be better than its own route (received from its eBGP session) then R2 will withdraw the route that it announced to R2. The routers will only announce their best route and if the best route is not from eBGP but from another iBGP neighbor, it will not announce it unless you have some route-reflection going on as well. You can easily end up in a scenario where both R1 and R2 each select all the routes learnt from their respective eBGP session as the best routes and that means they will always announce a full table to their iBGP buddies.

There's also something called "best external" which means you always announce your best eBGP route regardless if that's the one you have chosen as best path or not. Best external enhances convergence time by not having to go through "BGP path hunting" when something breaks. Obviously it consumes more memory.

Personally I don't think BGP Soft reset enhancement is much of an enhancement. RAM is cheap, so keeping the routes, exactly as received from your neighbor, in memory is quite easy, especially on a modern router. JUNOS doesn't even offer you the option to not store it - it always keeps the Adj-RIB-IN, but JUNOS was designed in the late 90s when RAM was plentiful and not in the 80s when IOS made the scene. If someone out there designing a router thinks "let's save on RAM because we have BGP soft reset enhancement", I'd like to shoot that person ;) On the other hand, if you are running a network with old routers running low on memory I totally understand if you don't enable soft-reconf.

Bottom line, it's difficult to say and depends on your environment.