Route Servers – What are Route Servers and Looking Glasses?

bgppeeringroute-serverterminology

I've heard the term "route server" and "looking glass" thrown around here. What are they, and why should I care?

Best Answer

(Note that these two terms are often used interchangeably, which can lead to some confusion.)

Looking Glasses

A looking glass is usually a website (most often CGI) that interfaces with routers that are owned and operated by a single ISP or other network operator. Most of the time these are publicly accessible, but there may be instances where they're not. The looking glass provides a view (in website form) into a BGP table of a particular router in an operator's network. Often, looking glass implementations will also include other utilities, such as the ability to run a traceroute to a destination as if it were run from the network operator's router itself. Looking glasses are useful because they provide a perspective into an upstream's BGP table. Level3, a well-known tier one carrier in the U.S. has a looking glass available here. These are widely used troubleshooting tools.

Route Servers

The term "route server" has evolved to mean two different things, both of which will be explained. I'm defining two route server "subtypes" here that should make the distinction more clear. These definitions are my own and are only used to try to dispel any possible confusion. In reality, the public route server is also commonly referred to as a "route collector" or a "route-views server" (the latter originating from the University of Oregon Route Views Project).

Public Route Server

These are systems (usually routers, but some exist that run open source routing daemons) that are publicly accessible, often via Telnet, and one is able to also run pings, traceroutes, and "show ip bgp" (there are also a couple Juniper route servers out there, don't worry!) commands. The difference between a public route server and a looking glass (aside from an LG having a nifty CGI) is that a wide variety of networks (including some tier one carriers) peer with a route server, so there's an "overall" bigger picture of which prefixes are coming from which ASN's. Command authorization policy will vary from route server to route server. Here's a list of IPv4 route servers. They have a separate page with IPv6 route servers as well. When it comes to differentiating looking glass from route server, one can think of the looking glass as a frontend to a route server, whether the route server itself be publicly or privately accessible.

Can't you run a looking glass on a public route server?

You can, but typically the people that run looking glasses are the ones that can afford to operate and maintain the servers that the looking glasses run on. With a public route server, all you need is a router (or a server running an open source routing daemon), a good AAA policy, and some folks that are willing to give you BGP feeds. It's also relevant to note that some network operators also host publicly accessible route servers, and you might even come across some operators that run a route server in addition to a looking glass.

IXP Route Server

This version of the route server is a little bit different. On shared peering fabrics at IXP's, the barrier of entry for an organization to begin peering is lower. You have a single port (that the IXP sells to you) connected to the IXP peering LAN, and you're given an IP address by the IXP. Now you can peer with anyone else on the fabric; contrast this with a PNI, which involves a separate dedicated physical connection between you and another entity. With a connection to an IXP's peering LAN, aside from your single port being the bottleneck, you have to manually configure eBGP sessions with whomever you wish to peer with. This is called bilateral interconnection - there is one session between you and the peer, and only you and the peer exchange announcements. If the IXP has many members, this can become cumbersome. In this case, a route server is typically deployed at the IXP in order to be the "one stop shop" for an organization to set up an eBGP session with in order to receive prefixes from whomever else is peered with the route server. This is called multilateral interconnection. One BGP session between you and the route server, and you get everyone else's announcements that also peers with the route server.

Some organizations will rely on the route server eBGP session(s), while others will use it as a backup to their existing eBGP peerings on the IXP fabric. Most IXP's will have redundant route servers and suggest that organizations peer with both if they peer with the route servers at all.

What about BGP?

Multilateral peering using route servers brings interesting challenges in regards to BGP. A route server itself is an eBGP speaker, however there have to be specific considerations for a route server and multilateral BGP peering. Some of these rules will be reminisce of iBGP route reflection, and there are indeed many similarities. There has been recent work to standardize behaviors of a route server when it comes to these specific features. The following caveats are worth noting here:

  • NEXT_HOP Attribute - This attribute must be passed on unmodified between the route server and its clients. While the route server implementation itself will not modify this attribute, it's still best practice to set "next-hop-self" on eBGP sessions from your router to a route server.
  • AS_PATH Attribute - Again, because the route server is intended to be transparent and not participate in any routing decisions, and modification of this attribute may affect the route server clients' best path decision making process, the route server implementation must not modify this attribute. In addition, the route server will not send their own ASN in BGP updates to its clients, therefore it's required to set "no bgp enforce-first-as" (Cisco-specific) on the client router in order to allow the eBGP session to form between the client and the route server.
  • MULTI_EXIT_DISC Attribute - MED is another attribute that should be propagated to route server clients without modification by the route server, since it can also used to influence the best path selection process.
  • Communities Attributes - These should not be modified by the route server, unless the community (or communities) is intended for processing by the route server itself. Typically communities are used by IXP route server implementations to allow the peers of the route server to manipulate routing updates to other route server peers.

It is important to remember two distinctions in regards to IXP route server implementation:

  • The route server does not participate in any routing or forwarding. It is supposed to be as transparent as possible.
  • Route server clients depend on the route server to perform their outbound filtering for them.

IXP Route Server Implementation Options

Typically IXP operators will deploy open source routing daemons as route servers. There are three popular options:

  1. Quagga, specifically bgpd. Runs on Linux and BSD. It's been around the longest and probably has the most information available. There have been multiple forks of Quagga over the years, including a train with sponsored development by EuroIX, a train developed by the Open Source Routing group (which more aims to improve IGP functionality with OSPF and ISIS), and the Quagga-RE (Release Engineering) train which has experimental features. Google has also created their own fork of Quagga. Quagga bgpd supports both IPv6 and IPv4, however most IXP operators (and even some Quagga maintainers) will advise against using the "mainline" Quagga train for operating a route server at an IXP.

  2. BIRD. Runs on Linux and BSD. BIRD has gained quite a bit of popularity due to its stability and powerful filtering language and capabilities, in addition to its very good scheduling system. There have been a couple of published comparisons and scale tests between Quagga and BIRD. Overall, BIRD tends to be more stable and isn't as susceptible to CPU and memory churn as Quagga is. Both BIRD and Quagga are single-threaded however, but this is intentional. In addition, although BIRD supports both IPv4 and IPv6, it requires two different processes (compiled binaries) for IPv4 and IPv6.

  3. OpenBGPD. BSD Only. OpenBGPD is the only multi-threaded open source BGP daemon available. It's known to be fairly stable under load however TCP MD5 support is somewhat poor.

In addition to open source daemons, Cisco also offers a route server implementation for IOS-XE, which runs on the ASR platform. At the time of this writing, Juniper doesn't offer a route server implementation on their hardware or operating system, but this may change in the future.

In terms of evaluating an open source routing daemon, in regards to memory handling and architecture, one can safely assume that BIRD > OpenBGPD > Quagga, however the ASR platform and IOS-XE is also reportedly much more capable at scale when compared to the open source daemon options available.

I hope this sheds some light on route servers/looking glasses in general.