Networking – Typical Network Latency for East to West Coast USA

hostinginternetlatencynetworking

At the moment we're trying to decide whether to move our datacenter from the west coast to the east coast.

However, I am seeing some disturbing latency numbers from my west coast location to the east coast. Here's a sample result, retrieving a small .png logo file in Google Chrome and using the dev tools to see how long the request takes:

  • West coast to east coast:
    215 ms latency, 46 ms transfer time, 261 ms total
  • West coast to west coast:
    114 ms latency, 41 ms transfer time, 155 ms total

It makes sense that Corvallis, OR is geographically closer to my location in Berkeley, CA so I expect the connection to be a bit faster.. but I'm seeing an increase in latency of +100ms when I perform the same test to the NYC server. That seems .. excessive to me. Particularly since the time spent transferring the actual data only increased 10%, yet the latency increased 100%!

That feels… wrong… to me.

I found a few links here that were helpful (through Google no less!) …

… but nothing authoritative.

So, is this normal? It doesn't feel normal. What is the "typical" latency I should expect when moving network packets from the east coast <–> west coast of the USA?

Best Answer

Speed of Light:
You are not going beat the speed of light as an interesting academic point. This link works out Stanford to Boston at ~40ms best possible time. When this person did the calculation he decided the internet operates at about "within a factor of two of the speed of light", so there is about ~85ms transfer time.

TCP Window Size:
If you are having transfer speed issues you may need to increase the receiving window tcp size. You might also need to enable window scaling if this is a high bandwidth connection with high latency (Called a "Long Fat Pipe"). So if you are transferring a large file, you need to have a big enough receiving window to fill the pipe without having to wait for window updates. I went into some detail on how to calculate that in my answer Tuning an Elephant.

Geography and Latency:
A failing point of some CDNs (Content Distribtuion Networks) is that they equate latency and geography. Google did a lot of research with their network and found flaws in this, they published the results in the white paper Moving Beyond End-to-End Path Information to Optimize CDN Performance:

First, even though most clients are served by a geographically nearby CDN node, a sizeable fraction of clients experience latencies several tens of milliseconds higher than other clients in the same region. Second, we find that queueing delays often override the benefits of a client interacting with a nearby server.

BGP Peerings:
Also if you start to study BGP (core internet routing protocol) and how ISPs choose peerings, you will find it is often more about finances and politics, so you might not always get the 'best' route to certain geographic locations depending on your ISP. You can look at how your IP is connected to other ISPs (Autonomous Systems) using a looking glass router. You can also use a special whois service:

whois -h v4-peer.whois.cymru.com "69.59.196.212"
PEER_AS | IP               | AS Name
25899   | 69.59.196.212    | LSNET - LS Networks
32869   | 69.59.196.212    | SILVERSTAR-NET - Silver Star Telecom, LLC

It also fun to explore these as peerings with a gui tool like linkrank, it gives you a picture of the internet around you.

Related Topic