Need help troubleshooting highly variable ping times

latencynetworkingping

I'm at work using Citrix (think Remote Desktop) to connect to client sites. With my job I have to write a fair bit of code while I'm connected remotely via Citrix, so the latency of my internet connection is important. If I'm getting ping times above 250ms, then it becomes almost impossible to scroll, click or type with accuracy.

Recently my Comcast business internet has been exhibiting highly variable ping times. If I ping google.com, I'll get pings that range from 9ms all the way up to 1300ms. The problem seems to be at its worst during the hours of 1PM to 4:30PM. Outside of those hours and the variance in pings settles down, mostly between 9ms and 50ms. The signal to noise ratio and upstream power are both fine on my modem–the values are here: http://pastebin.com/D4hWGPXf

I ran a trace route from my computer to google.com (the results of which are here: http://pastebin.com/GcdjYvMh) and did another test ping to the IP of the first hop outside of our local network (73.98.44.1)–the variance in ping times existed in exactly the same manner as if I were pinging Google. Connecting directly to the cable modem by CAT5 makes no difference.

Here is a screenshot demonstrating the variance of the ping times: http://postimage.org/image/haocdeauv/full/ — as you can see it can get pretty bad.

Three Comcast techs have been out (two of them were here when the problem wasn't happening) and they as well as the regional tier 2 Comcast support were unable to diagnose the problem. I now have a ticket open with tier 3 support, but have yet to hear back from them.

Does anyone know what could cause these sorts of problems or have any idea from the traceroute above where it could be originating? The regional tier 2 guy tried to tell me that what I'm seeing is normal–are highly variable ping times like that ever acceptable? Anything I should ask Comcast to do or look at to get this problem fixed?

Any tips/advice much appreciated!

Edit: This is Comcast cable internet at a small start-up, we've ruled out congestion in our private LAN as a cause (i.e., no one's watching YouTube when the pings become variable).

Update: Tier 3 Comcast support advised swapping out the modem, a tech came here today and did that–same problem persists.

EDIT: This problem was eventually resolved by Comcast, see my answer below.

Best Answer

The fact that you are seeing the similar poor response to your gateway as you see to Google suggests that the problem is fairly close to your machine. My first thought, because I've experienced very similar symptoms myself, is that the latency might be the result of over utilization, potentially on the upload side. I used to get very poor performance from interactive applications when I was uploading data. The limiting factor for the upload speed was at my cable modem, resulting in it maxing my bandwidth and killing performance for anything else.

Most residential class connections (including "business class" cable modem) tend to be heavily weighted towards download throughput, often offering something like 10Mb/s download, with 1Mb/sec upload. That makes it much easier to saturate the upload than the download, and when that happens, all traffic will suffer badly (if you're downloading, your ACK's will have issues, slowing the download, etc).

I would look for a way to monitor the bandwidth utilization, especially upstream, when the network congestion/latency occurs. Depending on your level of experience and available tools, wireshark, ntop, and your network router may all be able to provide some utilization information. If the problem does turn out to be bandwidth related, you can try moving non-interactive tasks to non-peak hours, or even look into traffic shaping and QoS options.

Related Topic