WAN Link Speed and TCP Window Sizes on Windows 2003 / XP

Searching for "high latency file transfer" brings up a lot of interesting hits. Clearly, this is a problem that both the CompSci community and the commercial community has put thougth into.

A few commercial offerings that appear to fit the bill:

• FileCatalyst has products that can stream data over high-latency networks either using UDP or multiple TCP streams. They've got a lot of other features, too (on-the-fly compression, delta transfers, etc).

• The fasp file transfer "technology" from Aspera appears to fit the bill for what you're looking for, as well.

In the open-source world, the uftp project looks promising. You don't particularly need its multicast capabilities, but the basic idea of blasting out a file to receivers, receiving NAKs for missed blocks at the end of the transfer, and then blasting out the NAK'd blocks (lather, rinse, repeat) sounds like it would do what you need, since there's no ACK'ing (or NAK'ing) from the receiver until after the file transfer has completed once. Assuming the network is just latent, and not lossy, this might do what you need, too.

The bottleneck I see when accessing that URL is clearly due to the window size.

When I try to download from your server I get 555KB/s. I have a roundtrip time of 108ms. Doing the math I get the following window size: 555KB/s * 108ms = 59.94KB.

As long as I do it from a host in a datacenter, I get a very consistent throughput and roundtrip. Additionally, if I start two downloads in parallel each get 555KB/s. That is exactly the symptoms you will see when the bottleneck is the window size.

Without window scaling, the window cannot be any larger than 64KB. But I do see window scaling be negotiated, so higher throughput should be possible. This leaves two hypotheses to investigate:

• Something is mangling the window scaling option on the path from the client to the server making the server think the window is scaled by a factor of 1.
• The server may be configured to never use more than 60KB send window on each connection.

The first is easy to verify if you can perform a packet capture on the server. Just look at the scaling option on incoming SYN packets to find out if a scaling factor higher than one is received by the server. I can recommend using Wireshark for analyzing the traffic.

Verifying the second hypothesis requires some knowledge of the operating system you are using. You happen to have chosen an operating system, which is unknown to me, so that I cannot help with. So I can only help with expertise in networking.