We had this exact same problem. Just disabling TCP timestamps solved the problem.
sysctl -w net.ipv4.tcp_timestamps=0
To make this change permanent, make an entry in /etc/sysctl.conf.
Be very careful about disabling the TCP Window Scale option. This option is important for providing maximum performance over the internet. Someone with a 10 megabit/sec connection will have a suboptimal transfer if the round trip time (basically same as ping) is more than 55 ms.
We really noticed this problem when there were multiple devices behind the same NAT. I suspect that the server might have been confused seeing timestamps from Android devices and OSX machines at the same time since they put completely different values in the timestamp fields.
"Does the box really need TCP Window Scaling?"
Well, you didn't really say what the box does, so it's kind of hard to say really. But in general, TCP Window Scaling is critical for good end user performance on modern WAN/Internet connections. It's less needed on LAN.
Is networking performance degraded? Latency does not bother me, but throughput would.
It depends on your users' network connections. But if some of your users are
- physically far away from you (i.e. have high latency), and
- have fast network connections (i.e. good DSL, fiber, etc)
... then without Window Scaling, these users will only be able to use a small fraction of their connection speed when downloading from you. Here is a good tutorial on bandwith-delay product and window sizes -- complete with nice animated cartoons.
"Enabling High Performance Data Transfers" is a classical document about TCP window scaling. It correctly mentions that if you're using a recent Linux kernel in the 2.6 series, then you normally don't need to tweak the TCP settings, because Linux now has agressive auto-tuning of these. It fails to mention that Windows 2008+ also auto-tunes TCP settings well, using Compound TCP.
Update:
the large files are sent at a regulated bit rate (~2Mbps) clients are on the Internet and 90% within 250km
With this updated info, it gets trickier. Since you are capping the maximum speed, maybe TCP windows are not a limitation in your case. Have a look at what kind of connections your end users are using, and do the math. You don't need to run benchmarks, you can calculate the required window size, and compare it to your servers actual default transmit window size.
Best Answer
It's "fine" to turn off both, but there might be drawbacks.
Window Scaling allows the scaling of the advertised window to values greater than 64 kilobytes. If your path has a bandwidth delay product in either direction (smallest bandwidth times the round trip time) larger than 64 kilobytes, you will need to use Window scaling to be able to reach higher bitrates.
Timestamps can help for example in some cases against spurious retransmissions or sequence number wraparound for example. Usually not critical, but as always "it depends.." :p