Why single file transfer is not faster than 50MB/s over Gigabit LAN

file-sharinggigabit-ethernetlocal-area-networknetworkingwindows 7

I ran some tests on our gigabit network.

When I transfer a big file (10GB) from a windows 7 computer to another, the troughtput never exceed 50Mb (it start at 80MB/s for one second then slows down to 32MB/s).

Using simultaneous transfers from several computers, the client can maxout its Gb connection.
Idem, when using several clients at the same time, the source computer reach near Gb troughput.

The tests were done on very fast drives (2 velociraptor in Raid 0, 4 WD red on LSI megaraid RAID5, SSD, Revodrive X3…). So they are not the bottleneck.

When I transfer from a windows server, the troughput is slightly better but far from the expected 112MB/s of a gibit LAN.

Any idea ?

Edit after strange walker comment : We are using integrated NIC (intel) on ASUS motherboards and DELL Workstation. Switches are Manageable Cisco SG 300-20, SF 300-08 and Netgear GS 108.

Best Answer

I'm guessing that you are using TCP/IP to transfer the file, although you also neglected to mention what protocol you are using.

The answer to this would fill a large book - and there are lots of them available. But amongst the possible reasons are:

  1. congestion control: your network stack throttles throughput, gradually increasing the rate over the lifetime of the connection (until it encounters packet loss at which point the rate is dropped massively). Using jumbo packets helps too.

  2. window size : a TCP/IP end point will only send as much data as it thinks the remote system can buffer. The receiving end must be able to buffer at least the bandwidth x the RTT. Although the RTT is low here, the bandwidth is big. When IPV4 was designed it did not expect to have to cope with such large buffers. There is a solution which is to enable window scaling.

  3. Your computer is not dedicated to the purpose of sending data - it multi-tasks. To process the data the NIC generates an interrupt - this adds latency. You can mitigate the effects of this on good network cards by enabling buffering on the card (interrupt coalesce)

  4. there is still a significant amount of work for the computer to do in wrapping data up into frames and squirting them down the wire - again this can be mitigated - in this case by using TCP offload if your OS/hardware supports it.

....and that's before looking at the peculiarities of Ethernet and/or the protocol you are using to transfer the files.

Related Topic