Slow File Copy observed copying 40GB files across network to iSCSI device

filesiscsiwindows-server-2003windows-server-2008

Here's a curious ones for the gurus:

Setup:

Source Machine: Windows Server 2003 R2 machine with local hard drive. VHD file of 40GB. 1 x 1Gbps network card, Cat6 cable, switch.

Target Machine: Windows Server 2008 R2 machine with iSCSI connection to iSCSI target on separate machine (1TB, RAID5). 1 x 1Gbps network card, Cat6 cable, connected to same switch as for Source Machine. Second 1Gbps network card, Cat6 cable, connected via isolated switch to the iSCSI target.

Switches are Netgear JGS524 model (web managed).

If I copy from the Win2003R2 machine to Win2008R2 machine local drive I get 40GB in 45 minutes, 36 seconds.

If I copy from the Win2008R2 machine to the iSCSI target (local drive to iSCSI target) I get 40GB in 37 minutes 56 seconds.

If I copy from the Win2003R2 machine to the iSCSI target via the Win2008R2 machine I get 40GB in 3 hours, 50 minutes, 24 seconds.

All copies were done via the following command issued on the Win2008R2 box:

XCOPY <source> <target> /J

XCOPY /J – Copies using unbuffered I/O. Recommended for very large files.

So, what's the bit I'm missing here? Why does a back-to-back copy take in total 1 hour, 23 minutes, 32 seconds when a "straight through" copy take almost 3 times as long?

Switches show no errors, network hovers around the 3% utilisation mark for the duration of the copy (whereas the "back-to-back" copies are around the 25% utilisation mark).

What have I missed?

Best Answer

Could it be the 'unbuffered' copy the problem? It is possible that Windows does some tricks that can speed up the copy the the source/target is a local disk, but it reverts to a safer behaviour if it is using two net devices.

I have played with disk testing in Unix, and the OSes can play lots of tricks with the disk subsystem. Good luck.