Bulk file copy while limiting disk utilization

backupfilesrobocopywindows 7

I saw that Robocopy can limit network bandwidth in this answer How do you limit the bandwidth for a file copy?

I have plenty of bandwidth on the LAN, but I don't want to saturate the disk on the system originating the copy (Windows 7).

I vaguely remember reading that Win7 has some facilities for disk prioritization and/or throttling.
Is there a version of RoboCopy that takes advantage of those OS features?

Or is there a different way to limit the disk impact of a large file copy job?

Best Answer

See this question over at Superuser on the topic of setting I/O priorities for a process. Basically, set the CPU priority of your process to "Idle" to also set the I/O priority to "very low".

There usually is only little value in setting a bandwidth limit for limiting disk I/O since disks are capable of delivering a high sustained throughput on contiguous data but would have great difficulties resulting in high latencies and low throughput with random I/O.

Related Solutions

Windows 7 extremely slow login, exchange performance, printer enumeration, etc

This problem is apparently rooted in the fact that we have multi-homed domain controllers. Kudos to @Bryan for sniffing out the DNS aspect of it. The symptoms are explained by our DNS reporting all available IP addresses for the DC's back to the Windows 7 client. The Windows 7 client is apparently interpreting this as a DNS round-robin situation. If it happens to get an IP its subnet, it works ok. If it gets an IP on different, or un-routable subnet, it fails. It looks like WinXP and Vista correctly pick the IP on the local subnet first, and ignore the rest of the IPs.

Our IT staff is currently working around this by setting up limited host files for the multi-homed DC's, but I don't think that will be the permanent solution.

Microsoft has various things to say about this here:

It's been mentioned on a couple of forums here:

File copy between two servers programatically

rsync, scp. Either one will work, and either are trivial to use "programmatically".

For instance, here's a shell script to copy some files from server-01 to server-02:

(this assumes that key auth is already configured between these servers)

#!/bin/sh

scp -R /path/to/files user@server-01:/path/to/destination

...and an rsync example:

#!/bin/sh

rsync -az /path/to/files user@server-01:/path/to/destination

Best Answer

Related Solutions

Windows 7 extremely slow login, exchange performance, printer enumeration, etc

File copy between two servers programatically

Related Topic