Slow backup rate for fileservers using external hdd

backuphard driveusb

I inherited a backup plan from the previous sysadmin that looks like this:

Backup each 150GB of data from two fileservers to two 3.5in external harddrive (USB 2.0) by mapping the data partitions to our quad-core Dell R200 server (Windows Server 2003) and runs ntbackup (full backup running on thursday night). Each backup takes 9 hours to complete. The external hdds will be taken home by the COO every friday (our company is closed on weekends).

I've tried to backup directly from the 2 fileservers to the USB disks, but it takes more than 24 hours to complete since each of them runs on a slow processor.

I think this plan is ridiculous (is it?), but please, my question is:

Does the backup speed is highly affected by the speed of the USB port?

Best Answer

It is. USB 2.0 is 480Mbps. Divide that by 8 to get 60Mbytes/sec, but you'll never see that in real practice due to bus contention, driver overhead, etc.

SATA1.0 is 1.5Gpbs (187.5Mbytes/sec), and since it's a dedicated point-to-point link you get to use all that bandwidth. Heck, around 2000 is when PATA UDMA-66 came out, which, at 66Mbytes/sec, matches USB 2.0.

Add to the mix that USB controllers need the CPU to do a lot of the work of transferring data, and yeah, things can be slow.

You should check out external SATA (e-SATA).

Related Solutions

Recommendations for efficient offsite remote backup solution of vm’s

This might not be possible in your situation, so I hope I don't get down-voted in that case, but it might be more efficient to change your backup strategy. If you back up specific data instead of VM snapshots, your backups would run much quicker, and it would be easier to capture changes.

Depending on your VMs and what they're used for, you can just have them back up data to where you store the snapshots now daily (or whatever schedule is appropriate), and then JungleDisk can back up just the data. That would more efficiently transfer changed files, and the space required for backups as well as time needed would be reduced. In addition, you could still take snapshots to retain, and just do that much less often (weekly, for example).

In this case, you could always just bring up a new VM and restore data, or use an older snapshot to restore the VM, and then use the data backup to restore to the most recent point.

Ubuntu – How to enforce rsync against performing checksum

Maybe the problem is in the timestamps of the files. If backups are created using a windows program it can mess the timestamps. If I replicate your sample with a bunch of pictures I obtain this log

2011/02/15 03:46:46 [61820] delta-transmission disabled for local transfer or --whole-file
2011/02/15 03:46:46 [61820] .d..t....... ./
2011/02/15 03:46:46 [61820] IMG_0055.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0056.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0057.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0058.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0059.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0060.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0061.JPG is uptodate
2011/02/15 03:46:46 [61820] >f..t....... IMG_0062.JPG
2011/02/15 03:46:46 [61820] IMG_0063.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0064.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0065.JPG is uptodate
2011/02/15 03:46:46 [61820] IMG_0066.JPG is uptodate
2011/02/15 03:46:46 [61820] total: matches=0  hash_hits=0  false_alarms=0 data=5911343
2011/02/15 03:46:46 [61820] sent 5912367 bytes  received 67 bytes  11824868.00 bytes/sec
2011/02/15 03:46:46 [61820] total size is 75450221  speedup is 12.76

So the files are tranferred each time otherwise you should have the log message file xxx is uptodate. Try to increase rsync verbosity with more than one v flag to find out what's happening.

Best Answer

Related Solutions

Recommendations for efficient offsite remote backup solution of vm’s

Ubuntu – How to enforce rsync against performing checksum

Related Topic