Linux – BackupPC Taking over 72 hours to backup 250GB’s

backupbackuppclinuxrsynctar

I am using BackupPC to backup some workstations at the office. One workstations in particular has a sizable amount of data when compared to the rest, however, it's not that large in general (talking about 250ish GB's of actual data).

BackupPC seems to take forever to back this system up (several days, ie. more than 72 hours+). All workstations are being backed up via rsync over an autofs local mount over the network.

Basically, autofs mounts the administrative C share on the workstation, then BackupPC treats it as a local directory, cd's into the automount directory, and rsync's all the data.

It is slow in general, and I generally attributed it to the BackupPC box being slow hardware, but this performance is more-or-less acceptable on all workstations except this one with the larger amount of data.

rsync flags:

/usr/bin/rsync --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive

These are the default args that BackupPC is setup with.

I read some stuff online that indicated atime on the mounts may be slowing things down — so I changed my autofs config to mount the directories with the noatime flag… no difference.

I read some stuff that indicated rsync may be the culprit due to how it's checking the files… and suggested switching to tar instead. So I did… but no difference.

tar flags:

tarPath="/bin/gtar"
env LC_ALL=C $tarPath -c -v -f - -C $shareName+ --totals

# where $shareName is the autofs mounted directory

These are the default args that BackupPC is setup with.

No change.

Monitoring the BackupPC box's network activity with iftop, it seems it will spike in usage for a while (sometimes up to 90Mbps) but then will drop back down into Kbps or even sometimes Bps range. While it's in the slow mode, top shows activity with the command BackupPC_dump which is the backup job… so it's doing stuff and isn't stuck.

The current backup has been running now for over 24 hours, yet has only transferred 75.9GB according to iftop.

Best Answer

It may be faster to run rsync directly on the server. You have about a million files to access over the network. There are a couple of minimal installs of rsync that you can run. I've setup BackupPC on Windows this way. You can run a full Cygwin install, or the minimal cygwin-rsycnd install available in the BackupPC project.

Related Topic