So I have a pretty simple backup script that runs as a cron job every night and it is:
rsync -azhv /company/shared_files/ /mnt/ext_drive/backups/shared_files/company_share_backup_"`date +\%Y-\%m-\%d`"
And previously the size of the backup was around 20 gigs and it ran in around ten minutes but as of two days ago it is 80 gigs and it takes 6+ hours to run. What could have gone wonky?
My general procedure is to keep each backup for 7 days and then keep the backup from Sunday each week to save space so ideally I would like to do a separate rsync for each day as opposed to doing rsync in its more natural fashion of just updating the files in a backup that have changed.
Additional Details
I'm running Debian Wheezy with a 2TB hard drive and 16G of memory and transferring these files from my Debian server to a WD My Passport Ultra with 2TB.
Best Answer
There are a few things you can do there. You don't need the
-zflag to rsync for a local copy. Compression is not used for transfers that aren't remote.You can optimize the rsync a bit better for small files and the type of changes with other options like
-W(to transfer whole files without the prescan).Also, shouldn't you be deleting files on the destination?
More details on the actual OS you're using, the disk capabilities and the backup target could help focus the solution a bit better.