RSYNC – How to Do Incremental Backups

backuprsyncsftp

How does rsync know which files are changed and which are not? Does it log its data anywhere in the file?

Because I want to do incremental backups, but first it will transfer all files.

So my main question is: if I upload the initial files via FTP but not by rsync.
Will rsync still skips those existing files or it will upload everything on the first run.

Best Answer

Rsync has a number of flags which control what it will look at and what it will copy over to the destination. Most often the "-a" flag is used which is the "Archive" flag, this is probably what you want. run rsync with the "-av" flags and have it do a first run against the data you want backed up. The next time it runs it'll do a block checksum of the file, and only copy over the parts which have been modified on existing files, copy new files over, and remove files which are no longer there. Check the "-a" options section on:

http://linux.die.net/man/1/rsync

The first run will be BandWidth intensive, the following runs will most likely be processor intensive but use little Bandwidth compared to the initial run. Unless you have a lot of churn over your data set.

Rsync doen't care how you got the files in the source, or the destination directories, it's only going to copy the changes between the two, unless you add flags to do something different.

If you want to log what was changed you can use the "--log-file" option. All in all something like this sounds like what you want:

rsync -av --log-file=/var/log/rsync.log -e "ssh -l backup-user" backup-user@source-machine::module /nas01/backups