Concurrent modification during backup: rsync vs dump vs tar vs

The ideal solution is to create a sparsebundle image and to sync that. This can be most easily done by sharing the drive over the network and pointing Time Machine to it and starting a fresh backup.

You can also find information on creating a custom sparsebundle to use as a Time Machine backup.

The reason to do this is because a sparsebundle stores its data spread over a collection of files (8MB 'bands'). If you have a sparsebundle, you can rync it to the remote server, and rsync can just transfer the bands that have changed. With Time Machine, you typically just append onto the end, so you'll only usually be syncing the last few bands up.

Once a given band is full and OS X creates another one, the now-full band won't be written to anymore. It gets copied up one last time, and never touched again. The new band gets copied every time rsync is run until it too fills up and stops being written to.

For extra fun, there are options you can pass to hdiutil to create an encrypted disk image, so that if someone breaks into your Dreamhost machine they won't be able to harvest your files.

bozosync.sh

Hmm, if you only want to move the changed files, why would the tar command be updating the archive? If only changed files are being sent, wouldn't the tar archive consist only of them, and so be necessarily recreated from scratch each time?

If I'm understanding this correctly you want something like this, I call it bozosync.sh:

TarFile=fun.tgz
TheScript=$0
result=
ls -1 . | (
  while read f; do
    if [ -f "$f" -a $f -nt $TheScript ]; then
      result="$result $f"
    fi
  done
  tar cvfz $TarFile $result
  touch $TheScript
)

This puts every plain file in the current directory in the tar archive, for each file that is newer than the script itself, and then it runs touch(1) on the script to update its mod time. This does have a race condition, in that a file that is modified in between the test and the touch will be ignored the next time around. If this is a problem you could get around that by copying the script prior to the loop, and then moving it back to itself at the end. (Then you still have a race but the failure is softer, it may simply include a file that hasn't really changed.)