I've been tasked with setting up a backup system for my small office (around 12 people). Most of our production stuff is on the AWS cloud, so what I need to back up are some small office/development files (under 100G right now), plus our operational VMs and development, which round out to a bit under 1T.
I just need something reliable, convenient, and straightforward. I'm comfortable with Linux, FreeBSD, and to some extent Solaris 10, so I'm leaning toward a full server rather than an appliance system ala Openfiler or FreeNAS.
What I'm contemplating is a small fileserver for general storage and nightly backups of the virtual machines, followed up by an offsite backup to Amazon's S3 storage service. It'd be the usual incremental backups nightly and full backup weekly.
My question is if using ZFS snapshots, both locally and dumped to S3 via 'zfs send [-i]', is a viable backup tool? Or should I stick to using Duplicity, or some other method entirely?
ZFS snapshots on the internal fileserver/backup machine sound like a perfect way to provide quick and convenient data recovery, so I'm likely to go with that for local redundancy. (If you folks see scenarios where relying on ZFS snapshots would be worse than a more traditional archiving backup, feel free to convince me.) But are snapshots flexible enough to lean on for recovery from the loss of my backup server? Or am I better off with something more traditional? (feel free to recommend free or commercial backup solutions you favor.)
Best Answer
Keep in mind that unless you quiesce the VMs before you snapshot the ZFS filesystem that their virtual hard disks reside on, you will in effect be making a backup of a crashed system. Also, in order to restore a single file that resides on a ZFS snapshot, you will need to load the entire snapshot somewhere, assuming the worst case of having to pull a snapshot that has been retrieved from remote storage.
While ZFS and snapshots are great for the "uh oh, something really bad happened, and I need the filesystem to be like it was an hour, day week ago", they really aren't a backup solution in the traditional sense. I've used ZFS and snapshots in conjunction with other backup software, such as bacula (www.bacula.org) with good results. It is very handy to be able to zfs send a snapshot of a filesystem off to the backup server where it is then spooled to tape without impacting the production system I/O.