I'll start by saying what we have in place currently:
- On site file server (Mac OS X Server) that is used by GFX designers and they have a working 1TB of data.
- Offsite server with 2TB available storage (Centos 6)
- Mac OS X server rsync data to offsite server every 6 hours (
rsync -avz --delete --progress -e ssh ...) - Mac OS X server does full data backup to LTO 4 tape on a 10 day recycle (Mon-Fri for 2 weeks)
- rsync pushes about 60GB of file changes a day.
The problem:
- The onsite tape backup is failing as 1TB of graphics files don't compress well to fit onto a 800GB LTO4 tape.
- Backup is incredibly slow doing a full backup.
- Pain in the backside getting people to remember to change the tape. Often gets forgotten
- etc
The quick solution:
- Buy LTO5 Drive and tapes. However this has been turned down because of the cost…
What I would like:
- Something that works in the same way rysnc works. Only changed data is sent over the wire and can be scheduled to run multiple times during the day. Data that is sent is compressed and sent over SSH.
- Something that keeps a 14day retention but doesn't keep duplicate data
- So as an example if I have 1TB of working data and 60GB of changes are made each day then I expect around 1.84TB of data to be stored on the offsite server.
- To work with the Mac OS X server and Centos 6 server.
- Not cost an arm and a leg. Must be a cheaper solution than buying an LTO5 drive with tapes (around £1500).
- Be able to be setup to run autonomously.
- Have some sort of control panel that will allow an admin to easily restore a file/folder.
Any recommendations?
Best Answer
Try rsnapshot (rsnapshot.org). It does exactly what you're after: it's in the RPMForge yum repos (so is packaged for CentOS), operates over rsync via SSH, and keeps a configurable number of incremental backups.
edit: You could implement a recovery front end by exporting the rsnapshot datastore over NFS or Samba (or a webserver/anything else) and let users pick up old copies of their files themselves.