I'm looking for a way to implement something close to the following backup scheme:
- Initially, a full image is copied to the backup target.
- Periodically (e.g. nightly), only blocks that have changed since the last backup are copied to the backup target.
- Ideally, it should be possible to mount snapshots from any point in time, or delete (flatten) some snapshots selectively.
Can this be implemented using LVM (or some other way)? It needs tracking which blocks have become dirty since the last backup, which I'm not sure LVM can do… I'd rather avoid the permanent performance cost of running on an LVM snapshot at all times.
Best Answer
A newcomer to the scene is Attic https://attic-backup.org/
We used rdiff-backup for a few years as our primary backup method. It was great for what it did, but created tens/hundreds of thousands of small diff files across the course of a year. Most file systems and disks are going to struggle to deal with a million-plus file count. Backing up our 90GB Maildir-based IMAP store would take a few hours. I had to constantly lower the number of weeks/diffs that we would keep for history.
In comparison, once we switched to Attic, nightly backups ran in only 15-20 minutes. That means it's much more viable to keep a year's worth of incremental backups to let you go back to any day within the past year.
Main features that drew me to Attic:
After using it for 6-9 months, I'm fairly confident that it's as stable as rdiff-backup. I still do a multi-generation copy of the Attic directories using removable media, but each removable media has a full copy of the Attic repository.