Duplicity for remote backup to AWS S3

amazon s3backupbaculaduplicitygpg

I am using Bacula with S3 for backups. I keep bacula volumes for 60 days or so. Using duplicity as I would like to use GPG encryption of the volumes.
Is there any way to do an rsync style sync using duplicity? What I want is exactly the same structure of files in S3 as I have on bacula. I would do this usually with rsync –delete.

Best Answer

The short answer is... sort of.

It's possible to back and retrieve your files. However, if you look in your Amazon bucket from the web interface, all you will see are backup archives, not the actually files that you can download. This is where Duplicity and Duply (simple Duplicity) come in.

Running a simple command from the server such as duply myfirstback list will give you a list of all files backed up.

Running duply myfirstback fetch <file on S3> <destination> will let you retrieve individual files or entire directory structures.

Running duply myfirstbackup status will give you a list of all full and incremental backups.

You can then tell Duply which backup set to retrieve files from. For example: duply myfirstbackup fetch 'home/myhome/importantdirectory' ./home/myhome/restore 3D will restore the /home/myhome/importantdirectory from backups made 3 days ago, and restore it locally in /home/myhome/restore (see "duplicity manpage", section TIME FORMATS)

How
Assuming you will be on a Linux OS, you may use Duply with Duplicity to simplify the encryption and incremental backup of your server data to S3.

Prerequisites: python-lockfile, librsync, boto, duplicity, duply

The config files for Duply, which will control Duplicity for you, can be put in /root/.duply/

It is then simply a matter of creating your backup configs, which can be done by issuing the command duply myfirstbackup create.

You can then edit the two files (conf,exlude) in /root/.duply/myfirstbackup. Here is a simplified example of both:

/root/.duply/myfirstbackup/conf 

GPG_PW='my_super_secret_password_which_cannot_lose!'
TARGET='s3://s3.amazonaws.com/<bucketname>/my_backup_folder'
TARGET_USER='AWS_ACCESS_ID'
TARGET_PASS='AWS_ACCESS_SECRET'
SOURCE='/'
MAX_AGE=1M
TEMP_DIR=/tmp

/root/.duply/myfirstbackup/exclude

+ /home/myhome
**

The above will backup everything in /home/myhome and keep a copy for 1 month. See duply docs for more on how to setup the configurations, for example for 60 days incremental. You can do things like set a full backup weekly with incremental every 2 hours. You have a lot of control over this.

Last step is then simply to ensure you setup a cronjob to run this however often you want. Daily, hourly, weekly etc.

Duply: http://www.duply.net

Related Topic