Duplicity Backup to S3 – BackendException Troubleshooting

amazon s3backupduplicitylinuxUbuntu

I'm trying to configure my Duplicity installation to backup to an Amazon S3 bucket. I keep getting a BackendException error with no further details.

EDIT: I've removed my Duply configuration from this question in an effort to isolate the issue more and switched to a plain duplicity command on a small directory (10 MB, 34 files).

Here's the command I'm trying to run:

duplicity full ./logs "s3://s3-us-east-1.amazonaws.com/bucketname" -v9

And here's the output:

root@ats:/var/ats# duplicity full ./logs "s3://s3-us-east-1.amazonaws.com/bucketname" -v9

Duplicity 0.6 series is being deprecated:
See http://www.nongnu.org/duplicity/

Using archive dir: /root/.cache/duplicity/876c7d0b54276e675d41f6ea6077d52f
Using backup name: 876c7d0b54276e675d41f6ea6077d52f
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.cfbackend Succeeded
Import of duplicity.backends.dpbxbackend Succeeded
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.ftpsbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.megabackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.swiftbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.~par2wrapperbackend Succeeded
Using temporary directory /tmp/duplicity-sQ3sGs-tempdir
Backend error detail: Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 1509, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 1503, in with_tempdir
    fn()
  File "/usr/local/bin/duplicity", line 1336, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/local/lib/python2.7/dist-packages/duplicity/commandline.py", line 1062, in ProcessCommandLine
    backup, local_pathname = set_backend(args[0], args[1])
  File "/usr/local/lib/python2.7/dist-packages/duplicity/commandline.py", line 955, in set_backend
    globals.backend = backend.get_backend(bend)
  File "/usr/local/lib/python2.7/dist-packages/duplicity/backend.py", line 163, in get_backend
    return _backends[pu.scheme](pu)
  File "/usr/local/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 163, in __init__
    self.resetConnection()
  File "/usr/local/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 189, in resetConnection
    raise BackendException(err.message)
BackendException

BackendException:

I've tested Boto (which is what Duplicity uses for S3 connections) with this script:

root@ats:/var/ats# python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto.s3
>>> conn = boto.s3.connect_to_region("us-east-1")
>>> bucket = conn.get_bucket("bucketname")
>>> for key in bucket.list(): print key, key.storage_class
...
<Key: bucketname,test.txt> STANDARD
>>> exit()

I've made sure that I'm running the latest appropriate versions of duplicity and python, and I've set the AWS_* variables which work fine in Boto.

Am I missing something? What should I do?

Best Answer

I don't know why the original command wasn't working, but in the end I figured out that changing the URL format from s3://s3-us-east-1.amazonaws.com/bucketname to s3+http://bucketname fixes the problem.

The reduced working command is now:

duplicity full ./logs "s3+http://bucketname" -v9
Related Topic