Using Duplicity with AWS S3 and IAM Folder policy

amazon s3duplicitypermissions

I'm trying to setup backups from multiple hosts to a single bucket. Each host has their own AWS login and credentials. The top most folder (prefix) maps to the AWS user id and I'm trying to limit access to that folder only.

During the backup I get "S3ResponseError: 403 Forbidden". I'm using the following policy:

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowGroupToSeeBucketListInTheConsole",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::*"
            ]
        },
        {
            "Sid": "AllowRootAndHomeListingOfCompanyBucket",
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::mybucketexample"
            ],
            "Condition": {
                "StringEquals": {
                    "s3:prefix": [
                        ""
                    ],
                    "s3:delimiter": [
                        "/"
                    ]
                }
            }
        },
        {
            "Sid": "AllowListingOfUserFolder",
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::mybucketexample"
            ],
            "Condition": {
                "StringLike": {
                    "s3:prefix": [
                        "${aws:username}/*"
                    ]
                }
            }
        },
        {
            "Sid": "AllowAllS3ActionsInUserFolder",
            "Action": [
                "s3:*"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::mybucketexample/${aws:username}/*"
            ]
        }
    ]
}

When I attach a policy that allow all access to s3, duplicity backups work just fine. So the policy above is missing something or has an error.

One interesting observation: there eventually are duplicity files in the S3 so I'm wondering if puts are allowed but listings are not.

Any ideas?

Best Answer

I also come across this issue, and there is a detailed discussion about this issue at AWS Forum.

The main problem is that, some S3 clients simply don't add the "prefix" in the "ListBucket" request. Therefore, there is no match on the "AllowRootAndHomeListingOfCompanyBucket", and hence access denied. This should be the case for duplicity.

Another work around is to use a deny rule instead, which was suggested by Daniel in that thread. The policy should have the same effect as the one you mentioned.

Below one is modified and added user variable, and it should work for duplicity:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowUserToSeeBucketListInTheConsole",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::*"
            ]
        },
        {
            "Sid": "AllowRootAndHomeListingOfCompanyBucket",
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::mybucketexample"
            ]
        },
        {
            "Sid": "DenyAllListingExceptRootAndUserFolders",
            "Effect": "Deny",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::mybucketexample"
            ],
            "Condition": {
                "Null": {
                    "s3:prefix": "false"
                },
                "StringNotLike": {
                    "s3:prefix": [
                        "",
                        "${aws:username}/*"
                    ]
                }
            }
        },
        {
            "Sid": "AllowAllS3ActionsInUserFolder",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::mybucketexample/${aws:username}/*"
            ]
        }
    ]
}
Related Topic