How to add lifecycle rule to an existing AWS S3 bucket with Terraform

amazon s3amazon-web-servicesterraform

I have an existing S3 bucket and I wish to add "folders" and lifecycle_rules to it.

(I say "folders" because that is how they are represented at the client end, as they are accessed via a Storage Gateway.)

I can create the folders, for example on to hold quarterly backups, like:

resource "aws_s3_bucket_object" "quarterly" {
    bucket  = "${var.bucket_id}"
    acl     = "private"
    key     = "quarterly"
    source  = "/dev/null"
}

But if I try and then add a lifecycle rule, as follows

resource "aws_s3_bucket" "quarterly" {
    bucket  = "${var.bucket_id}"
    acl     = "private"

    lifecycle_rule {
        id      = "quarterly_retention"
        prefix  = "quarterly/"
        enabled = true
        tags {
            "rule"  = "quarterly"
        }

        expiration {
            days = 92
        }
    }
}

I get an error when I do terraform apply.

* aws_s3_bucket.quarterly: Error creating S3 bucket: BucketAlreadyOwnedByYou: Your previous request to create the named bucket succeeded and you already own it.
    status code: 409, request id: 702396A7D2FA28BA, host id: IJDA+vszRBYl4zmvW56dSnC2va2qpQXlfgeEL7X1QQHHv8eEaYKvSUCL0ZIj/VsdvQ2hkBLGjAY=

I want to create the bucket first and then add folders and lifecycle rules afterwards, rather than embed the lifecycle rules on creation.

Am I missing something, have I got it wrong?

Thanks for your help!

Best Answer

Creating the bucket first and incrementally updating the configuration should work fine, the net result is that if you were to delete the bucket by other means Terraform would recreate it with all of the rules in place.

It looks like you've lost your terraform.tfstate file so Terraform doesn't know that it has already created your bucket, or you created the bucket outside of Terraform in the first place, so it's trying to create it and failing. Terraform needs to "own" the bucket to be able to update its configuration, i.e. the lifecycle rules.

You should be able to import the existing bucket into your state file with something like

terraform import aws_s3_bucket.quarterly <your bucket ID>

See the bottom of https://www.terraform.io/docs/providers/aws/r/s3_bucket.html

Running Terraform should then show it just updating the lifecycle rules.

Related Topic