No, S3 lifecycle policies cannot be used to create multiple point in time snapshots of S3.
Lifecycle policies can be used to move S3 objects to Glacier automatically , but the originals are no longer accessible in S3 storage without first restoring from Glacier.
I have read about the versioning feature for S3 buckets, but I cannot seem to find if >recovery is possible for files with no modification history. See the AWS docs here on >versioning:
I've just tried this. Yes, you can restore from the original version. When you delete the file it makes a delete marker and you can restore the version before that, i.e: the single, only, revision.
Then, we thought we may just backup the S3 files to Glacier using object lifecycle >management:
But, it seems this will not work for us, as the file object is not copied to Glacier but >moved to Glacier (more accurately it seems it is an object attribute that is changed, but >anyway...).
Glacier is really meant for long term storage, which is very infrequently accessed. It can also get very expensive to retrieve a large portion of your data in one go, as it's not meant for point-in-time restoration of lots of data (percentage wise).
Finally, we thought we would create a new bucket every month to serve as a monthly full >backup, and copy the original bucket's data to the new one on Day 1. Then using something >like duplicity (http://duplicity.nongnu.org/) we would synchronize the backup bucket every >night.
Don't do this, you can only have 100 buckets per account, so in 3 years you'll have taken up a third of your bucket allowance with just backups.
So, I guess there are a couple questions here. First, does S3 versioning allow recovery of >files that were never modified?
Yes
Is there some way to "copy" files from S3 to Glacier that I have missed?
Not that i know of
Best Answer
I see some useful comments are already here but will throw in just one comment: I would frankly go with putting your objects into S3 and then simply create a very short "Transition to Glacier" policy, like a few hours or 1 day old. You can access Glacier directly through the API, but the reason I would go this route is that you will still see your objects listed within S3, but you'll see in the "Storage Class" column that after a few days everything will have been moved into Glacier. Then, when/if you need to retrieve something out of Glacier, you can make the retrieval request (called "initiate restore" in the new ACTIONS menu of S3's web GUI) it's all right there for you to see and work with.
I'm a huge fan of the CLI for much of my AWS work, but I think until there's a really robust and usable CLI tool for Glacier, this seems easier to work with.