I'm having trouble understanding my large S3 bill, and figured I'd ask here before dropping $30 on AWS monthly support.
Basically, I have an Amazon EC2 instance that makes an API to different cryptocurrency exchanges and saves the responses to the instance HD. Calls are made about every 5 minutes, response objects are about 100 kb, is read by an R script, and added to a CSV file every ~8 minutes. That CSV file is synchronised to an Amazon S3 bucket about every 15 minutes.
The CSV files are usually 10 MB or so, for about 15 cryptocurrencies, every 15 minutes. So looking in the Amazon S3 bucket, there might be 0.5 GB of space used at the most.
However, the 'TimedStorage-ByteHours' reads at about 4 TB!
Amazon Simple Storage Service TimedStorage-ByteHrs $89.55
$0.000 per GB – storage under the monthly global free tier5 GB – Mo $0.00
$0.023 per GB – first 50 TB / month of storage used 3,893.399 GB – Mo $89.55
Any ideas?
Best Answer
Most likely you've got S3 Versioning enabled - that means old objects when overwritten with a newer version don't get deleted but are instead hidden in a history. To verify go to the Bucket -> Properties -> Versioning.
You can also view the old versions in the browser, like on this screenshot I've got several versions of the
108c05...json
file:If you've got versioning enabled but don't want to you can Suspend versioning but be aware that it won't delete the old versions, you'll have to either:
aws s3api list-object-versions
)Hope that helps :)