use aws-cli to extend expiration and restart the delete or archive countdown on objects in an S3 bucket
Background
S3 buckets allow you to specify lifecycle rules that tell AWS to automatically delete or archive any objects in that bucket after a specific number of days. You can also specify a prefix with each rule so that different objects in the same bucket stay for different amounts of time.
Example 1: I created a bucket named
logs.example.com(not the real name) that automatically archives an object to AWS Glacier after it has been sitting in S3 for 90 days.
Example 2: I created a bucket named
tmp.example.com(not the real name) that automatically delete a file after it has been sitting there for 30 days.
This works great until you realize that there are specific files that you want to keep around for just a bit longer than its original expiration.
You could download and then upload the object to reset its creation date, thus starting the countdown from zero; but through a little experimentation, I found that one easy way to reset the creation/modification timestamp on an S3 object is to ask S3 to change the object storage method to the same storage method it currently has.
The following example uses the new aws-cli command line tool to reset the timestamp of an S3 object, thus restarting the lifecycle counter. This has an effect similar to the Linux/Unix touch command.
Setup
Create a test S3 bucket and add a lifecycle to delete any object that is over 1 day old.
bucket=tmp-test.alestic.com
aws s3 mb s3://$bucket
aws s3api put-bucket-lifecycle --bucket $bucket --lifecycle-configuration '{"Rules":[{"Status":"Enabled","Prefix":"","Expiration":{"Days":1}}]}'
Upload a test file.
file=somefile.txt
aws s3 cp $file s3://$bucket/
Look at the timestamp on the test file in the test bucket.
aws s3 ls s3://$bucket
Wait a bit (seconds, hours, whatever).
Reset S3 Object Timestamp
Modify the timestamp on the file to be the current time, by telling Amazon to change the storage class to the current storage class. If you are storing objects with REDUCED-REDUNDANCY storage, then specify that instead of STANDARD.
aws s3api copy-object --storage-class STANDARD --copy-source $bucket/$file --bucket $bucket --key $file
Look at the timestamp again, to make sure it has been updated.
aws s3 ls s3://$bucket
The response JSON from the copy-object command will tell you the new expiration date.
Cleanup
If you followed the above commands to create a temporary test file and bucket, you can delete the file by waiting a couple days for it to expire, or run this:
aws s3 rm s3://$bucket/$file
Delete the (empty) bucket using:
aws s3 rb s3://$bucket
Notes
-
With the sample 1 day expiration, S3 lifecycle does not expire the file at exactly 24 hours, but rather rounds up to the next UTC midnight then deletes it at some point after that.
-
Amazon may decide at some point to not change the timestamp when the storage class does not actually change. The above approach works today, but has no warranty.