Enable DynamoDB expiration time (TTL) using CloudFormation
One of the features of DynamoDB is TTL.
Amazon DynamoDB Time to Live (TTL) allows you to define a per-item timestamp to determine when an item is no longer needed. Shortly after the date and time of the specified timestamp, DynamoDB deletes the item from your table without consuming any write throughput.
Expiring items by using DynamoDB Time to Live (TTL)
The specifics of TTL are covered in the following official AWS page.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html
The purpose of this page is to reproduce the above page using CloudFormation.
Specifically, we will create a DynamoDB table with TTL enabled, and confirm that when an item expires, it is automatically deleted.
Environment
Create a DynamoDB table.
In addition to the usual settings for partition keys, sort keys, etc., configure the table with settings related to TTL activation.
Create a Lambda function.
The function’s action is to create the sample item data and store it in the DynamoDB table.
The runtime environment for the function is Python 3.8.
CloudFormation template files
The above configuration is built with CloudFormation.
The CloudFormation templates are located at the following URL
https://github.com/awstut-an-r/awstut-fa/tree/main/107
Explanation of key points of the template files
DynamoDB
Resources:
Table:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
- AttributeName: UserName
AttributeType: S
- AttributeName: SessionId
AttributeType: S
BillingMode: PAY_PER_REQUEST
KeySchema:
- AttributeName: UserName
KeyType: HASH
- AttributeName: SessionId
KeyType: RANGE
TableClass: STANDARD
TableName: !Ref Prefix
TimeToLiveSpecification:
AttributeName: TTL
Enabled: true
Code language: YAML (yaml)For basic information on DynamoDB, please refer to the following page
Set “UserName” as the partition key and “SessionId” as the sort key.
Settings regarding TTL are made in the TimeToLiveSpecification property.
Specify the attribute related to the TTL expiration date with the AttributeName property.
In this case, specify the attribute named “TTL”.
Lambda Function
Resources:
Function:
Type: AWS::Lambda::Function
Properties:
Environment:
Variables:
TABLE_NAME: !Ref Table
Code:
ZipFile: |
import boto3
import datetime
import json
import os
import random
import string
import time
TABLE_NAME = os.environ['TABLE_NAME']
dynamodb_client = boto3.client('dynamodb')
def lambda_handler(event, context):
now = datetime.datetime.now()
for i in range(10):
username = 'user' + str(i)
sessionid = ''.join(random.choices(string.ascii_letters + string.digits, k=10))
ttl = now + datetime.timedelta(minutes=i+5)
ttl_str = ttl.strftime('%Y-%m-%d %H:%M:%S.%f')
ttl_unix = str(time.mktime(ttl.timetuple()))
item = {
'UserName': {'S': username},
'SessionId': {'S': sessionid},
'TTL_str': {'S': ttl_str},
'TTL': {'N': ttl_unix}
}
dynamodb_response = dynamodb_client.put_item(
TableName=TABLE_NAME,
Item=item
)
print(dynamodb_response)
FunctionName: !Sub "${Prefix}-function"
Handler: !Ref Handler
Runtime: !Ref Runtime
Role: !GetAtt FunctionRole.Arn
Timeout: !Ref Timeout
Code language: YAML (yaml)The code to be executed by the Lambda function in inline format.
For more information, please see the following page
After creating a client object for DynamoDB, use the put_item method to store items in a table.
In this case, we will create and save 10 sample data items.
The expiration time to be set for each item is based on the time when the function is executed, and is increased by one minute.
The key point is “TTL,” which is the item to be saved.
This value is used as the expiration date for this item.
The value to be used as the TTL is explained in the AWS Official as follows
The TTL attribute’s value must be a top-level Number data type.
The TTL attribute’s value must be a timestamp in Unix epoch time format in seconds.
Using DynamoDB Time to Live (TTL)
Calculate UNIX time from date/time data according to the above and specify it in “TTL”.
Confirm the IAM role for the Lambda function.
Resources:
FunctionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: !Sub "${Prefix}-DynamodbPutItemPolicy"
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- dynamodb:PutItem
Resource:
- !Sub "arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${Table}"
Code language: YAML (yaml)Authorization to store items against a DynamoDB table.
Architecting
Use CloudFormation to build this environment and check the actual behavior.
Create CloudFormation stacks and check resources in stacks
Create a CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following page
After checking the resources in each stack, information on the main resources created this time is as follows
- DynamoDB table: fa-107
- Lambda function: fa-107-function
Check each resource from the AWS Management Console.
Check DynamoDB.
The TTL is indeed enabled.
Next, check the Lambda function.
This function is also successfully created.
When this function is invoked, the sample data is saved in the DynamoDB table.
Checking Action
Now that everything is ready, invoke the Lambda function.
The Lambda function has been successfully executed.
The function execution log is also checked.
We can see that the data was indeed saved 10 times in the DynamoDB table.
Check the table.
It is true that 10 sample data are stored in the table.
Looking at the TTL (TTL) item, we see that the expiration time is set in UNIX time format.
Wait for a while until the expiration time of some items has passed.
The number of items has indeed decreased.
It means that the expired items have been deleted automatically.
Wait for a while longer.
All items have been deleted.
As you can see, by enabling TTL in DynamoDB and setting the expiration date and time for each item, you can automatically delete items.
Summary
We have confirmed that when you create a DynamoDB table with TTL enabled and an item expires, the item will be automatically deleted.