AWS_EN

Initial setup of DynamoDB with CFN custom resources

スポンサーリンク
Initial setup of DynamoDB with CloudFormation custom resources AWS_EN
スポンサーリンク
スポンサーリンク

Perform initial setup of DynamoDB with CloudFormation custom resources

When creating DynamoDB with CloudFormation, we will consider adding test records as part of the DB initialization.

In this case, we will use a CloudFormation custom resource to perform the initial setup of the DynamoDB table.

Environment

Diagram of initial setup of DynamoDB with CloudFormation custom resources

Create a DynamoDB table using CloudFormation.

Create a Lambda function.
Associate the function with a CloudFormation custom resource so that it will be automatically executed when the CloudFormation stack is created.
The function’s action is to initialize the DynamoDB table.
The initialization process reads JSON data placed in an S3 bucket and registers it as an item in the DynamoDB table.
The runtime environment for the function is Python 3.8.

CloudFormation template files

The above configuration is built using CloudFormation.
The CloudFormation templates are located at the following URL

awstut-fa/105 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Explanation of key points of the template files

DynamoDB table

Resources: Table: Type: AWS::DynamoDB::Table Properties: AttributeDefinitions: - AttributeName: Artist AttributeType: S - AttributeName: SongTitle AttributeType: S BillingMode: PROVISIONED KeySchema: - AttributeName: Artist KeyType: HASH - AttributeName: SongTitle KeyType: RANGE ProvisionedThroughput: ReadCapacityUnits: !Ref ReadCapacityUnits WriteCapacityUnits: !Ref WriteCapacityUnits TableClass: STANDARD TableName: !Sub "${Prefix}-Music"
Code language: YAML (yaml)

No special configuration is required.
The configuration on the following official AWS page is exported to a CloudFormation template.

Step 1: Create a table - Amazon DynamoDB
Get started with Amazon DynamoDB by creating a new table using the console or the AWS CLI.

For basic information about DynamoDB, please also check the following page

Lambda Function

Resources: Function: Type: AWS::Lambda::Function Properties: Environment: Variables: JSON_S3_BUCKET: !Ref JsonS3Bucket JSON_S3_KEY: !Ref JsonS3Key TABLE_NAME: !Ref Table Code: ZipFile: | import boto3 import cfnresponse import json import os JSON_S3_BUCKET = os.environ['JSON_S3_BUCKET'] JSON_S3_KEY = os.environ['JSON_S3_KEY'] TABLE_NAME = os.environ['TABLE_NAME'] CREATE = 'Create' response_data = {} s3_client = boto3.client('s3') dynamodb_client = boto3.client('dynamodb') def lambda_handler(event, context): try: if event['RequestType'] == CREATE: s3_response = s3_client.get_object( Bucket=JSON_S3_BUCKET, Key=JSON_S3_KEY) body = s3_response['Body'].read() json_data = json.loads(body.decode('utf-8')) print(json_data) for item in json_data: dynamodb_response = dynamodb_client.put_item( TableName=TABLE_NAME, Item=item ) print(dynamodb_response) cfnresponse.send(event, context, cfnresponse.SUCCESS, response_data) except Exception as e: print(e) cfnresponse.send(event, context, cfnresponse.FAILED, response_data) FunctionName: !Sub "${Prefix}-function" Handler: !Ref Handler Runtime: !Ref Runtime Role: !GetAtt FunctionRole.Arn Timeout: !Ref Timeout
Code language: YAML (yaml)

Define the code to be executed by the Lambda function in inline notation.
For more information, please refer to the following page

Use the cfnresponse module to implement functions as Lambda-backed custom resources.
For more information, please see the following page

Environment variables can be defined with the Variables property.
Environment variables can be referenced from within functions.
In this case, we will pass the name of the S3 bucket that contains the data for the initialization process and the file name as environment variables.

The code to be executed is as follows

  • Access os.environ to obtain the environment variables defined in the CloudFormation template.
  • Create a client object for S3 in Boto3 and retrieve the JSON file for initialization from the S3 bucket.
  • Create a DynamoDB client object in Boto3 and store the sample data in a table.
  • Notify CloudFormation of the successful completion of the custom resource using the send function of the cfnresponse module.

The JSON file to be read from the S3 bucket is as follows

[ {"Artist": {"S": "No One You Know"}, "SongTitle": {"S": "Call Me Today"}, "AlbumTitle": {"S": "Somewhat Famous"}, "Awards": {"N": "1"}}, {"Artist": {"S": "No One You Know"}, "SongTitle": {"S": "Howdy"}, "AlbumTitle": {"S": "Somewhat Famous"}, "Awards": {"N": "2"}}, {"Artist": {"S": "Acme Band"}, "SongTitle": {"S": "Happy Day"}, "AlbumTitle": {"S": "Songs About Life"}, "Awards": {"N": "10"}}, {"Artist": {"S": "Acme Band"}, "SongTitle": {"S": "PartiQL Rocks"}, "AlbumTitle": {"S": "Another Album Title"}, "Awards": {"N": "8"}} ]
Code language: JSON / JSON with Comments (json)

This data is also presented on the official AWS page.

Step 2: Write data to a table using the console or AWS CLI - Amazon DynamoDB
Learn to write data to a table in DynamoDB using the console or the AWS CLI.

The IAM roles for the functions are as follows

Resources: FunctionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: sts:AssumeRole Principal: Service: - lambda.amazonaws.com ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole Policies: - PolicyName: !Sub "${Prefix}-S3GetObjectPolicy" PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - s3:GetObject Resource: - !Sub "arn:aws:s3:::${JsonS3Bucket}/*" - PolicyName: !Sub "${Prefix}-DynamodbPutItemPolicy" PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - dynamodb:PutItem Resource: - !Sub "arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${Table}"
Code language: YAML (yaml)

In addition to the AWS management policy AWSLambdaBasicExecutionRole, an inline policy grants two permissions.

The first is the authority to retrieve objects from the S3 bucket.
This is the permission needed to retrieve the JSON file that is the sample data.

The second is the authority to store items in DynamoDB.
This is the authorization needed to write the data in the JSON file to a DynamoDB table.

CloudFormation Custom Resources

Next, we will check the CloudFormation custom resource itself.

Resources: CustomResource: Type: Custom::CustomResource Properties: ServiceToken: !Ref FunctionArn
Code language: YAML (yaml)

Associate the custom resource with the Lambda function described above.

Architecting

Use CloudFormation to build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stack.
For information on how to create a stack and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • DynamoDB table: fa-105-Music
  • Lambda function: fa-105-function

Check each resource from the AWS Management Console.
First, check the CloudFormation custom resources and Lambda functions.

Detail of CloudFormation Custom Resources 1.
Detail of CloudFormation Custom Resources 2.

We can see that the Lambda function and the custom resource itself have indeed been created successfully.
This means that the Lambda function was automatically created and invoked when the CloudFormation stack was created.

Next, check the DynamoDB.

Detail of DynamoDB 1.
Detail of DynamoDB 2.

The DynamoDB table is created as specified in the CloudFormation template.

Checking Action

Checking the Operation of the CloudFormation custom resource.

Check the execution log of the Lambda function associated with the CloudFormation custom resource.

Detail of CloudFormation Custom Resources 3.

The JSON data in the S3 bucket is read and saved one row at a time in the DynamoDB table.
Finally, a message is notified to CloudFormation.

Check the contents of the DynamoDB table.

Detail of DynamoDB 3.

The items are indeed stored in the table.

From the above, we can confirm that when the CloudFormation stack is created, the Lambda function associated with it is automatically invoked by the CloudFormation custom resource, and the item is saved in the DynamoDB table.

Summary

We have seen how to perform the initial setup of a DynamoDB table using a CloudFormation custom resource.

タイトルとURLをコピーしました