SAA_EN

Create Aurora custom endpoints with CFN custom resources

スポンサーリンク
Create Aurora custom endpoints with CFN custom resource. SAA_EN
スポンサーリンク
スポンサーリンク

Create Aurora custom endpoints with CloudFormation custom resources

One of the topics covered in the AWS SAA is the design of a high-performance architecture.

The following pages cover read replicas and endpoints for Aurora clusters.

In addition to the default endpoints, Aurora has the ability to create custom endpoints.

A custom endpoint for an Aurora cluster represents a set of DB instances that you choose. When you connect to the endpoint, Aurora performs load balancing and chooses one of the instances in the group to handle the connection. You define which instances this endpoint refers to, and you decide what purpose the endpoint serves.

Amazon Aurora connection management

We will create two custom endpoints.
The two endpoints are intended to be used to connect to a read replica for a specific use.
Creation of the custom endpoints will be performed using CloudFormation custom resources.

Environment

Diagram of create Aurora custom endpoints with CFN custom resource.

An Aurora cluster will be deployed across three private subnets with different AZs.
Create three DB instances in the cluster.
One will act as the primary server and the other two as read replicas.
The Aurora cluster will be of type MySQL.

Place an EC2 instance on a private subnet.
This EC2 instance will be used as a client to access the Aurora cluster.
The EC2 instance is based on the latest version of Amazon Linux2 AMI.

Create a VPC endpoint for S3.
This is to install a client to connect to the Aurora cluster on the EC2 instance.

Create a VPC endpoint for SSM.
To access the EC2 instance via SSM Session Manager.

CloudFormation template files

Build the above configuration with CloudFormation.
The CloudFormation templates are located at the following URL

awstut-saa/02/008 at main · awstut-an-r/awstut-saa
Contribute to awstut-an-r/awstut-saa development by creating an account on GitHub.

Explanation of key points of the template files

This page focuses on how to create custom endpoints for Aurora using CloudFormation custom resources.

For basic information on CloudFormation custom resources, please refer to the following page

Aurora

Resources: DBCluster: Type: AWS::RDS::DBCluster Properties: DatabaseName: !Ref DBName DBClusterIdentifier: !Sub "${Prefix}-dbcluster" DBSubnetGroupName: !Ref DBSubnetGroup Engine: !Ref DBEngine EngineVersion: !Ref DBEngineVersion MasterUsername: !Ref DBUser # cannot use "-". MasterUserPassword: !Ref DBPassword # cannot use "/@'" StorageEncrypted: true VpcSecurityGroupIds: - !Ref DBSecurityGroup DBSubnetGroup: Type: AWS::RDS::DBSubnetGroup Properties: DBSubnetGroupName: !Sub "${Prefix}-dbsubnetgroup" # must be lowercase alphanumeric characters or hyphens. DBSubnetGroupDescription: Test DBSubnetGroup for Aurora. SubnetIds: - !Ref DBSubnet1 - !Ref DBSubnet2 - !Ref DBSubnet3 DBInstance1: Type: AWS::RDS::DBInstance Properties: DBClusterIdentifier: !Ref DBCluster DBSubnetGroupName: !Ref DBSubnetGroup DBInstanceIdentifier: !Sub "${Prefix}-dbinstance1" DBInstanceClass: !Ref DBInstanceClass Engine: !Ref DBEngine AvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone1}" PubliclyAccessible: false DBInstance2: Type: AWS::RDS::DBInstance Properties: DBClusterIdentifier: !Ref DBCluster DBSubnetGroupName: !Ref DBSubnetGroup DBInstanceIdentifier: !Sub "${Prefix}-dbinstance2" DBInstanceClass: !Ref DBInstanceClass Engine: !Ref DBEngine AvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone2}" PubliclyAccessible: false DBInstance3: Type: AWS::RDS::DBInstance Properties: DBClusterIdentifier: !Ref DBCluster DBSubnetGroupName: !Ref DBSubnetGroup DBInstanceIdentifier: !Sub "${Prefix}-dbinstance3" DBInstanceClass: !Ref DBInstanceClass Engine: !Ref DBEngine AvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone3}" PubliclyAccessible: false
Code language: YAML (yaml)

Create an Aurora cluster.
Create 3 DB instances in the cluster.
No special configuration is required in creating custom endpoints.

CloudFormation Custom Resources

Custom Resources

Resources: CustomResource: Type: Custom::CustomResource Properties: ServiceToken: !GetAtt Function.Arn
Code language: YAML (yaml)

Set the ARN of the resource to be used for the backend action to the ServiceToken property.
In this case, the Lambda function described below will be used for the action, so set the ARN for this function.

Lambda function

Resources: Function: Type: AWS::Lambda::Function Properties: Code: ZipFile: | import boto3 import cfnresponse import os db_cluster_identifier = os.environ['DB_CLUSTER_IDENTIFIER'] db_cluster_custom_endpoint1 = os.environ['DB_CLUSTER_CUSTOM_ENDPOINT1'] db_cluster_custom_endpoint2 = os.environ['DB_CLUSTER_CUSTOM_ENDPOINT2'] custom_endpoints = [ db_cluster_custom_endpoint1, db_cluster_custom_endpoint2 ] client = boto3.client('rds') CREATE = 'Create' response_data = {} def lambda_handler(event, context): try: if event['RequestType'] == CREATE: response = client.describe_db_clusters( DBClusterIdentifier=db_cluster_identifier ) read_replica_instances = [member for member in response['DBClusters'][0]['DBClusterMembers'] if member['IsClusterWriter'] == False] for (endpoint_name, read_replica_instance) in zip(custom_endpoints, read_replica_instances): instance_name = read_replica_instance['DBInstanceIdentifier'] create_response = client.create_db_cluster_endpoint( DBClusterIdentifier=db_cluster_identifier, DBClusterEndpointIdentifier=endpoint_name, EndpointType='READER', StaticMembers=[instance_name]) print(create_response) response_data[endpoint_name] = create_response['Endpoint'] cfnresponse.send(event, context, cfnresponse.SUCCESS, response_data) except Exception as e: print(e) cfnresponse.send(event, context, cfnresponse.FAILED, response_data) Environment: Variables: DB_CLUSTER_CUSTOM_ENDPOINT1: !Ref DBClusterCustomEndpoint1 DB_CLUSTER_CUSTOM_ENDPOINT2: !Ref DBClusterCustomEndpoint2 DB_CLUSTER_IDENTIFIER: !Ref DBClusterIdentifier FunctionName: !Sub "${Prefix}-function" Handler: !Ref Handler Runtime: !Ref Runtime Role: !GetAtt FunctionRole.Arn
Code language: YAML (yaml)

There are no special items in the configuration of the function itself.
One point to mention is the environment variable.
Pass a custom endpoint name or Aurora cluster name to the function as an environment variable.

Refer to the value of event[‘RequestType’] to implement the processing according to the stack operation.
In this case, we will execute the process when this value is ‘Create’, that is, when the stack is created.

Execute the describe_db_clusters method to obtain the details of the Aurora cluster.
In the retrieved information, check the data about the DB instances that are members of the cluster and extract only the instances that are read replicas.
Create custom endpoints for them.
Specifically, execute the create_db_cluster_endpoint method.

Use cfnresponse.send to return a function execution completion message to the CloudFormation stack, passing the name of the created custom endpoint as a list type.
You will now be able to access these two addresses from your CloudFormation template.

IAM Role

Resources: FunctionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: sts:AssumeRole Principal: Service: - lambda.amazonaws.com ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole Policies: - PolicyName: !Sub "${Prefix}-CreateAuroraCustomEndpointPolicy" PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - rds:CreateDBClusterEndpoint - rds:DescribeDBClusters Resource: - !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:cluster:${DBClusterIdentifier}" - !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:cluster-endpoint:*"
Code language: YAML (yaml)

The Lambda function will access information about the Aurora cluster and create a custom endpoint, so allow the “rds:DescribeDBClusters” and “rds:CreateDBClusterEndpoint” actions.

Outputs Section

Outputs: DBClusterCustomEndpoint1: Value: !GetAtt - CustomResource - !Ref DBClusterCustomEndpoint1 DBClusterCustomEndpoint2: Value: !GetAtt - CustomResource - !Ref DBClusterCustomEndpoint2
Code language: YAML (yaml)

Get custom endpoint information created by a CloudFormation custom resource.
Combines the built-in functions Fn::GetAtt and Fn::Ref to retrieve custom endpoint information from custom resources.

Architecting

Use CloudFormation to build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • Aurora cluster: saa-02-008-dbcluster
  • EC2 instance: i-014c001efcf83db1e

The resource creation status is also checked from the AWS Management Console.
Check the Aurora cluster.

Detail of Aurora 1.

We can see that the cluster has been created and that there are three DB instances inside.

In the Endpoints section, we can see that two custom endpoints have been created.
We will verify the details of each.

Detail of Aurora 2.
Detail of Aurora 3.

The following is a summary of the information we have confirmed.

Endpoint NameDB InstanceRole
custom-endpoint1.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.comsaa-02-008-dbinstance3Reader
custom-endpoint2.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.comsaa-02-008-dbinstance2Reader

Check the execution results of the Lambda function, a CloudFormation custom resource.

Detail of Lambda 1.
Detail of Lambda 2.

You can see that indeed the Lambda function was automatically executed and the custom endpoint we just checked was created.

Check Action

Now that everything is ready, let’s access the EC2 instance.

Use SSM Session Manager to access the instance.

% aws ssm start-session --target i-014c001efcf83db1e ... sh-4.2$
Code language: Bash (bash)

For more information on SSM Session Manager, please see the following page

Make sure the client package (MariaDB) for accessing Aurora is installed.

sh-4.2$ sudo yum list installed | grep maria mariadb.aarch64 1:5.5.68-1.amzn2 @amzn2-core mariadb-libs.aarch64 1:5.5.68-1.amzn2 installed sh-4.2$ mysql -V mysql Ver 15.1 Distrib 5.5.68-MariaDB, for Linux (aarch64) using readline 5.1
Code language: Bash (bash)

The package has indeed installed.

Check the name resolution status of the custom endpoints with the nslookup command.

sh-4.2$ nslookup custom-endpoint1.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com Server: 10.0.0.2 Address: 10.0.0.2#53 Non-authoritative answer: custom-endpoint1.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com canonical name = saa-02-008-dbinstance3.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com. Name: saa-02-008-dbinstance3.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com Address: 10.0.4.161
Code language: Bash (bash)
sh-4.2$ nslookup custom-endpoint2.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com Server: 10.0.0.2 Address: 10.0.0.2#53 Non-authoritative answer: custom-endpoint2.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com canonical name = saa-02-008-dbinstance2.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com. Name: saa-02-008-dbinstance2.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com Address: 10.0.3.221
Code language: Bash (bash)

As confirmed in the AWS Management Console, you can see that a DB instance is associated with each custom endpoint.

Attempt to connect to the custom endpoint.

sh-4.2$ mysql -h custom-endpoint1.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com -P 3306 -u testuser -p testdb Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MySQL connection id is 20 Server version: 8.0.23 Source distribution Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MySQL [testdb]> MySQL [testdb]> create table testtable (datetime datetime); ERROR 1836 (HY000): Running in read-only mode
Code language: JavaScript (javascript)
sh-4.2$ mysql -h custom-endpoint2.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com -P 3306 -u testuser -p testdb Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MySQL connection id is 20 Server version: 8.0.23 Source distribution Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MySQL [testdb]> MySQL [testdb]> create table testtable (datetime datetime); ERROR 1836 (HY000): Running in read-only mode
Code language: Bash (bash)

Both endpoints were able to connect successfully.
Because the role is Reader, an error occurred when I tried to perform a write.

Summary

We have seen how to create an Aurora custom endpoint using a CloudFormation custom resource.

タイトルとURLをコピーしました