Create Aurora custom endpoints with CloudFormation custom resources
One of the topics covered in the AWS SAA is the design of a high-performance architecture.
The following pages cover read replicas and endpoints for Aurora clusters.
In addition to the default endpoints, Aurora has the ability to create custom endpoints.
A custom endpoint for an Aurora cluster represents a set of DB instances that you choose. When you connect to the endpoint, Aurora performs load balancing and chooses one of the instances in the group to handle the connection. You define which instances this endpoint refers to, and you decide what purpose the endpoint serves.
Amazon Aurora connection management
We will create two custom endpoints.
The two endpoints are intended to be used to connect to a read replica for a specific use.
Creation of the custom endpoints will be performed using CloudFormation custom resources.
Environment
An Aurora cluster will be deployed across three private subnets with different AZs.
Create three DB instances in the cluster.
One will act as the primary server and the other two as read replicas.
The Aurora cluster will be of type MySQL.
Place an EC2 instance on a private subnet.
This EC2 instance will be used as a client to access the Aurora cluster.
The EC2 instance is based on the latest version of Amazon Linux2 AMI.
Create a VPC endpoint for S3.
This is to install a client to connect to the Aurora cluster on the EC2 instance.
Create a VPC endpoint for SSM.
To access the EC2 instance via SSM Session Manager.
CloudFormation template files
Build the above configuration with CloudFormation.
The CloudFormation templates are located at the following URL
https://github.com/awstut-an-r/awstut-saa/tree/main/02/008
Explanation of key points of the template files
This page focuses on how to create custom endpoints for Aurora using CloudFormation custom resources.
For basic information on CloudFormation custom resources, please refer to the following page
Aurora
Resources:
DBCluster:
Type: AWS::RDS::DBCluster
Properties:
DatabaseName: !Ref DBName
DBClusterIdentifier: !Sub "${Prefix}-dbcluster"
DBSubnetGroupName: !Ref DBSubnetGroup
Engine: !Ref DBEngine
EngineVersion: !Ref DBEngineVersion
MasterUsername: !Ref DBUser # cannot use "-".
MasterUserPassword: !Ref DBPassword # cannot use "/@'"
StorageEncrypted: true
VpcSecurityGroupIds:
- !Ref DBSecurityGroup
DBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupName: !Sub "${Prefix}-dbsubnetgroup" # must be lowercase alphanumeric characters or hyphens.
DBSubnetGroupDescription: Test DBSubnetGroup for Aurora.
SubnetIds:
- !Ref DBSubnet1
- !Ref DBSubnet2
- !Ref DBSubnet3
DBInstance1:
Type: AWS::RDS::DBInstance
Properties:
DBClusterIdentifier: !Ref DBCluster
DBSubnetGroupName: !Ref DBSubnetGroup
DBInstanceIdentifier: !Sub "${Prefix}-dbinstance1"
DBInstanceClass: !Ref DBInstanceClass
Engine: !Ref DBEngine
AvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone1}"
PubliclyAccessible: false
DBInstance2:
Type: AWS::RDS::DBInstance
Properties:
DBClusterIdentifier: !Ref DBCluster
DBSubnetGroupName: !Ref DBSubnetGroup
DBInstanceIdentifier: !Sub "${Prefix}-dbinstance2"
DBInstanceClass: !Ref DBInstanceClass
Engine: !Ref DBEngine
AvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone2}"
PubliclyAccessible: false
DBInstance3:
Type: AWS::RDS::DBInstance
Properties:
DBClusterIdentifier: !Ref DBCluster
DBSubnetGroupName: !Ref DBSubnetGroup
DBInstanceIdentifier: !Sub "${Prefix}-dbinstance3"
DBInstanceClass: !Ref DBInstanceClass
Engine: !Ref DBEngine
AvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone3}"
PubliclyAccessible: false
Code language: YAML (yaml)Create an Aurora cluster.
Create 3 DB instances in the cluster.
No special configuration is required in creating custom endpoints.
CloudFormation Custom Resources
Custom Resources
Resources:
CustomResource:
Type: Custom::CustomResource
Properties:
ServiceToken: !GetAtt Function.Arn
Code language: YAML (yaml)Set the ARN of the resource to be used for the backend action to the ServiceToken property.
In this case, the Lambda function described below will be used for the action, so set the ARN for this function.
Lambda function
Resources:
Function:
Type: AWS::Lambda::Function
Properties:
Code:
ZipFile: |
import boto3
import cfnresponse
import os
db_cluster_identifier = os.environ['DB_CLUSTER_IDENTIFIER']
db_cluster_custom_endpoint1 = os.environ['DB_CLUSTER_CUSTOM_ENDPOINT1']
db_cluster_custom_endpoint2 = os.environ['DB_CLUSTER_CUSTOM_ENDPOINT2']
custom_endpoints = [
db_cluster_custom_endpoint1,
db_cluster_custom_endpoint2
]
client = boto3.client('rds')
CREATE = 'Create'
response_data = {}
def lambda_handler(event, context):
try:
if event['RequestType'] == CREATE:
response = client.describe_db_clusters(
DBClusterIdentifier=db_cluster_identifier
)
read_replica_instances = [member for member in response['DBClusters'][0]['DBClusterMembers'] if member['IsClusterWriter'] == False]
for (endpoint_name, read_replica_instance) in zip(custom_endpoints, read_replica_instances):
instance_name = read_replica_instance['DBInstanceIdentifier']
create_response = client.create_db_cluster_endpoint(
DBClusterIdentifier=db_cluster_identifier,
DBClusterEndpointIdentifier=endpoint_name,
EndpointType='READER',
StaticMembers=[instance_name])
print(create_response)
response_data[endpoint_name] = create_response['Endpoint']
cfnresponse.send(event, context, cfnresponse.SUCCESS, response_data)
except Exception as e:
print(e)
cfnresponse.send(event, context, cfnresponse.FAILED, response_data)
Environment:
Variables:
DB_CLUSTER_CUSTOM_ENDPOINT1: !Ref DBClusterCustomEndpoint1
DB_CLUSTER_CUSTOM_ENDPOINT2: !Ref DBClusterCustomEndpoint2
DB_CLUSTER_IDENTIFIER: !Ref DBClusterIdentifier
FunctionName: !Sub "${Prefix}-function"
Handler: !Ref Handler
Runtime: !Ref Runtime
Role: !GetAtt FunctionRole.Arn
Code language: YAML (yaml)There are no special items in the configuration of the function itself.
One point to mention is the environment variable.
Pass a custom endpoint name or Aurora cluster name to the function as an environment variable.
Refer to the value of event[‘RequestType’] to implement the processing according to the stack operation.
In this case, we will execute the process when this value is ‘Create’, that is, when the stack is created.
Execute the describe_db_clusters method to obtain the details of the Aurora cluster.
In the retrieved information, check the data about the DB instances that are members of the cluster and extract only the instances that are read replicas.
Create custom endpoints for them.
Specifically, execute the create_db_cluster_endpoint method.
Use cfnresponse.send to return a function execution completion message to the CloudFormation stack, passing the name of the created custom endpoint as a list type.
You will now be able to access these two addresses from your CloudFormation template.
IAM Role
Resources:
FunctionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: !Sub "${Prefix}-CreateAuroraCustomEndpointPolicy"
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- rds:CreateDBClusterEndpoint
- rds:DescribeDBClusters
Resource:
- !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:cluster:${DBClusterIdentifier}"
- !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:cluster-endpoint:*"
Code language: YAML (yaml)The Lambda function will access information about the Aurora cluster and create a custom endpoint, so allow the “rds:DescribeDBClusters” and “rds:CreateDBClusterEndpoint” actions.
Outputs Section
Outputs:
DBClusterCustomEndpoint1:
Value: !GetAtt
- CustomResource
- !Ref DBClusterCustomEndpoint1
DBClusterCustomEndpoint2:
Value: !GetAtt
- CustomResource
- !Ref DBClusterCustomEndpoint2
Code language: YAML (yaml)Get custom endpoint information created by a CloudFormation custom resource.
Combines the built-in functions Fn::GetAtt and Fn::Ref to retrieve custom endpoint information from custom resources.
Architecting
Use CloudFormation to build this environment and check the actual behavior.
Create CloudFormation stacks and check resources in stacks
Create a CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following page
After checking the resources in each stack, information on the main resources created this time is as follows
- Aurora cluster: saa-02-008-dbcluster
- EC2 instance: i-014c001efcf83db1e
The resource creation status is also checked from the AWS Management Console.
Check the Aurora cluster.
We can see that the cluster has been created and that there are three DB instances inside.
In the Endpoints section, we can see that two custom endpoints have been created.
We will verify the details of each.
The following is a summary of the information we have confirmed.
| Endpoint Name | DB Instance | Role |
| custom-endpoint1.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com | saa-02-008-dbinstance3 | Reader |
| custom-endpoint2.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com | saa-02-008-dbinstance2 | Reader |
Check the execution results of the Lambda function, a CloudFormation custom resource.
You can see that indeed the Lambda function was automatically executed and the custom endpoint we just checked was created.
Check Action
Now that everything is ready, let’s access the EC2 instance.
Use SSM Session Manager to access the instance.
% aws ssm start-session --target i-014c001efcf83db1e
...
sh-4.2$
Code language: Bash (bash)For more information on SSM Session Manager, please see the following page
Make sure the client package (MariaDB) for accessing Aurora is installed.
sh-4.2$ sudo yum list installed | grep maria
mariadb.aarch64 1:5.5.68-1.amzn2 @amzn2-core
mariadb-libs.aarch64 1:5.5.68-1.amzn2 installed
sh-4.2$ mysql -V
mysql Ver 15.1 Distrib 5.5.68-MariaDB, for Linux (aarch64) using readline 5.1
Code language: Bash (bash)The package has indeed installed.
Check the name resolution status of the custom endpoints with the nslookup command.
sh-4.2$ nslookup custom-endpoint1.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com
Server: 10.0.0.2
Address: 10.0.0.2#53
Non-authoritative answer:
custom-endpoint1.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com canonical name = saa-02-008-dbinstance3.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com.
Name: saa-02-008-dbinstance3.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com
Address: 10.0.4.161
Code language: Bash (bash)sh-4.2$ nslookup custom-endpoint2.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com
Server: 10.0.0.2
Address: 10.0.0.2#53
Non-authoritative answer:
custom-endpoint2.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com canonical name = saa-02-008-dbinstance2.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com.
Name: saa-02-008-dbinstance2.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com
Address: 10.0.3.221
Code language: Bash (bash)As confirmed in the AWS Management Console, you can see that a DB instance is associated with each custom endpoint.
Attempt to connect to the custom endpoint.
sh-4.2$ mysql -h custom-endpoint1.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com -P 3306 -u testuser -p testdb
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 20
Server version: 8.0.23 Source distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [testdb]>
MySQL [testdb]> create table testtable (datetime datetime);
ERROR 1836 (HY000): Running in read-only modeCode language: JavaScript (javascript)sh-4.2$ mysql -h custom-endpoint2.cluster-custom-cl50iikpthxs.ap-northeast-1.rds.amazonaws.com -P 3306 -u testuser -p testdb
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 20
Server version: 8.0.23 Source distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [testdb]>
MySQL [testdb]> create table testtable (datetime datetime);
ERROR 1836 (HY000): Running in read-only mode
Code language: Bash (bash)Both endpoints were able to connect successfully.
Because the role is Reader, an error occurred when I tried to perform a write.
Summary
We have seen how to create an Aurora custom endpoint using a CloudFormation custom resource.