Create RDS Cross-Region Read Replica using CloudFormation
One of the AWS SAA questions is about designing resilient architectures.
In the following pages, we introduced RDS read replicas.
In the above page, the lead replicas were placed on the same region, but AWS official mentions the location of the replicas as follows.
With Amazon RDS, you can create a read replica in a different AWS Region from the source DB instance.
Creating a read replica in a different AWS Region
This page uses CloudFormation to create a cross-region read replica.
Environment
Create VPCs in two regions.
In one region, create a DB instance of RDS.
In the other region, create a read replica of this instance.
The engine of the DB instance is MySQL.
Place an EC2 instance in each VPC.
Use it as a client to connect to the DB instance.
The instance is the latest version of Amazon Linux 2.
On the read replica side of the region, create a Lambda function and configure this as a CloudFormation custom resource.
The function’s action is to change the security group of the read replica.
The runtime environment for the function is Python 3.8.
Create two types of VPC endpoints.
The first is for SSM.
SSM Session Manager to connect to an EC2 instance in a private subnet.
The second is for S3.
This is for accessing yum repositories built on S3 buckets.
CloudFormation template files
The above configuration is built with CloudFormation.
The CloudFormation templates are placed at the following URL
https://github.com/awstut-an-r/awstut-saa/tree/main/01/004
Explanation of key points of template files
Cross Region Read Replica
Resources:
DBInstance:
Type: AWS::RDS::DBInstance
DeletionPolicy: Delete
Properties:
AvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone}"
DBInstanceIdentifier: !Sub "${Prefix}-dbinstance"
DBSubnetGroupName: !Ref DBSubnetGroup
SourceDBInstanceIdentifier: !Sub "arn:aws:rds:${SourceDBInstanceRegion}:${AWS::AccountId}:db:${SourceDBInstanceIdentifier}"
Code language: YAML (yaml)For basic information on read replicas, please refer to the page mentioned at the beginning of this document.
The key setting for creating a cross-region read replica is the SourceDBInstanceIdentifier property.
Specify the ARN of the primary instance located in a different region for this property.
Custom Resource and Lambda Function
Use custom resources to change the security group of the read replica when creating the CloudForamtion stack.
This is due to the following specifications
The read replica uses the default security group.
Creating a read replica in a different AWS Region
This means that when a read replica is created, the security group cannot be set and the default one is applied.
Therefore, after the read replica is created, the appropriate security group must be manually re-configured.
This time this is done with a Lambda function associated with a custom resource.
Custom Resource
Resources:
CustomResource:
Type: Custom::CustomResource
Properties:
ServiceToken: !GetAtt Function.Arn
Code language: YAML (yaml)For basic information on custom resources, please see the following pages.
Create a custom resource and specify the Lambda function described below.
Lambda Function
Resources:
Function:
Type: AWS::Lambda::Function
Properties:
Environment:
Variables:
DB_INSTANCE: !Ref DBInstance
DB_SECURITY_GROUP: !Ref DBSecurityGroup
REGION: !Ref AWS::Region
Code:
ZipFile: |
import boto3
import cfnresponse
import os
db_instance = os.environ['DB_INSTANCE']
db_security_group = os.environ['DB_SECURITY_GROUP']
region = os.environ['REGION']
CREATE = 'Create'
response_data = {}
def lambda_handler(event, context):
try:
if event['RequestType'] == CREATE:
client = boto3.client('rds', region_name=region)
response = client.modify_db_instance(
DBInstanceIdentifier=db_instance,
VpcSecurityGroupIds=[
db_security_group
]
)
print(response)
cfnresponse.send(event, context, cfnresponse.SUCCESS, response_data)
except Exception as e:
print(e)
cfnresponse.send(event, context, cfnresponse.FAILED, response_data)
FunctionName: !Sub "${Prefix}-function"
Handler: index.lambda_handler
Runtime: !Ref Runtime
Role: !GetAtt FunctionRole.Arn
Timeout: !Ref Timeout
Code language: YAML (yaml)The code to be executed by the Lambda function in inline format.
For more information, please refer to the following page.
The code to be executed is to execute the modify_db_instance method of the RDS client object and change the security group of the read replica.
Implement this process so that it is invocated when the stack is created.
IAM Role
Resources:
FunctionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
Policies:
- PolicyName: ModifyDBInstancePolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- rds:ModifyDBInstance
Resource:
- !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:pg:*"
- !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:secgrp:*"
- !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:og:*"
- !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:db:*"
Code language: YAML (yaml)IAM role for Lambda functions.
function to execute the modify_db_instance method.
In this case, the following page was used as a reference for setting up the system.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/security_iam_id-based-policy-examples.html
(Reference) EC2 instance
Resources:
Instance:
Type: AWS::EC2::Instance
Properties:
IamInstanceProfile: !Ref InstanceProfile
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
NetworkInterfaces:
- DeviceIndex: 0
SubnetId: !Ref InstanceSubnet
GroupSet:
- !Ref InstanceSecurityGroup
UserData: !Base64 |
#!/bin/bash -xe
yum update -y
yum install -y mariadb
Code language: YAML (yaml)To access a DB instance from an EC2 instance, a client package must be prepared.
In this case, we will use user data to install the package.
For more information on user data, please see the following pages
To access a MySQL-type RDS from Amazon Linux 2, the mariadb package must be installed.
Please see the following page for client packages to connect to various RDS from Amazon Linux 2.
Architecting
Use CloudFormation to build this environment and check its actual behavior.
Create CloudFormation stacks and check the resources in the stacks
Create CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following pages.
This time, the following command will create a stack in two regions.
$ aws cloudformation create-stack \
--stack-name saa-01-004-1 \
--template-url https://[bucket-url]/saa-01-004-1.yaml \
--capabilities CAPABILITY_IAM \
--region ap-northeast-1
$ aws cloudformation create-stack \
--stack-name saa-01-004-2 \
--template-url https://[bucket-url]/saa-01-004-2.yaml \
--capabilities CAPABILITY_IAM \
--region us-east-1
Code language: Bash (bash)After reviewing the resources in each stack, information on the main resources created in this case is as follows
- EC2 instance in Tokyo region: i-01dff604f44d82f1d
- RDS endpoint in Tokyo Region: saa-01-004-1-dbinstance.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com
- EC2 instance in Virginia Region: i-0ddbcbbbaffb35b48
- RDS endpoint for Virginia Region: saa-01-004-2-dbinstance.csx6ivwzoymt.us-east-1.rds.amazonaws.com
- Lambda function for CloudFormation custom resource: saa-01-004-2
Check each resource from the AWS Management Console.
Check the DB instance in the Tokyo Region.
The Role item indicates that this DB instance is created as the primary instance.
The replication item shows that the DB instance in the Virginia Region (described below) exists as a read replica.
Check the DB instance in the Virginia Region.
The Role entry shows that this DB instance was created as a read replica instance.
The Replication item shows that replication is set up with the primary instance we just checked as the source.
Also, if you look at the Security Groups section, you will see that the security group created this time has been specified.
This means that the CloudFormation custom resource has successfully worked and replaced the default one with the appropriate one.
Check the execution log of the Lambda function for custom resources.
Indeed, the Lambda function is executed by the CloudFormation custom resource when the CloudFormation stack is created.
Action Check
Access to the primary instance in the Tokyo Region
Now that everything is ready, connect to the primary instance.
Connection to the DB instance is made from the EC2 instance in the Tokyo region.
SSM Session Manager is used to access the instance.
% aws ssm start-session --target i-01dff604f44d82f1d --region ap-northeast-1
...
sh-4.2$
Code language: Bash (bash)For more information on SSM Session Manager, please refer to the following page.
Check the execution status of the EC2 instance initialization process by user data.
sh-4.2$ sudo yum list installed | grep mariadb
mariadb.aarch64 1:5.5.68-1.amzn2 @amzn2-core
mariadb-libs.aarch64 1:5.5.68-1.amzn2 installed
sh-4.2$ mysql -V
mysql Ver 15.1 Distrib 5.5.68-MariaDB, for Linux (aarch64) using readline 5.1
Code language: Bash (bash)You can see that the MySQL client package has been successfully installed.
Use this client package to connect to the DB instance.
sh-4.2$ mysql -h saa-01-004-1-dbinstance.cl50iikpthxs.ap-northeast-1.rds.amazonaws.com -P 3306 -u testuser -p testdb
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 8.0.27 Source distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [testdb]>
Code language: Bash (bash)Successfully connected.
Create a test table and save the test data.
MySQL [testdb]> CREATE TABLE planet (id INT UNSIGNED AUTO_INCREMENT, name VARCHAR(30), PRIMARY KEY(id));
Query OK, 0 rows affected (0.03 sec)
MySQL [testdb]> INSERT INTO planet (name) VALUES ("Mercury");
Query OK, 1 row affected (0.01 sec)
MySQL [testdb]>
MySQL [testdb]> select * from planet;
+----+---------+
| id | name |
+----+---------+
| 1 | Mercury |
+----+---------+
1 row in set (0.01 sec)
Code language: Bash (bash)Data was saved successfully.
Access to the read replica instance in the Virginia Region
Then connect to the cross-region read replica instance.
This also connects to the lead replica from the EC2 instance in the Virginia region.
% aws ssm start-session --target i-0ddbcbbbaffb35b48 --region us-east-1
...
sh-4.2$Code language: Bash (bash)Instances could be accessed.
Connects to the read replica instance.
sh-4.2$ mysql -h saa-01-004-2-dbinstance.csx6ivwzoymt.us-east-1.rds.amazonaws.com -P 3306 -u testuser -p testdb
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 22
Server version: 8.0.27 Source distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [testdb]>Code language: Bash (bash)I was able to connect to the read replica.
Try it out and see how it works on read/write.
MySQL [testdb]> show tables;
+----------------+
| Tables_in_test |
+----------------+
| planet |
+----------------+
1 row in set (0.00 sec)
MySQL [testdb]> select * from planet;
+----+---------+
| id | name |
+----+---------+
| 1 | Mercury |
+----+---------+
1 row in set (0.01 sec)
MySQL [testdb]> INSERT INTO planet (name) VALUES ("Venus");
ERROR 1290 (HY000): The MySQL server is running with the --read-only option so it cannot execute this statementCode language: Bash (bash)The read was successful.
The contents written by the primary instance located in the Tokyo region are also reflected in the read replica located in the Virginia region.
Writing, on the other hand, failed.
This is because this instance is read-only.
Summary
How to create an RDS cross-region read replica and how it works.