AWS_EN

Connect to RDS from Lambda in VPC via RDS Proxy

スポンサーリンク
Connect to RDS from Lambda in VPC via RDS Proxy AWS_EN
スポンサーリンク
スポンサーリンク

Connect to RDS from Lambda in VPC via RDS Proxy

Consider a configuration where Lambda is deployed in a VPC and connects to RDS.
When accessing RDS from Lambda, it is best practice to connect via RDS Proxy rather than directly.

Many applications, including those built on modern serverless architectures, can have a large number of open connections to the database server, and may open and close database connections at a high rate, exhausting database memory and compute resources. Amazon RDS Proxy allows applications to pool and share connections established with the database, improving database efficiency and application scalability.

Amazon RDS Proxy

In this example, we will use the RDS Proxy to verify the configuration of connecting to the RDS from Lambda.

Environment

Diagram of connect from Lambda in VPC via RDS Proxy

Create three subnets in the VPC.
One is for Lambda. Normally, Lambda is created outside the VPC, but as shown here, it can also be created inside the VPC.
The other two are for RDS. In this case, one DB instance is created.

The Lambda function will not connect directly to the DB instance endpoint, but to the RDS Proxy endpoint.
Register the DB instance as a target group in the RDS Proxy so that it can connect to the DB instance via the RDS Proxy.
The function will be created in Python 3.8.

Create an API Gateway and set up a Lambda function as the backend.
The idea is to set up the Lambda function so that it is executed by accessing the API Gateway endpoint.

The following is a brief description of the behavior of the configuration we will create.
Create a table to store date/time data in a DB instance.
When an HTTP request arrives at the API Gateway, the back-end Lambda is called to retrieve the current date and time.
The acquired date/time data is stored in the DB instance via RDS Proxy.
In addition, all data stored in the DB instance is retrieved and returned as an HTTP response.

CloudFormation template files

The above configuration is built with CloudFormation.
The CloudFormation template is located at the following URL

awstut-fa/037 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Explanation of key points of the template files

Creating Proxy and Target Group

This page covers the contents related to RDS Proxy.
For RDS itself, please refer to the following page

First, check the RDS Proxy itself.

Resources: DBProxy: Type: AWS::RDS::DBProxy Properties: Auth: - IAMAuth: DISABLED AuthScheme: SECRETS SecretArn: !Ref Secret DBProxyName: !Sub "${Prefix}-DBProxy" EngineFamily: !Ref DBProxyEngineFamily IdleClientTimeout: 120 RequireTLS: false RoleArn: !GetAtt DBProxyRole.Arn VpcSecurityGroupIds: - !Ref DBProxySecurityGroup VpcSubnetIds: - !Ref DBSubnet1 - !Ref DBSubnet2
Code language: YAML (yaml)

Set the authentication method for connecting to the RDS (DB instance) in the Auth property.
In this case, we will use the secrets of the Secrets Manager for authentication.

The EngineFamily property specifies the type of DB instance to connect to.
This time, set “MYSQL” since it is a MySQL type DB instance.

Set the security group to be applied to RDS Proxy in the VpcSecurityGroupIds property.
This time we will connect to a MySQL type DB instance, so we will apply a security group that allows inbound 3306/tcp communication from the Lambda subnet.

In the VpcSubnetIds property, specify the subnet to which you want to associate the RDS Proxy.
The target subnet will be the one where the DB instance is located.

The DB instance to which the RDS Proxy will be connected is defined in the target group.

Resources: DBProxyTargetGroup: Type: AWS::RDS::DBProxyTargetGroup Properties: DBProxyName: !Ref DBProxy DBInstanceIdentifiers: - !Ref DBInstance TargetGroupName: default ConnectionPoolConfigurationInfo: MaxConnectionsPercent: 100 MaxIdleConnectionsPercent: 50 ConnectionBorrowTimeout: 120
Code language: YAML (yaml)

Specify the DB instance in the DBInstanceIdentifiers property.

Check the Secrets Manager secret.

Resources: Secret: Type: AWS::SecretsManager::Secret Properties: Name: !Sub "${Prefix}-Secret" SecretString: !Sub '{"username":"${DBMasterUsername}","password":"${DBMasterUserPassword}"}'
Code language: YAML (yaml)

Specify the username and password set for the DB instance in JSON format to create the secret.

Create an IAM role to be associated with the RDS Proxy.

Resources: DBProxyRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: sts:AssumeRole Principal: Service: - rds.amazonaws.com Policies: - PolicyName: !Sub "${Prefix}-DBProxyPolicy" PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: secretsmanager:GetSecretValue Resource: !Ref Secret - Effect: Allow Action: kms:Decrypt Resource: '*' # use default key.
Code language: YAML (yaml)

Grant permission to retrieve the encrypted value from the secret defined earlier and to decrypt the KMS key used for this purpose.
The following page was used to set this up

Getting started with RDS Proxy - Amazon Relational Database Service
Learn how to create an RDS Proxy and use it to connect to a database.

Lambda function to connect to RDS Proxy

Check the Lambda function.

Resources: Function: Type: AWS::Lambda::Function Properties: Environment: Variables: DB_ENDPOINT_PORT: !Ref MySQLPort DB_NAME: !Ref DBName DB_PASSWORD: !Ref DBMasterUserPassword DB_PROXY_ENDPOINT_ADDRESS: !Ref DBProxyEndpointAddress DB_TABLENAME: !Ref DBTableName DB_USER: !Ref DBMasterUsername REGION: !Ref AWS::Region Code: S3Bucket: !Ref CodeS3Bucket S3Key: !Ref CodeS3Key FunctionName: !Sub "${Prefix}-function" Handler: index.lambda_handler Runtime: python3.8 Role: !GetAtt FunctionRole.Arn Timeout: 10 VpcConfig: SecurityGroupIds: - !Ref FunctionSecurityGroup SubnetIds: - !Ref FunctionSubnet
Code language: YAML (yaml)

This time, we will create a Lambda function from a deployment package (Zip file) uploaded to an S3 bucket.
Please refer to the following page for basic information on Lambda functions.

There are two key points.

The first is to set environment variables.
The Environment and Variables properties allow you to define variables that can be used within the function.
In this case, we will define the RDS Proxy endpoint to connect to the RDS Proxy and DB instances, user names and passwords, table names, etc.

The second is to configure settings related to the VPC.
This time, since the Lambda function will be set up inside the VPC, we will specify the subnet where it will be installed and the security group to be applied.
The content of the security group is to allow HTTPS (443/tcp) inbound communication.

Check the script to be executed by the function.

import boto3 import datetime import json import mysql.connector import os db_name = os.environ['DB_NAME'] db_password = os.environ['DB_PASSWORD'] db_proxy_endpoint_address = os.environ['DB_PROXY_ENDPOINT_ADDRESS'] db_tablename = os.environ['DB_TABLENAME'] db_user = os.environ['DB_USER'] region = os.environ['REGION'] def lambda_handler(event, context): conn = mysql.connector.connect( host=db_proxy_endpoint_address, port=db_endpoint_port, user=db_user, password=db_password, database=db_name ) cur = conn.cursor() table_sql = 'create table if not exists {db}.{tbl} (dt datetime);'.format( db=db_name, tbl=db_tablename ) cur.execute(table_sql) now = datetime.datetime.now() now_str = now.strftime('%Y-%m-%d %H:%M:%S') write_sql = 'insert into {tbl} values ("{now}");'.format( tbl=db_tablename, now=now_str ) cur.execute(write_sql) cur.close() conn.commit() cur = conn.cursor() read_sql = 'select * from {tbl};'.format(tbl=db_tablename) cur.execute(read_sql) content = [record[0].strftime('%Y-%m-%d %H:%M:%S') for record in cur] cur.close() conn.close() return { 'statusCode': 200, 'body': json.dumps(content, indent=2) }
Code language: Python (python)

This page will focus on the RDS Proxy, so we will not go into details of the Lambda function itself.
We will review the points for connecting to the RDS Proxy.

Get environment variables from os.environ.
These are the variables set when defining the Lambda function described above.
A particularly important variable is the RDS Proxy endpoint.
The connection is made towards the RDS Proxy endpoint, not the endpoint created in the DB instance.

To connect to MySQL from Python, we will use mysql-connector-python, the official MySQL driver.
When creating the Zip file to deploy, download the package locally and deploy it together using the following command.

pip3 install mysql-connector-python -t . zip -r deploy.zip .
Code language: Bash (bash)

Here is a brief description of the code.

  1. connect to the DB instance via RDS Proxy by referencing environment variables.
  2. create a table to store the current date and time for the first time only. 3. retrieve the current date and time.
  3. retrieve the current date/time and store it in the table. 4.
  4. retrieve the date/time data stored in the table and return it to the user.

Set up Lambda for API Gateway backend

To create an API Gateway, the following five resources must be created

  • API Gateway main body
  • Stage
  • Integration
  • Root
  • IAM Roles/Permissions

Please see the following page for details.

Architecting

Use CloudFormation to build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • DB instance: fa-037-dbinstance
  • RDS Proxy: fa-037-dbproxy
  • API Gateway endpoint: https://oeebpxb1r2.execute-api.ap-northeast-1.amazonaws.com/

The resource creation status is also checked from the AWS Management Console.
Check the RDS Proxy.

RDS Proxy Detail 1
RDS Proxy Detail 2

RDS Proxy has been successfully created.

An endpoint has been created.
The Lambda function will access this endpoint.

A DB instance is associated with the target group of the RDS Proxy.
When you connect to this RDS Proxy, you are connecting to the associated DB instance.

Authentication is configured using the Secrets Manager.

Confirmation of Operation

Now that everything is ready, access the API Gateway.

RDS Proxy test 1
RDS Proxy test 2

The response is returned normally.
Date and time data is displayed more frequently each time it is accessed.
In this way, when accessing a DB instance from Lambda, a connection can be made via RSD Proxy.

Summary

We have confirmed how to connect from Lambda to DB instances via RDS Proxy.

タイトルとURLをコピーしました