DVA_EN

Connect to RDS outside of Lambda handler method to improve performance

スポンサーリンク
Connect to RDS outside of Lambda handler method to improve performance DVA_EN
スポンサーリンク
スポンサーリンク

Connecting to RDS inside and outside of Lambda function handler methods to measure performance

This section is about refactoring, which is the scope of the AWS DBA.

One of the best practices for connecting to RDS from a Lambda function is to connect to RDS outside of the handler method.

Objects declared outside of the function’s handler method remain initialized, providing additional optimization when the function is invoked again. For example, if your Lambda function establishes a database connection, instead of reestablishing the connection, the original connection is used in subsequent invocations.

AWS Lambda execution environment

In this page, we will create two Lambda functions and measure how much performance can be improved by connecting to RDS inside and outside the handler methods, respectively.

Environment

Diagram of connect to RDS outside of Lambda handler method to improve performance.

Four subnets will be created in the VPC.
Two are for Lambda.
The other two are for RDS. In this case, we will create one MySQL type DB instance.

Create an RDS Proxy.
By registering a DB instance to the target group of the RDS Proxy, the Lambda function can connect to the DB instance via the RDS Proxy.

Create two Lambda functions.
After connecting to the DB instance, both functions will store and return the current date and time.
The difference between the two functions is the implementation of the part that connects to the DB instance.
Function 1 connects to the DB instance inside the handler method, while Function 2 connects externally.
The runtime environment for the functions is Python 3.8, and Function URLs are created.

Apache Bench is used for verification.
For each Function URL, send 100 requests and measure the time taken to process each request.

CloudFormation Template Files

The above configuration is built using CloudFormation.
The CloudFormation template is located at the following URL

awstut-dva/04/001 at main · awstut-an-r/awstut-dva
Contribute to awstut-an-r/awstut-dva development by creating an account on GitHub.

Explanation of key points of template files

This page focuses on the content related to how to connect to a DB instance in a Lambda function.

For information on how to connect to RDS from a Lambda function via RDS Proxy, please refer to the following page

Use mysql-connector-python to connect to a MySQL type DB instance.
Create a Lambda layer and place this package in it.
For more information on the Lambda layer, see the following page

The created Lambda function is executed using a Function URL.
For more information on Function URLs, please refer to the following page

Connecting to DB instance inside handler method

Check the code of the function (function 1) that connects inside the handler method.

import datetime import mysql.connector import os db_endpoint_port = os.environ['DB_ENDPOINT_PORT'] db_name = os.environ['DB_NAME'] db_password = os.environ['DB_PASSWORD'] db_proxy_endpoint_address = os.environ['DB_PROXY_ENDPOINT_ADDRESS'] db_tablename = os.environ['DB_TABLENAME'] db_user = os.environ['DB_USER'] region = os.environ['REGION'] def lambda_handler(event, context): conn = mysql.connector.connect( host=db_proxy_endpoint_address, port=db_endpoint_port, user=db_user, password=db_password, database=db_name ) cur = conn.cursor() now = datetime.datetime.now() now_str = now.strftime('%Y-%m-%d %H:%M:%S') write_sql = 'insert into {tbl} values ("{now}");'.format( tbl=db_tablename, now=now_str ) cur.execute(write_sql) cur.close() conn.commit() return { 'statusCode': 200, 'body': str(now) }
Code language: Python (python)

Create an object for connection to the DB instance with mysql.connector.connect.
As you can see, it is executed within the handler method.

Connecting to DB instance outside handler method

The following is the code for the function that connects outside the handler method (function 2).

import datetime import mysql.connector import os db_endpoint_port = os.environ['DB_ENDPOINT_PORT'] db_name = os.environ['DB_NAME'] db_password = os.environ['DB_PASSWORD'] db_proxy_endpoint_address = os.environ['DB_PROXY_ENDPOINT_ADDRESS'] db_tablename = os.environ['DB_TABLENAME'] db_user = os.environ['DB_USER'] region = os.environ['REGION'] conn = mysql.connector.connect( host=db_proxy_endpoint_address, port=db_endpoint_port, user=db_user, password=db_password, database=db_name ) def lambda_handler(event, context): cur = conn.cursor() now = datetime.datetime.now() now_str = now.strftime('%Y-%m-%d %H:%M:%S') write_sql = 'insert into {tbl} values ("{now}");'.format( tbl=db_tablename, now=now_str ) cur.execute(write_sql) cur.close() conn.commit() return { 'statusCode': 200, 'body': str(now) }
Code language: Python (python)

The code is almost identical to the previous one.
The only difference is that the scope in which the connection object is created in mysql.connector.connect is the global scope. This means that the object is defined as a global variable.

(Reference) Initialization process of DB instance

In order to write the current date and time, it is necessary to create a table for date and time data in the DB instance.
This time, a function for initialization processing (Function 3) is prepared and executed.
The code of Function 3 is shown below for reference.

import datetime import json import mysql.connector import os db_endpoint_port = os.environ['DB_ENDPOINT_PORT'] db_name = os.environ['DB_NAME'] db_password = os.environ['DB_PASSWORD'] db_proxy_endpoint_address = os.environ['DB_PROXY_ENDPOINT_ADDRESS'] db_tablename = os.environ['DB_TABLENAME'] db_user = os.environ['DB_USER'] region = os.environ['REGION'] def lambda_handler(event, context): conn = mysql.connector.connect( host=db_proxy_endpoint_address, port=db_endpoint_port, user=db_user, password=db_password, database=db_name ) cur = conn.cursor() table_sql = 'create table if not exists {db}.{tbl} (dt datetime);'.format( db=db_name, tbl=db_tablename ) cur.execute(table_sql) now = datetime.datetime.now() now_str = now.strftime('%Y-%m-%d %H:%M:%S') write_sql = 'insert into {tbl} values ("{now}");'.format( tbl=db_tablename, now=now_str ) cur.execute(write_sql) cur.close() conn.commit() cur = conn.cursor() read_sql = 'select * from {tbl};'.format(tbl=db_tablename) cur.execute(read_sql) content = [record[0].strftime('%Y-%m-%d %H:%M:%S') for record in cur] cur.close() conn.close() return { 'statusCode': 200, 'body': json.dumps(content, indent=2) }
Code language: Python (python)

If the table does not exist, the content is to create a new table.
It then works to retrieve and return all stored data.

Architecting

Use CloudFormation to build this environment and check the actual behavior.

Prepare deployment package for Lambda function

There are three ways to create a Lambda function.
In this case, we will choose the method of uploading a deployment package to an S3 bucket.
For more information, please refer to the following page

Create a CloudFormation stacks and check resources in stacks

Create a CloudFormation stack.
For more information on how to create stacks and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • Function URL for Function 1: https://42lzbfe7dehb5jcnphzp555hly0evucx.lambda-url.ap-northeast-1.on.aws/
  • Function URL for function 2: https://6oy3eianevtfz7xw3dopmbnnrm0ijdyr.lambda-url.ap-northeast-1.on.aws/
  • Function URL for function 3: https://7poj6cjntirj24de2hlbz2xeia0katuf.lambda-url.ap-northeast-1.on.aws/

Confirmation of Operation

Preliminary Preparation

First, create a table in the DB instance to store date/time data.
Access the Function URL of Function 3 and execute the function.

The result of Lambda Function 3 invocation.

The table is created and all currently stored data is returned.
The initialization process is successfully completed.

Verification 1: Connecting to DB instance inside handler method

Now that we are ready, we can start the verification.
At a glance, we check the operation of function 1.

The result of Lambda Function 1 invocation.

The current date and time are returned.
The function is working normally.

We evaluate the performance of this function using Apache Bench.
The following is the execution result.

% ab -n 100 https://42lzbfe7dehb5jcnphzp555hly0evucx.lambda-url.ap-northeast-1.on.aws/ This is ApacheBench, Version 2.3 <$Revision: 1879490 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 42lzbfe7dehb5jcnphzp555hly0evucx.lambda-url.ap-northeast-1.on.aws (be patient).....done Server Software: Server Hostname: 42lzbfe7dehb5jcnphzp555hly0evucx.lambda-url.ap-northeast-1.on.aws Server Port: 443 SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES128-GCM-SHA256,2048,128 Server Temp Key: ECDH P-256 256 bits TLS Server Name: 42lzbfe7dehb5jcnphzp555hly0evucx.lambda-url.ap-northeast-1.on.aws Document Path: / Document Length: 26 bytes Concurrency Level: 1 Time taken for tests: 50.102 seconds Complete requests: 100 Failed requests: 0 Total transferred: 27800 bytes HTML transferred: 2600 bytes Requests per second: 2.00 [#/sec] (mean) Time per request: 501.024 [ms] (mean) Time per request: 501.024 [ms] (mean, across all concurrent requests) Transfer rate: 0.54 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 50 57 4.5 56 78 Processing: 377 443 96.2 426 1307 Waiting: 377 443 96.2 426 1307 Total: 427 501 96.0 482 1359 Percentage of the requests served within a certain time (ms) 50% 482 66% 507 75% 512 80% 512 90% 534 95% 580 98% 717 99% 1359 100% 1359 (longest request)
Code language: plaintext (plaintext)

The average processing speed was 501 ms.

Verification 2: Connecting to DB instance outside handler method

The same method is used for verification.
First, check the behavior of function 2.

The result of Lambda Function 2 invocation.

The current date and time are returned.
The function is working normally.

We evaluate the performance of this function using Apache Bench.
The following is the execution result.

% ab -n 100 https://6oy3eianevtfz7xw3dopmbnnrm0ijdyr.lambda-url.ap-northeast-1.on.aws/ This is ApacheBench, Version 2.3 <$Revision: 1879490 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 6oy3eianevtfz7xw3dopmbnnrm0ijdyr.lambda-url.ap-northeast-1.on.aws (be patient).....done Server Software: Server Hostname: 6oy3eianevtfz7xw3dopmbnnrm0ijdyr.lambda-url.ap-northeast-1.on.aws Server Port: 443 SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES128-GCM-SHA256,2048,128 Server Temp Key: ECDH P-256 256 bits TLS Server Name: 6oy3eianevtfz7xw3dopmbnnrm0ijdyr.lambda-url.ap-northeast-1.on.aws Document Path: / Document Length: 26 bytes Concurrency Level: 1 Time taken for tests: 11.825 seconds Complete requests: 100 Failed requests: 0 Total transferred: 27800 bytes HTML transferred: 2600 bytes Requests per second: 8.46 [#/sec] (mean) Time per request: 118.255 [ms] (mean) Time per request: 118.255 [ms] (mean, across all concurrent requests) Transfer rate: 2.30 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 47 57 5.1 56 82 Processing: 41 61 75.4 51 798 Waiting: 41 61 75.4 51 798 Total: 90 118 76.2 109 860 Percentage of the requests served within a certain time (ms) 50% 109 66% 113 75% 115 80% 118 90% 124 95% 137 98% 180 99% 860 100% 860 (longest request)
Code language: plaintext (plaintext)

The average processing speed was 118 ms.

Summary of Verification

The following is a summary of the results of this verification.

Speed Comparison Results

All indicators show that the outside connection is faster.
The average value shows that the outside connection was 4.2 times faster than the inside connection in this verification.

Summary

We have verified the extent to which performance varies depending on whether the Lambda function connects to the RDS inside or outside the handler method.
In this verification, we found that connecting outside the handler method is about 4.2 times faster than connecting inside.

タイトルとURLをコピーしました