Use CFNs WaitCondition to wait for the Lambda deploy package to build
Consider creating a Lambda function using CloudFormation.
As described in the following pages, there are three patterns for creating functions.
This page will focus specifically on how to place a ZIP file deployment package in an S3 bucket.
In CloudFormation, when creating a Lambda function in this way, the deployment package must be placed in the S3 bucket before the function resource is generated.
But depending on what you include in the package, it may take some time to build and you may not be able to create the function resource in time.
So this time we will use CloudFormation’s WaitCondition.
Specifically, we will wait to generate the Lambda function resource until the deployment package is created.
Environment
The following flow creates Lambda function 3.
- Create a deployment package for the function in CodeBuild.
- Place the package in an S3 bucket.
- Specify EventBridge as the event notification destination for the S3 bucket.
- Create EventBridge rules for S3 object creation and trigger Lambda function 2.
- Function 2 signals the WaitCondition of CloudFormation.
- Upon receiving a WaitCondition signal, Lambda function 3 is created using the deploy package in the S3 bucket.
The kicker to start a CodeBuild project is to use a CloudFormation custom resource.
Specifically, we trigger the CodeBuild project with a Lambda function tied to the custom resource.
The runtime environment for Lambda functions is Python 3.8.
CloudFormation template files
The above configuration is built with CloudFormation.
The CloudFormation templates are placed at the following URL
https://github.com/awstut-an-r/awstut-fa/tree/main/142
Explanation of key points of template files
S3 bucket
Resources:
Bucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: Private
BucketName: !Ref Prefix
NotificationConfiguration:
EventBridgeConfiguration:
EventBridgeEnabled: true
Code language: YAML (yaml)S3 bucket in which to place the deployment package.
The key point is the event notification function.
In this configuration, we need to execute Lambda function 2 triggered by the package being placed in the S3 bucket.
Enable the EventBridge notification feature in the NotificationConfiguration property.
In this case, we have selected EventBridge as the event notification destination, but you can also invoke the Lambda function directly.
For details, please refer to the following page.
CodeBuild
Resources:
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: NO_ARTIFACTS
Cache:
Type: NO_CACHE
Environment:
ComputeType: !Ref ProjectEnvironmentComputeType
EnvironmentVariables:
- Name: BUCKET_NAME
Type: PLAINTEXT
Value: !Ref BucketName
- Name: SOURCE_BUNDLE_NAME
Type: PLAINTEXT
Value: !Ref SourceBundleName
- Name: SOURCE_FILE_NAME
Type: PLAINTEXT
Value: !Ref SourceFileName
Image: !Ref ProjectEnvironmentImage
ImagePullCredentialsType: CODEBUILD
Type: !Ref ProjectEnvironmentType
PrivilegedMode: true
LogsConfig:
CloudWatchLogs:
Status: DISABLED
S3Logs:
Status: DISABLED
Name: !Ref Prefix
ServiceRole: !GetAtt CodeBuildRole.Arn
Source:
Type: NO_SOURCE
BuildSpec: !Sub |
version: 0.2
phases:
pre_build:
commands:
- |
cat << EOF > $SOURCE_FILE_NAME
def lambda_handler(event, context):
return 'hogehoge'
EOF
build:
commands:
- zip $SOURCE_BUNDLE_NAME -r * .[^.]*
- sleep 300
post_build:
commands:
- aws s3 cp $SOURCE_BUNDLE_NAME s3://$BUCKET_NAME/
Visibility: PRIVATE
Code language: YAML (yaml)Use CodeBuild to create a deployment package for the Lambda function.
Describe the contents of buildspec.yaml in the BuildSpec property.
Create the source code for the function in the pre_build phase.
Create the deployment package in the build phase.
Specifically, zip the files created in the previous phase.
In addition, this build should be time-consuming, so we will wait 5 minutes with the sleep command.
Upload the deployment package to the S3 bucket during the post_build phase.
The following are the IAM roles for CodeBuild.
Resources:
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- codebuild.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: PutS3ObjectPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- s3:PutObject
Resource:
- !Sub "arn:aws:s3:::${BucketName}/*"
Code language: YAML (yaml)Grant permissions to place objects in S3 buckets.
Lambda function 1
Resources:
CustomResource:
Type: Custom::CustomResource
Properties:
ServiceToken: !GetAtt Function1.Arn
Function1:
Type: AWS::Lambda::Function
Properties:
Architectures:
- !Ref Architecture
Environment:
Variables:
CODEBUILD_PROJECT: !Ref CodeBuildProject
Code:
ZipFile: |
import boto3
import cfnresponse
import os
codebuild_project = os.environ['CODEBUILD_PROJECT']
CREATE = 'Create'
response_data = {}
client = boto3.client('codebuild')
def lambda_handler(event, context):
try:
if event['RequestType'] == CREATE:
response = client.start_build(
projectName=codebuild_project
)
print(response)
cfnresponse.send(event, context, cfnresponse.SUCCESS, response_data)
except Exception as e:
print(e)
cfnresponse.send(event, context, cfnresponse.FAILED, response_data)
FunctionName: !Sub "${Prefix}-function-01"
Handler: !Ref Handler
Runtime: !Ref Runtime
Role: !GetAtt FunctionRole1.Arn
Code language: YAML (yaml)Lambda function 1 is used to automatically start a CodeBuild project.
Specifically, associate this function with a CloudFormation custom resource.
For more information on CloudFormation custom resources, please see the following pages
This time, when creating the custom resource, we will execute the start_build method of the boto3 client object for CodeBuild.
This will automatically start the CodeBuild project.
The following is the IAM role for this function.
Resources:
FunctionRole1:
Type: AWS::IAM::Role
DeletionPolicy: Delete
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: StartCodeBuildPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- codebuild:StartBuild
Resource:
- !Sub "arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:project/${CodeBuildProject}"
Code language: YAML (yaml)Authorization to start a CodeBuild project.
EventBridge
Resources:
EventsRule:
Type: AWS::Events::Rule
Properties:
EventBusName: !Ref EventBusName
EventPattern:
source:
- aws.s3
detail-type:
- Object Created
detail:
bucket:
name:
- !Ref BucketName
Name: !Sub "${Prefix}-EventsRule"
State: ENABLED
Targets:
- Arn: !GetAtt Function2.Arn
Id: !Ref Function2
Code language: YAML (yaml)EventBridge Rule.
When an object is created in the aforementioned S3 bucket, Lambda function 2 described below is executed.
If you specify a Lambda function as the target, the behavior is that EventBridge calls the function.
Therefore, it is necessary to authorize EventBridge to call the function.
Resources:
EventsRulePermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !Ref Function2
Principal: events.amazonaws.com
SourceArn: !GetAtt EventsRule.Arn
Code language: YAML (yaml)WaitCondition
The official AWS website explains WaitCondition as follows
You can use the wait condition and wait condition handle to make AWS CloudFormation pause the creation of a stack and wait for a signal before it continues to create the stack.
Creating wait conditions in a template
Specifically, both resources are defined as follows
Resources:
WaitConditionHandle:
Type: AWS::CloudFormation::WaitConditionHandle
WaitCondition:
Type: AWS::CloudFormation::WaitCondition
Properties:
Handle: !Ref WaitConditionHandle
Timeout: !Ref WaitConditionTimeout
Code language: YAML (yaml)We have specified 600 for the Timeout property of WaitCondition.
This will wait for signal reception for up to 10 minutes.
If no signal is received after 10 minutes of waiting, a failure will occur and the CloudFormation stack will roll back.
Lambda Function 2
Resources:
Function2:
Type: AWS::Lambda::Function
Properties:
Architectures:
- !Ref Architecture
Environment:
Variables:
SIGNAL_URL: !Ref WaitConditionHandle
Code:
ZipFile: |
import json
import os
import urllib3
import uuid
def lambda_handler(event, context):
body = json.dumps({
"Status": "SUCCESS",
"Reason": "Lambda Deploy Package Setup Successed",
"UniqueId": str(uuid.uuid4()),
"Data": "Lambda Deploy Package Setup Successed"
})
http = urllib3.PoolManager()
http.request('PUT', os.environ['SIGNAL_URL'], body=body)
FunctionName: !Sub "${Prefix}-function-02"
Handler: !Ref Handler
Runtime: !Ref Runtime
Role: !GetAtt FunctionRole2.Arn
Code language: YAML (yaml)The action of this function is to notify a success signal to WaitCondition. For the implementation, we refer to the following page.
This function is related to the EventBridge rule described above.
This function is triggered when a deployed package is placed in an S3 bucket by CodeBuild.
So we can wait for CloudFomration to create resources until the deployment package is placed in the S3 bucket.
Lambda Function 3
Resources:
Function3:
Type: AWS::Lambda::Function
DependsOn:
- WaitCondition
Properties:
Architectures:
- !Ref Architecture
Code:
S3Bucket: !Ref BucketName
S3Key: !Ref SourceBundleName
FunctionName: !Sub "${Prefix}-function-03"
Handler: !Ref Handler
PackageType: Zip
Runtime: !Ref Runtime
Role: !GetAtt FunctionRole2.Arn
Code language: YAML (yaml)This function uses a deployment package placed in an S3 bucket.
The key point is the DependsOn property.
WaitCondition is specified earlier.
This means that the creation of this function will wait until the WaitCondition is notified of a success signal.
Using WaitCondition prevents resource creation from starting before the deployment package is placed in the S3 bucket.
Architecting
Use CloudFormation to build this environment and check its actual behavior.
Create CloudFormation stacks and check the resources in the stacks
Create CloudFormation stacks.
For information on how to create stacks and check each stack, please see the following page.
After reviewing the resources in each stack, information on the main resources created in this case is as follows
- S3 bucket: fa-142
- CodeBuild project: fa-142
- Lambda function 1: fa-142-function-01
- Lambda function 2: fa-142-function-02
- Lambda function 3: fa-142-function-03
- EventBridgeRule: fa-142-EventsRule
Check the created resource from the AWS Management Console.
Check the CodeBuild project.
The resource is created as defined in CloudFormation.
Of particular importance is the buildspec.
The deploy package for the function is placed in the S3 bucket after waiting 300 seconds in the execution process.
This reproduces the situation where the build takes a long time.
Check the S3 bucket.
Event notification to EventBridge is enabled.
Check the EventBridge rules.
The event pattern shows that the content of the event notification is when the object is placed in the S3 bucket mentioned above.
Looking at the event target, Lambda function 2, described below, is specified.
Check Lambda function 1.
This function is used in association with CloudFormation custom resources.
This function works when creating a CloudFormation stack.
The function’s action is to start the CodeBuild project described above.
Check Lambda function 2.
The function’s action is to notify WaitCondition of a success signal.
Action Check
Now that we are ready, we will check the actual operation.
Check the CodeBuild project again.
The project has been started.
This means that the action of the CloudFormation custom resource has executed the Lambda function 1 associated with it and started this project.
The phase details indicate that the build is in progress.
Because we are supposed to execute the sleep command in the build phase, this means we are waiting.
Check the status of CloudFormation stack creation.
There are two points.
The first point is that the WaitCondition is being created.
This means that the stack is waiting to be created until a success signal is received.
The second point is that the creation of Lambda function 3 has not started.
This function will not be created until WaitCondition receives a success signal.
This allows the function resource to wait to be created until the deployment package is placed in the S3 bucket.
After a short wait, the CodeBuild project will exit successfully.
It is indeed taking 300 seconds to build.
The successful completion of the CodeBuild project should have placed the deployment package in the S3 bucket.
Check the S3 bucket.
Indeed, the deployment package is in place.
Upon object placement, EventBridge triggers Lambda function 2.
Once again, check the status of the CloudFormation stack creation.
The status of WaitCondition is marked as completed.
In response, creation of Lambda function 3 is initiated.
After waiting for a while, check Lambda function 3.
Indeed, Lambda function 3 is created.
Thus, by using WaitCondition, the timing of resource creation can be controlled.
Summary
We have shown how to use CloudFormation’s WaitCondition to wait for the creation of a Lambda function resource until the deployment package is created.