AWS_EN

Use CFNs WaitCondition to wait for the Lambda deploy package to build

Use CFNs WaitCondition to wait for the Lambda deploy package to build

Consider creating a Lambda function using CloudFormation.

As described in the following pages, there are three patterns for creating functions.

This page will focus specifically on how to place a ZIP file deployment package in an S3 bucket.

In CloudFormation, when creating a Lambda function in this way, the deployment package must be placed in the S3 bucket before the function resource is generated.
But depending on what you include in the package, it may take some time to build and you may not be able to create the function resource in time.

So this time we will use CloudFormation’s WaitCondition.
Specifically, we will wait to generate the Lambda function resource until the deployment package is created.

Environment

Diagram of using CloudFormation's WaitCondition to wait for the Lambda deploy package to build.

The following flow creates Lambda function 3.

  1. Create a deployment package for the function in CodeBuild.
  2. Place the package in an S3 bucket.
  3. Specify EventBridge as the event notification destination for the S3 bucket.
  4. Create EventBridge rules for S3 object creation and trigger Lambda function 2.
  5. Function 2 signals the WaitCondition of CloudFormation.
  6. Upon receiving a WaitCondition signal, Lambda function 3 is created using the deploy package in the S3 bucket.

The kicker to start a CodeBuild project is to use a CloudFormation custom resource.
Specifically, we trigger the CodeBuild project with a Lambda function tied to the custom resource.

The runtime environment for Lambda functions is Python 3.8.

CloudFormation template files

The above configuration is built with CloudFormation.
The CloudFormation templates are placed at the following URL

awstut-fa/142 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Explanation of key points of template files

S3 bucket

Resources:
  Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: Private
      BucketName: !Ref Prefix
      NotificationConfiguration:
        EventBridgeConfiguration: 
          EventBridgeEnabled: true
Code language: YAML (yaml)

S3 bucket in which to place the deployment package.

The key point is the event notification function.
In this configuration, we need to execute Lambda function 2 triggered by the package being placed in the S3 bucket.
Enable the EventBridge notification feature in the NotificationConfiguration property.

In this case, we have selected EventBridge as the event notification destination, but you can also invoke the Lambda function directly.
For details, please refer to the following page.

CodeBuild

Resources:
  CodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties: 
      Artifacts:
        Type: NO_ARTIFACTS
      Cache: 
        Type: NO_CACHE
      Environment: 
        ComputeType: !Ref ProjectEnvironmentComputeType
        EnvironmentVariables:
          - Name: BUCKET_NAME
            Type: PLAINTEXT
            Value: !Ref BucketName
          - Name: SOURCE_BUNDLE_NAME
            Type: PLAINTEXT
            Value: !Ref SourceBundleName
          - Name: SOURCE_FILE_NAME
            Type: PLAINTEXT
            Value: !Ref SourceFileName
        Image: !Ref ProjectEnvironmentImage
        ImagePullCredentialsType: CODEBUILD
        Type: !Ref ProjectEnvironmentType
        PrivilegedMode: true
      LogsConfig: 
        CloudWatchLogs:
          Status: DISABLED
        S3Logs:
          Status: DISABLED
      Name: !Ref Prefix
      ServiceRole: !GetAtt CodeBuildRole.Arn
      Source: 
        Type: NO_SOURCE
        BuildSpec: !Sub |
          version: 0.2
          
          phases:
            pre_build:
              commands:
                - |
                  cat << EOF > $SOURCE_FILE_NAME
                  def lambda_handler(event, context):
                    return 'hogehoge'
                  EOF
            build:
              commands:
                - zip $SOURCE_BUNDLE_NAME -r * .[^.]*
                - sleep 300
            post_build:
              commands:
                - aws s3 cp $SOURCE_BUNDLE_NAME s3://$BUCKET_NAME/
      Visibility: PRIVATE
Code language: YAML (yaml)

Use CodeBuild to create a deployment package for the Lambda function.

Describe the contents of buildspec.yaml in the BuildSpec property.
Create the source code for the function in the pre_build phase.
Create the deployment package in the build phase.
Specifically, zip the files created in the previous phase.
In addition, this build should be time-consuming, so we will wait 5 minutes with the sleep command.
Upload the deployment package to the S3 bucket during the post_build phase.

The following are the IAM roles for CodeBuild.

Resources:
  CodeBuildRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - codebuild.amazonaws.com
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: PutS3ObjectPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - s3:PutObject
                Resource:
                  - !Sub "arn:aws:s3:::${BucketName}/*"
Code language: YAML (yaml)

Grant permissions to place objects in S3 buckets.

Lambda function 1

Resources:
  CustomResource:
    Type: Custom::CustomResource
    Properties:
      ServiceToken: !GetAtt Function1.Arn
      
  Function1:
    Type: AWS::Lambda::Function
    Properties:
      Architectures:
        - !Ref Architecture
      Environment:
        Variables:
          CODEBUILD_PROJECT: !Ref CodeBuildProject
      Code:
        ZipFile: |
          import boto3
          import cfnresponse
          import os
          
          codebuild_project = os.environ['CODEBUILD_PROJECT']
          
          CREATE = 'Create'
          response_data = {}
          
          client = boto3.client('codebuild')
          
          def lambda_handler(event, context):
            try:
              if event['RequestType'] == CREATE:
                response = client.start_build(
                  projectName=codebuild_project
                )
                print(response)
                
              cfnresponse.send(event, context, cfnresponse.SUCCESS, response_data)
              
            except Exception as e:
              print(e)
              cfnresponse.send(event, context, cfnresponse.FAILED, response_data)
      FunctionName: !Sub "${Prefix}-function-01"
      Handler: !Ref Handler
      Runtime: !Ref Runtime
      Role: !GetAtt FunctionRole1.Arn
Code language: YAML (yaml)

Lambda function 1 is used to automatically start a CodeBuild project.
Specifically, associate this function with a CloudFormation custom resource.

For more information on CloudFormation custom resources, please see the following pages

This time, when creating the custom resource, we will execute the start_build method of the boto3 client object for CodeBuild.
This will automatically start the CodeBuild project.

The following is the IAM role for this function.

Resources:
  FunctionRole1:
    Type: AWS::IAM::Role
    DeletionPolicy: Delete
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: sts:AssumeRole
            Principal:
              Service:
                - lambda.amazonaws.com
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: StartCodeBuildPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - codebuild:StartBuild
                Resource:
                  - !Sub "arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:project/${CodeBuildProject}"
Code language: YAML (yaml)

Authorization to start a CodeBuild project.

EventBridge

Resources:
  EventsRule:
    Type: AWS::Events::Rule
    Properties: 
      EventBusName: !Ref EventBusName
      EventPattern:
        source:
          - aws.s3
        detail-type:
          - Object Created
        detail:
          bucket:
            name:
              - !Ref BucketName
      Name: !Sub "${Prefix}-EventsRule"
      State: ENABLED
      Targets: 
        - Arn: !GetAtt Function2.Arn
          Id: !Ref Function2
Code language: YAML (yaml)

EventBridge Rule.
When an object is created in the aforementioned S3 bucket, Lambda function 2 described below is executed.

If you specify a Lambda function as the target, the behavior is that EventBridge calls the function.
Therefore, it is necessary to authorize EventBridge to call the function.

Resources:
  EventsRulePermission: 
    Type: AWS::Lambda::Permission
    Properties: 
      Action: lambda:InvokeFunction
      FunctionName: !Ref Function2
      Principal: events.amazonaws.com
      SourceArn: !GetAtt EventsRule.Arn
Code language: YAML (yaml)

WaitCondition

The official AWS website explains WaitCondition as follows

You can use the wait condition and wait condition handle to make AWS CloudFormation pause the creation of a stack and wait for a signal before it continues to create the stack.

Creating wait conditions in a template

Specifically, both resources are defined as follows

Resources:
  WaitConditionHandle:
    Type: AWS::CloudFormation::WaitConditionHandle

  WaitCondition:
    Type: AWS::CloudFormation::WaitCondition
    Properties:
      Handle: !Ref WaitConditionHandle
      Timeout: !Ref WaitConditionTimeout
Code language: YAML (yaml)

We have specified 600 for the Timeout property of WaitCondition.
This will wait for signal reception for up to 10 minutes.
If no signal is received after 10 minutes of waiting, a failure will occur and the CloudFormation stack will roll back.

Lambda Function 2

Resources:
  Function2:
    Type: AWS::Lambda::Function
    Properties:
      Architectures:
        - !Ref Architecture
      Environment:
        Variables:
          SIGNAL_URL: !Ref WaitConditionHandle
      Code:
        ZipFile: |
          import json
          import os
          import urllib3
          import uuid
    
          def lambda_handler(event, context):
            body = json.dumps({
                "Status": "SUCCESS",
                "Reason": "Lambda Deploy Package Setup Successed",
                "UniqueId": str(uuid.uuid4()),
                "Data": "Lambda Deploy Package Setup Successed"
            })
            http = urllib3.PoolManager()
            http.request('PUT', os.environ['SIGNAL_URL'], body=body)
      FunctionName: !Sub "${Prefix}-function-02"
      Handler: !Ref Handler
      Runtime: !Ref Runtime
      Role: !GetAtt FunctionRole2.Arn
Code language: YAML (yaml)

The action of this function is to notify a success signal to WaitCondition. For the implementation, we refer to the following page.

Waiting for AWS EventBridge events with AWS CloudFormation wait conditions
AWS CloudFormation has a feature called wait conditions, which is basically a generic way to wait for something to finish in…

This function is related to the EventBridge rule described above.
This function is triggered when a deployed package is placed in an S3 bucket by CodeBuild.
So we can wait for CloudFomration to create resources until the deployment package is placed in the S3 bucket.

Lambda Function 3

Resources:
  Function3:
    Type: AWS::Lambda::Function
    DependsOn:
      - WaitCondition
    Properties:
      Architectures:
        - !Ref Architecture
      Code:
        S3Bucket: !Ref BucketName
        S3Key: !Ref SourceBundleName
      FunctionName: !Sub "${Prefix}-function-03"
      Handler: !Ref Handler
      PackageType: Zip
      Runtime: !Ref Runtime
      Role: !GetAtt FunctionRole2.Arn
Code language: YAML (yaml)

This function uses a deployment package placed in an S3 bucket.

The key point is the DependsOn property.
WaitCondition is specified earlier.
This means that the creation of this function will wait until the WaitCondition is notified of a success signal.
Using WaitCondition prevents resource creation from starting before the deployment package is placed in the S3 bucket.

Architecting

Use CloudFormation to build this environment and check its actual behavior.

Create CloudFormation stacks and check the resources in the stacks

Create CloudFormation stacks.
For information on how to create stacks and check each stack, please see the following page.

After reviewing the resources in each stack, information on the main resources created in this case is as follows

  • S3 bucket: fa-142
  • CodeBuild project: fa-142
  • Lambda function 1: fa-142-function-01
  • Lambda function 2: fa-142-function-02
  • Lambda function 3: fa-142-function-03
  • EventBridgeRule: fa-142-EventsRule

Check the created resource from the AWS Management Console.

Check the CodeBuild project.

Detail of CodeBuild 1.

The resource is created as defined in CloudFormation.

Of particular importance is the buildspec.
The deploy package for the function is placed in the S3 bucket after waiting 300 seconds in the execution process.
This reproduces the situation where the build takes a long time.

Check the S3 bucket.

Detail of S3 1.

Event notification to EventBridge is enabled.

Check the EventBridge rules.

Detail of EventBridge 1.
Detail of EventBridge 2.

The event pattern shows that the content of the event notification is when the object is placed in the S3 bucket mentioned above.

Looking at the event target, Lambda function 2, described below, is specified.

Check Lambda function 1.

Detail of Lambda 1.

This function is used in association with CloudFormation custom resources.
This function works when creating a CloudFormation stack.
The function’s action is to start the CodeBuild project described above.

Check Lambda function 2.

Detail of Lambda 2.

The function’s action is to notify WaitCondition of a success signal.

Action Check

Now that we are ready, we will check the actual operation.

Check the CodeBuild project again.

Detail of CodeBuild 2.

The project has been started.
This means that the action of the CloudFormation custom resource has executed the Lambda function 1 associated with it and started this project.

The phase details indicate that the build is in progress.
Because we are supposed to execute the sleep command in the build phase, this means we are waiting.

Check the status of CloudFormation stack creation.

Detail of CloudFormation 1.

There are two points.

The first point is that the WaitCondition is being created.
This means that the stack is waiting to be created until a success signal is received.

The second point is that the creation of Lambda function 3 has not started.
This function will not be created until WaitCondition receives a success signal.
This allows the function resource to wait to be created until the deployment package is placed in the S3 bucket.

After a short wait, the CodeBuild project will exit successfully.

Detail of CodeBuild 3.

It is indeed taking 300 seconds to build.
The successful completion of the CodeBuild project should have placed the deployment package in the S3 bucket.

Check the S3 bucket.

Detail of S3 2.

Indeed, the deployment package is in place.

Upon object placement, EventBridge triggers Lambda function 2.

Once again, check the status of the CloudFormation stack creation.

Detail of CloudFormation 2.

The status of WaitCondition is marked as completed.

In response, creation of Lambda function 3 is initiated.

After waiting for a while, check Lambda function 3.

Detail of Lambda 3.

Indeed, Lambda function 3 is created.

Thus, by using WaitCondition, the timing of resource creation can be controlled.

Summary

We have shown how to use CloudFormation’s WaitCondition to wait for the creation of a Lambda function resource until the deployment package is created.

タイトルとURLをコピーしました