AWS_EN

Use CodePipeline to trigger CodeCommit pushes to push images to ECR

スポンサーリンク
Use CodePipeline to trigger CodeCommit pushes to push images to ECR. AWS_EN
スポンサーリンク
スポンサーリンク

Use CodePipeline to trigger CodeCommit pushes to push images to ECR

Using CodePipeline, you can build a CI/CD configuration.

In this example, we will configure a pipeline to build a Docker image with CodeBuild and push it to the ECR when code is pushed to CodeCommit.

Environment

Diagram of use CodePipeline to trigger CodeCommit pushes to push images to ECR.

We will configure CodePipeline to link the two resources.
We will define two stages: source and build.

The first is CodeCommit.
This is used as a Git repository.

The second is CodeBuild.
Build a Docker image from code pushed to CodeCommit.
Push the built image to ECR.

Create an S3 bucket.
Use it to store artifacts generated by CodePipeline.

Store DockerHub account information in the SSM parameter store.
Use these to sign in to DockerHub and pull base images when generating images with DockerBuild.

The trigger for CodePipeline to be started is conditional on a push to CodeCommit.
Specifically, we will have a rule in EventBridge that satisfies the above.

CloudFormation template files

The above configuration is built using CloudFormation.
The following URL contains CloudFormation templates, etc.

awstut-fa/075 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Explanation of key points of the template files

S3 Bucket

Resources: Bucket: Type: AWS::S3::Bucket Properties: BucketName: !Ref Prefix AccessControl: Private
Code language: YAML (yaml)

Bucket to store artifacts generated by CodePipeline.
No special configuration is required.

SSM Parameter Store

Resources: SSMParameterDockerHubPassword: Type: AWS::SSM::Parameter Properties: Name: !Sub "${Prefix}-DockerHubPassword" Type: String Value: !Ref DockerHubPassword SSMParameterDockerHubUsername: Type: AWS::SSM::Parameter Properties: Name: !Sub "${Prefix}-DockerHubUsername" Type: String Value: !Ref DockerHubUsername
Code language: YAML (yaml)

Account information for signing in to DockerHub.
Register username and password as parameters.

ECR

Resources: ECRRepository: Type: AWS::ECR::Repository Properties: RepositoryName: !Ref Prefix
Code language: YAML (yaml)

Repository to store home-made repository images.
No special configuration is required.

CodeCommit

Resources: CodeCommitRepository: Type: AWS::CodeCommit::Repository Properties: RepositoryName: !Ref Prefix
Code language: YAML (yaml)

Code repository.
No special configuration is required.

CodeBuild

Resources: CodeBuildProject: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Cache: Type: NO_CACHE Environment: ComputeType: !Ref ProjectEnvironmentComputeType EnvironmentVariables: - Name: DOCKERHUB_PASSWORD Type: PARAMETER_STORE Value: !Ref SSMParameterDockerHubPassword - Name: DOCKERHUB_USERNAME Type: PARAMETER_STORE Value: !Ref SSMParameterDockerHubUsername Image: !Ref ProjectEnvironmentImage ImagePullCredentialsType: CODEBUILD Type: !Ref ProjectEnvironmentType PrivilegedMode: true LogsConfig: CloudWatchLogs: Status: DISABLED S3Logs: Status: DISABLED Name: !Ref Prefix ServiceRole: !GetAtt CodeBuildRole.Arn Source: Type: CODEPIPELINE BuildSpec: !Sub | version: 0.2 phases: pre_build: commands: - echo Logging in to Amazon ECR... - aws --version - aws ecr get-login-password --region ${AWS::Region} | docker login --username AWS --password-stdin ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com - REPOSITORY_URI=${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRRepositoryName} - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7) - IMAGE_TAG=${!COMMIT_HASH:=latest} - echo Logging in to Docker Hub... - echo $DOCKERHUB_PASSWORD | docker login -u $DOCKERHUB_USERNAME --password-stdin build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t $REPOSITORY_URI:latest . - docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG post_build: commands: - echo Build completed on `date` - echo Pushing the Docker images... - docker push $REPOSITORY_URI:latest - docker push $REPOSITORY_URI:$IMAGE_TAG Visibility: PRIVATE
Code language: YAML (yaml)

We will cover the key parameters.

The Artifacts property sets the settings related to the artifacts generated by CodeBuild.
The internal Type property sets the type of artifact.
In this case, we will specify “CODEPIPELINE” because CodeBuild will be executed within CodePipeline, which will be described later.

Define the build environment in the Environment property.
Set the resources to be allocated to the build environment in the ComputeType property. In this case, specify “BUILD_GENERAL1_SMALL,” which means the minimum environment, and allocate 3 GB of memory and 2 vCPU resources.
The EnvironmentVariables property allows you to define environment variables for the build environment. In this case, the values of the two SSM parameter stores mentioned above will be set as environment variables. Specify the parameter name of SSM parameter store.
In the Image property, set the Docker image to be used as the build environment. In this case, specify “aws/codebuild/amazonlinux2-aarch64-standard:2.0” to use the ARM version of Amazon Linux 2 image.
In the ImagePullCredentialsType property, set the credential type to be used to pull the image for the build environment. To use the aforementioned image for CodeBuild, specify “CODEBUILD”.
The PrivilegedMode property should be set to true to allow the Docker daemon to run inside the Docker container. This is required to build the Docker image.
Set the type of build environment in the Type property. In this case, specify “ARM_CONTAINER” to use the ARM version container.

Set the source code to be used in the build and the build specification file (buildspec.yml) in the Source property.
Set the type of source code repository in the Type property. In this case, we will use CodeBuild in CodePipeline, so we will specify “CODEPIPELINE”.
The BuildSpec property is a parameter related to buildspec.yml. In this case, the content of buildspec.yml is directly described in this property. The contents are described with reference to the official AWS page.

Tutorial: Amazon ECS Standard Deployment with CodePipeline - AWS CodePipeline
This tutorial helps you to create a complete, end-to-end continuous deployment (CD) pipeline with Amazon ECS with CodePipeline.

There is one change. When building an image, the base image is pulled from DockerHub, but we added a process to sign in to DockerHub. If you try to pull an image from DockerHub without signing in, the pull may fail. The following page is a reference for the description.

“Too Many Requests.” でビルドが失敗する…。AWS CodeBuild で IP ガチャを回避するために Docker Hub ログインしよう!という話 | DevelopersIO
Docker Hubのダウンロード回数制限、もうはじまってたんですね…。

Check the IAM role for CodeBuild.

Resources: CodeBuildRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - codebuild.amazonaws.com Action: - sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser Policies: - PolicyName: PipelineExecutionPolicy PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - ssm:GetParameters Resource: - !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${SSMParameterDockerHubPassword}" - !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${SSMParameterDockerHubUsername}" - Effect: Allow Action: - s3:PutObject - s3:GetObject - s3:GetObjectVersion - s3:GetBucketAcl - s3:GetBucketLocation Resource: - !Sub "arn:aws:s3:::${BucketName}" - !Sub "arn:aws:s3:::${BucketName}/*"
Code language: YAML (yaml)

Attach the AWS admin policy AmazonEC2ContainerRegistryPowerUser to push the image to ECR.
When setting environment variables, allow them to retrieve parameters from the SSM parameter store.
Allow to retrieve CodePipeline artifacts stored in S3 buckets when building images.

CodePipeline

Resources: Pipeline: Type: AWS::CodePipeline::Pipeline Properties: ArtifactStore: Location: !Ref BucketName Type: S3 Name: !Ref Prefix RoleArn: !GetAtt CodePipelineRole.Arn Stages: - Actions: - ActionTypeId: Category: Source Owner: AWS Provider: CodeCommit Version: 1 Configuration: BranchName: !Ref BranchName OutputArtifactFormat: CODE_ZIP PollForSourceChanges: false RepositoryName: !GetAtt CodeCommitRepository.Name Name: SourceAction OutputArtifacts: - Name: !Ref PipelineSourceArtifact Region: !Ref AWS::Region RunOrder: 1 Name: Source - Actions: - ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: 1 Configuration: ProjectName: !Ref CodeBuildProject InputArtifacts: - Name: !Ref PipelineSourceArtifact Name: Build OutputArtifacts: - Name: !Ref PipelineBuildArtifact Region: !Ref AWS::Region RunOrder: 1 Name: Build
Code language: YAML (yaml)

We will cover the key parameters.

In the ArtifactStore property, specify the S3 bucket that will store the artifacts generated while the pipeline is running.

In the Stages property, define the stages that make up the pipeline.
In this case, we will define two stages.

The first stage is the source stage.
Refer to the following official AWS page to set it up.

CodeCommit - AWS CodePipeline
Starts the pipeline when a new commit is made on the configured CodeCommit repository and branch.

In the ActionTypeId property, set CodeCommit to be the source of the pipeline.
In the Configuration property, set the parameters to treat CodeCommit as the source, and in the RepositoryName and BranchName properties, specify the CodeCommit repository name and branch name to be referenced. The OutputArtifactFormat property specifies the default “CODE_ZIP” The PollForSourceChanges property is the parameter that triggers the start of the pipeline. If it is set to true, CodePipeline will poll CodeCommit to detect activities that should start the pipeline. In this case, this parameter is set to false because the activity will be detected using EventBridge, which will be described later.
The OutputArtifacts property is a parameter related to the artifact names generated in the source stage.

The second stage is the build stage.
Refer to the following official AWS page to set it up.

AWS CodeBuild - AWS CodePipeline
Allows you to run builds and tests as part of your pipeline. When you run a CodeBuild build or test action, commands specified in the build spec are run inside ...

The ActionTypeId property sets CodeBuild to perform the build in the pipeline.
Configuration property is a parameter to build using CodeBuild; specify the CodeBuild to be associated with the pipeline in the ProjectName property.
In the InputArtifacts property, specify the names of the artifacts to be used in the build. The point is that the value must be the same as the value of the OutputArtifacts property specified in the source stage. This is because the artifacts generated in the source stage will be used for the build.
The OutputArtifacts property is a parameter related to the artifact names generated in the build stage.

Check the IAM role for CodePipeline.

Resources: CodePipelineRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - codepipeline.amazonaws.com Action: - sts:AssumeRole Policies: - PolicyName: PipelinePolicy PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - codecommit:CancelUploadArchive - codecommit:GetBranch - codecommit:GetCommit - codecommit:GetRepository - codecommit:GetUploadArchiveStatus - codecommit:UploadArchive Resource: - !GetAtt CodeCommitRepository.Arn - Effect: Allow Action: - codebuild:BatchGetBuilds - codebuild:StartBuild - codebuild:BatchGetBuildBatches - codebuild:StartBuildBatch Resource: - !GetAtt CodeBuildProject.Arn - Effect: Allow Action: - s3:PutObject - s3:GetObject - s3:GetObjectVersion - s3:GetBucketAcl - s3:GetBucketLocation Resource: - !Sub "arn:aws:s3:::${BucketName}" - !Sub "arn:aws:s3:::${BucketName}/*"
Code language: YAML (yaml)

To pull source code from CodeCommit in the source stage, grant permissions to allow this.
To build using CodeBuild in the build stage, grant permissions to execute the build.
Grant permission to retrieve and save artifacts on the S3 bucket during pipeline execution.\

EventBridge

Resources: EventsRule: Type: AWS::Events::Rule Properties: EventPattern: source: - aws.codecommit detail-type: - CodeCommit Repository State Change resources: - !GetAtt CodeCommitRepository.Arn detail: event: - referenceCreated - referenceUpdated referenceType: - branch referenceName: - !Ref BranchName Name: !Ref Prefix Targets: - Arn: !Sub "arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}" Id: !Sub "${Prefix}-CodePipeline-CodeCommit" RoleArn: !GetAtt EventsRuleRole.Arn
Code language: YAML (yaml)

Use EventBridge as the trigger for CodePipeline to start.

Refer to the following official AWS page to set it up.

Create a CloudWatch Events rule for a CodeCommit source (AWS CloudFormation template) - AWS CodePipeline
In the template, under Resources , use the AWS::IAM::Role AWS CloudFormation resource to configure the IAM role that allows your event to start your pipeline. T...

Specify the event to be detected with the EventPattern property. In this case, we will set it to detect when a branch is created or updated in CodeCommit.
Specify the resource to be notified with the Targets property. In this case, we will specify CodePipeline to start the pipeline.

Confirm the IAM role for EventBridge.

Resources: EventsRuleRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - events.amazonaws.com Action: - sts:AssumeRole Policies: - PolicyName: PipelineExecutionPolicy PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - codepipeline:StartPipelineExecution Resource: - !Sub "arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}"
Code language: YAML (yaml)

Grant settings to allow CodePipeline pipeline execution.

App Container

Dockerfile

FROM amazonlinux RUN yum update -y && yum install python3 python3-pip -y RUN pip3 install bottle COPY main.py ./ CMD ["python3", "main.py"] EXPOSE 8080
Code language: Dockerfile (dockerfile)

The image for the app container will be based on Amazon Linux 2.

We will use Bottle, a Python web framework.
So after installing Python and pip, install this.

Copy the Python script (main.py) describing the app logic and set this to run.

As mentioned earlier, the app listens for HTTP requests on tcp/8080, so expose this port.

main.py

from bottle import route, run @route('/') def hello(): return 'Hello CodePipeline!' if __name__ == '__main__': run(host='0.0.0.0', port=8080)
Code language: Python (python)

Use Bottle to build a simple web server.
The simple configuration is to listen for HTTP requests at 8080/tcp and return “Hello CodePipeline!”.

Architecting

We will use CloudFormation to build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • ECR: fa-075
  • CodeCommit: fa-075
  • CodeBuild project: fa-075
  • CodePipeline: fa-075
  • S3 bucket: fa-075
  • SSM parameter parameter name 1: fa-075-DcokerHubUsername
  • SSM parameter parameter name2: fa-075-DcokerHubPassword

Confirm the created resource from the AWS Management Console.
Check the ECR.

Detail of ECR 1.

Empty.
The image will be pushed here when the pipeline is executed.

Check CodeCommit.

Detail of CodeCommit 1.

This is also empty.
By pushing code here, the pipeline will be executed.

Check CodeBuild.

Detail of CodeBuild Settings 1.
Detail of CodeBuild Settings 2.

CodeBuild has been created as specified in the CloudFormation template.

Check CodePipeline.

Result of CodePipeline 1.

The pipeline has failed to execute.
This is because the pipeline was triggered by the creation of CodeCommit and its internal branch when the CloudFormation stack was created.
Since we are not pushing any code to CodeCommit at this time, an error occurred during the pipeline execution process.

Checking Action

Now that we are ready, we push the code to CodeCommit.

First, pull CodeCommit.

$ git clone https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-075 Cloning into 'fa-075'... warning: You appear to have cloned an empty repository.
Code language: Bash (bash)

An empty repository has been pulled.

Add a Dockerfile and main.py to the repository.

$ ls -al fa-075 total 8 drwxrwxr-x 3 ec2-user ec2-user 51 Aug 11 23:53 . drwxrwxr-x 3 ec2-user ec2-user 20 Aug 11 23:53 .. -rw-rw-r-- 1 ec2-user ec2-user 186 Aug 11 07:02 Dockerfile drwxrwxr-x 7 ec2-user ec2-user 119 Aug 11 23:52 .git -rw-rw-r-- 1 ec2-user ec2-user 630 Aug 11 07:42 main.py
Code language: Bash (bash)

Push the two files to CodeCommit.

$ git add . $ git commit -m "first commit" [master (root-commit) 2596718] first commit ... 2 files changed, 37 insertions(+) create mode 100644 Dockerfile create mode 100644 main.py $ git push ... * [new branch] master -> master
Code language: JavaScript (javascript)

The push was successful.

Check CodeCommit again.

Detail of CodeCommit 2.

Two files have indeed been pushed.

CodePipeline has started running.
Wait for a while.

Result of CodePipeline 2.

The pipeline has successfully completed execution.

Check the execution history of CodeBuild.

Result of CodeBuild execution.

The build has indeed been successfully executed.

Check the ECR.

Detail of ECR 2.

The image has been pushed.
This means that the image built by CodeBuild has been pushed to the ECR.

In this way, by configuring a pipeline with CodePipeline, you can use the code pushed to CodeCommit as the source, build the Docker image with CodeBuild, and push the image to the ECR.

Summary

We have seen how CodePipeline can be used to configure a pipeline to build a Docker image in CodeBuild and push it to ECR when code is pushed to CodeCommit.

タイトルとURLをコピーしました