AWS_EN

Use CodePipeline to build and deploy images to Fargate

スポンサーリンク
Use CodePipeline to build and deploy images to Fargate. AWS_EN
スポンサーリンク
スポンサーリンク

Use CodePipeline to build and deploy images to Fargate

On the following page, we showed you how to configure a pipeline using CodePipeline to push images to ECR by linking CodeCommit and CodeBuild.

This time, we will create a deployment stage in CodePipeline and aim to deploy the built image to Fargate.

Environment

Diagram of use CodePipeline to build and deploy images to Fargate.

We will configure CodePipeline to link two resources.

The first is CodeCommit.
CodeCommit is responsible for the source stage of CodePipeline.
It is used as a Git repository.

The second is CodeBuild.
CodeBuild is in charge of the build stage of CodePipeline.
It builds a Docker image from code pushed to CodeCommit.
Push the built image to ECR.

Creates a deploy stage in CodePipeline.
Configure it to deploy to Fargate, described below.

Store your DockerHub account information in the SSM parameter store.
These will be used to pull the base image when generating images with DockerBuild, after signing in to DockerHub.

The trigger for CodePipeline to be started is conditional on a push to CodeCommit.
Specifically, we will have a rule in EventBridge that satisfies the above.

Create a Fargate type ECS on a private subnet.

Create VPC endpoints for ECR and S3 to retrieve images pushed to ECR.

Create an EC2 instance.
Use it as a client to access the container created on Fargate.

CloudFormation template files

Build the above configuration with CloudFormation.
The CloudFormation templates are located at the following URL

awstut-fa/076 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Explanation of key points of the template files

The basic configuration is the same as the page introduced at the beginning of this document.
This page focuses on how to deploy a Docker image built with CodeBuild to Fargate in CodePipeline.

For information on how to build Fargate on a private subnet, please refer to the following page

CodeBuild

Resources: CodeBuildProject: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Cache: Type: NO_CACHE Environment: ComputeType: !Ref ProjectEnvironmentComputeType EnvironmentVariables: - Name: CONTAINER_NAME Type: PLAINTEXT Value: !Ref ContainerName - Name: DOCKERHUB_PASSWORD Type: PARAMETER_STORE Value: !Ref SSMParameterDockerHubPassword - Name: DOCKERHUB_USERNAME Type: PARAMETER_STORE Value: !Ref SSMParameterDockerHubUsername - Name: IMAGE_DEFINITION Type: PLAINTEXT Value: !Ref ImageDefinitionFileName Image: !Ref ProjectEnvironmentImage ImagePullCredentialsType: CODEBUILD Type: !Ref ProjectEnvironmentType PrivilegedMode: true LogsConfig: CloudWatchLogs: Status: DISABLED S3Logs: Status: DISABLED Name: !Ref Prefix ServiceRole: !GetAtt CodeBuildRole.Arn Source: Type: CODEPIPELINE BuildSpec: !Sub | version: 0.2 phases: pre_build: commands: - echo Logging in to Amazon ECR... - aws --version - aws ecr get-login-password --region ${AWS::Region} | docker login --username AWS --password-stdin ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com - REPOSITORY_URI=${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRRepositoryName} - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7) - IMAGE_TAG=${!COMMIT_HASH:=latest} - echo Logging in to Docker Hub... - echo $DOCKERHUB_PASSWORD | docker login -u $DOCKERHUB_USERNAME --password-stdin build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t $REPOSITORY_URI:latest . - docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG post_build: commands: - echo Build completed on `date` - echo Pushing the Docker images... - docker push $REPOSITORY_URI:latest - docker push $REPOSITORY_URI:$IMAGE_TAG - echo Writing image definitions file... - printf '[{"name":"%s","imageUri":"%s"}]' $CONTAINER_NAME $REPOSITORY_URI:$IMAGE_TAG > $IMAGE_DEFINITION artifacts: files: $IMAGE_DEFINITION Visibility: PRIVATE
Code language: YAML (yaml)

The BuildSpec property is the key.
In this case, the contents of buildspec.yml are directly described in the CloudFormation template.

The key to deploying an image built with CodeBuild to ECS (Fargate) is the post_build phase.
According to the official AWS page, a file named imagedefinition.json must be created.

Write a file called imagedefinitions.json in the build root that has your Amazon ECS service’s container name and the image and tag. The deployment stage of your CD pipeline uses this information to create a new revision of your service’s task definition, and then it updates the service to use the new task definition. The imagedefinitions.json file is required for the ECS job worker.

Tutorial: Amazon ECS Standard Deployment with CodePipeline

In this case, we store the string imaginedefinitions.json in the environment variable IMAGE_DEFINITION and refer to it in two places.

CodePipeline

Resources: Pipeline: Type: AWS::CodePipeline::Pipeline Properties: ArtifactStore: Location: !Ref BucketName Type: S3 Name: !Ref Prefix RoleArn: !GetAtt CodePipelineRole.Arn Stages: - Actions: - ActionTypeId: Category: Source Owner: AWS Provider: CodeCommit Version: 1 Configuration: BranchName: !Ref BranchName OutputArtifactFormat: CODE_ZIP PollForSourceChanges: false RepositoryName: !GetAtt CodeCommitRepository.Name Name: SourceAction OutputArtifacts: - Name: !Ref PipelineSourceArtifact Region: !Ref AWS::Region RunOrder: 1 Name: Source - Actions: - ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: 1 Configuration: ProjectName: !Ref CodeBuildProject InputArtifacts: - Name: !Ref PipelineSourceArtifact Name: Build OutputArtifacts: - Name: !Ref PipelineBuildArtifact Region: !Ref AWS::Region RunOrder: 1 Name: Build - Actions: - ActionTypeId: Category: Deploy Owner: AWS Provider: ECS Version: 1 Configuration: ClusterName: !Ref ECSClusterName FileName: !Ref ImageDefinitionFileName ServiceName: !Ref ECSServiceName InputArtifacts: - Name: !Ref PipelineBuildArtifact Name: Deploy Region: !Ref AWS::Region RunOrder: 1 Name: Deploy
Code language: YAML (yaml)

The Stages property is the key point.
The third element of this property is the stage to deploy to ECS(Fargate).

In the ActionTypeId property, set the stage to deploy to ECS(Fargate).
Specify the file name of the imagedefinitions.json file mentioned above in the FileName property. Specify the ECS cluster service to deploy to in the ClusterName and ServiceName properties.
The InputArtifacts property sets the artifacts to be used in this stage. The InputArtifacts property sets the artifacts to be used in this stage. The point is that it must be the same as the value of the OutputArtifacts property specified in the build stage. This is because the artifacts generated in the build stage will be used for deployment.

Check the IAM role for CodePipeline.

Resources: CodePipelineRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - codepipeline.amazonaws.com Action: - sts:AssumeRole Policies: - PolicyName: PipelinePolicy PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - codecommit:CancelUploadArchive - codecommit:GetBranch - codecommit:GetCommit - codecommit:GetRepository - codecommit:GetUploadArchiveStatus - codecommit:UploadArchive Resource: - !GetAtt CodeCommitRepository.Arn - Effect: Allow Action: - codebuild:BatchGetBuilds - codebuild:StartBuild - codebuild:BatchGetBuildBatches - codebuild:StartBuildBatch Resource: - !GetAtt CodeBuildProject.Arn - Effect: Allow Action: - s3:PutObject - s3:GetObject - s3:GetObjectVersion - s3:GetBucketAcl - s3:GetBucketLocation Resource: - !Sub "arn:aws:s3:::${BucketName}" - !Sub "arn:aws:s3:::${BucketName}/*" - Effect: Allow Action: - ecs:* Resource: "*" - Effect: Allow Action: - iam:PassRole Resource: "*" Condition: StringLike: iam:PassedToService: - ecs-tasks.amazonaws.com
Code language: YAML (yaml)

The key points are permissions regarding ECS and iam:PassRole.
These privileges are required to deploy images to ECS.

This IAM role is referenced by the RoleArn property of the pipeline, which is described as follows

The Amazon Resource Name (ARN) for CodePipeline to use to either perform actions with no actionRoleArn, or to use to assume roles for actions with an actionRoleArn.

AWS::CodePipeline::Pipeline

This means that in this deployment stage, this IAM role will be used to deploy the image to ECS.

Check the trust policy of this IAM role.
The principal is codepipeline.amazonaws.com, which means that CodePipeline can use this IAM role to perform all actions on ECS, for example.

However, the action to deploy the Docker image in this case will be performed by an ECS task.
So give this IAM role the iam:PassRole privilege and allow this IAM role to pass ECS tasks.
This behavior is implemented by setting the Resource to “*” and the Condition to the condition that the pass target is the “ecs-tasks.amazonaws.com” service.

(Reference) ECS Service

Resources: Service: Type: AWS::ECS::Service Properties: Cluster: !Ref Cluster DesiredCount: 0 LaunchType: FARGATE NetworkConfiguration: AwsvpcConfiguration: SecurityGroups: - !Ref ContainerSecurityGroup Subnets: - !Ref ContainerSubnet ServiceName: !Sub "${Prefix}-service" TaskDefinition: !Ref TaskDefinition
Code language: YAML (yaml)

No special settings will be made.

There is one point.
In the DesiredCount property, set the number of tasks to be created on the ECS service to 0.
If this value is greater than 1, an attempt will be made to create a task when the CloudFormation stack is created.
However, during the initial build, the task creation will fail because no image has been pushed to the ECR.
Therefore, set this property to 0 to prevent task creation.

(Reference) Application Container

Dockerfile

FROM amazonlinux RUN yum update -y && yum install python3 python3-pip -y RUN pip3 install bottle COPY main.py ./ CMD ["python3", "main.py"] EXPOSE 8080
Code language: YAML (yaml)

This is the same page as the one shown at the beginning of this document.

Install Python and the web framework Bottle based on Amazon Linux 2.
Copy the Python script (main.py) describing the application logic and configure it to run.
The app will listen for HTTP requests at 8080/tcp, so expose this port.

main.py

from bottle import route, run @route('/') def hello(): return 'Hello CodePipeline.' if __name__ == '__main__': run(host='0.0.0.0', port=8080)
Code language: YAML (yaml)

We will use Bottle to build a simple web server.
The simple configuration is to listen for HTTP requests at 8080/tcp and return “Hello CodePipeline.

Architecting

We will use CloudFormation to build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stack.
For information on how to create stacks and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • ECS cluster: fa-076-cluster
  • ECS service: fa-076-service
  • ECR: fa-076
  • CodeCommit: fa-076
  • CodeBuild project: fa-076
  • CodePipeline: fa-076
  • SSM parameter parameter name 1: fa-076-DcokerHubUsername
  • SSM parameter parameter name2: fa-076-DcokerHubPassword
  • EC2 instance: i-0024a483b8e4a8778

Confirm the created resource from the AWS Management Console.
Check ECS(Fargate).

Detail of ECS 1.
Detail of ECS 2.

The ECS Cluster Service task has been successfully created.
The Desired Count is 0, indicating that no tasks have been created.

Check ECR.

Detail of ECR 1.

It is empty.
The pipeline will be executed and the image will be pushed here.

Check CodeCommit.

Detail of CodeCommit 1.

This is also empty.
By pushing code here, the pipeline will be executed.

Check CodeBuild.

Detail of CodeBuild 1.
Detail of CodeBuild 2.

CodeBuild has been created as specified in the CloudFormation template.

Check CodePipeline.

Detail of CodePipeline 1.

The pipeline has failed to execute.
This is because the pipeline was triggered by the creation of CodeCommit and its internal branches when the CloudFormation stack was created.
Since we are not pushing code to CodeCommit at this time, this means that an error occurred during the pipeline execution process.

Checking Action

Pipeline execution 1st

Now that we are ready, we push the code to CodeCommit.

First, pull CodeCommit.

$ git clone https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-076 Cloning into 'fa-076'warning: You appear to have cloned an empty repository.
Code language: Bash (bash)

An empty repository has been pulled.

Add a Dockerfile and main.py to the repository.

$ ls -al total 8 drwxrwxr-x 3 ec2-user ec2-user 51 Aug 14 11:56 . drwxrwxr-x 3 ec2-user ec2-user 20 Aug 14 11:56 .. -rw-rw-r-- 1 ec2-user ec2-user 187 Aug 12 11:01 Dockerfile drwxrwxr-x 7 ec2-user ec2-user 119 Aug 14 11:56 .git -rw-rw-r-- 1 ec2-user ec2-user 681 Aug 13 12:30 main.py
Code language: Bash (bash)

Push 2 files to CodeCommit.

$ git add . $ git commit -m "first commit" [master (root-commit) 738b02d2] first commit ... 2 files changed, 39 insertions(+) create mode 100644 Dockerfile create mode 100644 main.py $ git push ... * [new branch] master -> master
Code language: Bash (bash)

I was able to push successfully.

Check CodeCommit again.

Detail of CodeCommit 2.

Two files have indeed been pushed.

CodePipeline has started running.
Wait for a while.

Detail of CodePipeline 2.

The pipeline has successfully completed execution.

Check the ECR.

Detail of ECR 2.

The image has been pushed.
This means that the image built by CodeBuild has been pushed to ECR.

Change the configuration of the ECS service.

Detail of ECS 33.

Update Desired Count from 0 to 1.

Check the ECS service.

Detail of ECS 5.

One task has been created.

Check the details of the created task.

Detail of ECS 4.

You can see that the automatically assigned private address is “10.0.2.246”.

Access the EC2 instance to make an HTTP request to the container.
Use SSM Session Manager to access the instance.

% aws ssm start-session --target i-0024a483b8e4a8778 Starting session with SessionId: root-00be6118d01682117 sh-4.2$
Code language: Bash (bash)

For more information on SSM Session Manager, please refer to the following page

Use the curl command to access the container in the task.

sh-4.2$ curl http://10.0.2.246:8080 Hello CodePipeline.
Code language: Bash (bash)

The container responded.
It is indeed the string we set up in the Bottle app.
This indicates that the image was built and the ECS task was generated from this image by executing the pipeline created by CodePipeline.

Pipeline execution 2nd

We will run the pipeline again.

Make a few changes to the code in main.py and push it to CodeComit.

$ cat main.py from bottle import route, run @route('/') def hello(): return 'Hello CodePipeline. Update!' if __name__ == '__main__': run(host='0.0.0.0', port=8080)
Code language: Bash (bash)
$ git add . $ git commit -m "second commit" [master e3dc93e] second commit Committer: EC2 Default User <ec2-user@ip-172-31-26-46.ap-northeast-1.compute.internal> ... 1 file changed, 1 insertion(+), 1 deletion(-) $ git push ... To https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-076 738b02d..e3dc93e master -> master
Code language: Bash (bash)

Check CodePipeline.

Detail of CodePipeline 3.

The pipeline is now running again.
Wait for a while and the pipeline will complete.

Check ECR again.

Detail of ECR 3.

A new image has been built and pushed.
From now on, ECS tasks should be generated using this image.

Check the behavior of the ECS service during deployment.

Detail of ECS 7.

During the ECS deployment, two tasks can be checked the running.
One is the task that was originally running. This task was generated from the old image.
The other is a newly created task. This task is generated from the new image.
Thus, when deployment begins, there are times when new tasks are generated and old and new tasks coexist.

After the deployment is complete, check the ECS service again.

Detail of ECS 8.

Only the new task is in action.
The old task has been stopped.
When the deployment is complete, only the tasks generated from the new image will be in action.

Check the details of the newly generated task.

Detail of ECS 9.

The private address assigned to the newly generated task was “10.0.2.224”.

Access the ECS task from the EC2 instance again.

sh-4.2$ curl http://10.0.2.224:8080/ Hello CodePipeline. Update!
Code language: Bash (bash)

The container responded.
The string reflects the updated code.
This indicates that the latest image has been deployed to ECS (Fargate) by running the pipeline created by CodePipeline.

Summary

In this case, we were able to create a deployment stage in CodePipeline and deploy the built image to Fargate.

タイトルとURLをコピーしました