Use CodePipeline to build and deploy images to Fargate
On the following page, we showed you how to configure a pipeline using CodePipeline to push images to ECR by linking CodeCommit and CodeBuild.
This time, we will create a deployment stage in CodePipeline and aim to deploy the built image to Fargate.
Environment
We will configure CodePipeline to link two resources.
The first is CodeCommit.
CodeCommit is responsible for the source stage of CodePipeline.
It is used as a Git repository.
The second is CodeBuild.
CodeBuild is in charge of the build stage of CodePipeline.
It builds a Docker image from code pushed to CodeCommit.
Push the built image to ECR.
Creates a deploy stage in CodePipeline.
Configure it to deploy to Fargate, described below.
Store your DockerHub account information in the SSM parameter store.
These will be used to pull the base image when generating images with DockerBuild, after signing in to DockerHub.
The trigger for CodePipeline to be started is conditional on a push to CodeCommit.
Specifically, we will have a rule in EventBridge that satisfies the above.
Create a Fargate type ECS on a private subnet.
Create VPC endpoints for ECR and S3 to retrieve images pushed to ECR.
Create an EC2 instance.
Use it as a client to access the container created on Fargate.
CloudFormation template files
Build the above configuration with CloudFormation.
The CloudFormation templates are located at the following URL
https://github.com/awstut-an-r/awstut-fa/tree/main/076
Explanation of key points of the template files
The basic configuration is the same as the page introduced at the beginning of this document.
This page focuses on how to deploy a Docker image built with CodeBuild to Fargate in CodePipeline.
For information on how to build Fargate on a private subnet, please refer to the following page
CodeBuild
Resources:
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: CODEPIPELINE
Cache:
Type: NO_CACHE
Environment:
ComputeType: !Ref ProjectEnvironmentComputeType
EnvironmentVariables:
- Name: CONTAINER_NAME
Type: PLAINTEXT
Value: !Ref ContainerName
- Name: DOCKERHUB_PASSWORD
Type: PARAMETER_STORE
Value: !Ref SSMParameterDockerHubPassword
- Name: DOCKERHUB_USERNAME
Type: PARAMETER_STORE
Value: !Ref SSMParameterDockerHubUsername
- Name: IMAGE_DEFINITION
Type: PLAINTEXT
Value: !Ref ImageDefinitionFileName
Image: !Ref ProjectEnvironmentImage
ImagePullCredentialsType: CODEBUILD
Type: !Ref ProjectEnvironmentType
PrivilegedMode: true
LogsConfig:
CloudWatchLogs:
Status: DISABLED
S3Logs:
Status: DISABLED
Name: !Ref Prefix
ServiceRole: !GetAtt CodeBuildRole.Arn
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.2
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- aws --version
- aws ecr get-login-password --region ${AWS::Region} | docker login --username AWS --password-stdin ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com
- REPOSITORY_URI=${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRRepositoryName}
- COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
- IMAGE_TAG=${!COMMIT_HASH:=latest}
- echo Logging in to Docker Hub...
- echo $DOCKERHUB_PASSWORD | docker login -u $DOCKERHUB_USERNAME --password-stdin
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t $REPOSITORY_URI:latest .
- docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push $REPOSITORY_URI:latest
- docker push $REPOSITORY_URI:$IMAGE_TAG
- echo Writing image definitions file...
- printf '[{"name":"%s","imageUri":"%s"}]' $CONTAINER_NAME $REPOSITORY_URI:$IMAGE_TAG > $IMAGE_DEFINITION
artifacts:
files: $IMAGE_DEFINITION
Visibility: PRIVATE
Code language: YAML (yaml)The BuildSpec property is the key.
In this case, the contents of buildspec.yml are directly described in the CloudFormation template.
The key to deploying an image built with CodeBuild to ECS (Fargate) is the post_build phase.
According to the official AWS page, a file named imagedefinition.json must be created.
Write a file called imagedefinitions.json in the build root that has your Amazon ECS service’s container name and the image and tag. The deployment stage of your CD pipeline uses this information to create a new revision of your service’s task definition, and then it updates the service to use the new task definition. The imagedefinitions.json file is required for the ECS job worker.
Tutorial: Amazon ECS Standard Deployment with CodePipeline
In this case, we store the string imaginedefinitions.json in the environment variable IMAGE_DEFINITION and refer to it in two places.
CodePipeline
Resources:
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref BucketName
Type: S3
Name: !Ref Prefix
RoleArn: !GetAtt CodePipelineRole.Arn
Stages:
- Actions:
- ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeCommit
Version: 1
Configuration:
BranchName: !Ref BranchName
OutputArtifactFormat: CODE_ZIP
PollForSourceChanges: false
RepositoryName: !GetAtt CodeCommitRepository.Name
Name: SourceAction
OutputArtifacts:
- Name: !Ref PipelineSourceArtifact
Region: !Ref AWS::Region
RunOrder: 1
Name: Source
- Actions:
- ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref CodeBuildProject
InputArtifacts:
- Name: !Ref PipelineSourceArtifact
Name: Build
OutputArtifacts:
- Name: !Ref PipelineBuildArtifact
Region: !Ref AWS::Region
RunOrder: 1
Name: Build
- Actions:
- ActionTypeId:
Category: Deploy
Owner: AWS
Provider: ECS
Version: 1
Configuration:
ClusterName: !Ref ECSClusterName
FileName: !Ref ImageDefinitionFileName
ServiceName: !Ref ECSServiceName
InputArtifacts:
- Name: !Ref PipelineBuildArtifact
Name: Deploy
Region: !Ref AWS::Region
RunOrder: 1
Name: Deploy
Code language: YAML (yaml)The Stages property is the key point.
The third element of this property is the stage to deploy to ECS(Fargate).
In the ActionTypeId property, set the stage to deploy to ECS(Fargate).
Specify the file name of the imagedefinitions.json file mentioned above in the FileName property. Specify the ECS cluster service to deploy to in the ClusterName and ServiceName properties.
The InputArtifacts property sets the artifacts to be used in this stage. The InputArtifacts property sets the artifacts to be used in this stage. The point is that it must be the same as the value of the OutputArtifacts property specified in the build stage. This is because the artifacts generated in the build stage will be used for deployment.
Check the IAM role for CodePipeline.
Resources:
CodePipelineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: PipelinePolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetRepository
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
Resource:
- !GetAtt CodeCommitRepository.Arn
- Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
- codebuild:BatchGetBuildBatches
- codebuild:StartBuildBatch
Resource:
- !GetAtt CodeBuildProject.Arn
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketAcl
- s3:GetBucketLocation
Resource:
- !Sub "arn:aws:s3:::${BucketName}"
- !Sub "arn:aws:s3:::${BucketName}/*"
- Effect: Allow
Action:
- ecs:*
Resource: "*"
- Effect: Allow
Action:
- iam:PassRole
Resource: "*"
Condition:
StringLike:
iam:PassedToService:
- ecs-tasks.amazonaws.com
Code language: YAML (yaml)The key points are permissions regarding ECS and iam:PassRole.
These privileges are required to deploy images to ECS.
This IAM role is referenced by the RoleArn property of the pipeline, which is described as follows
The Amazon Resource Name (ARN) for CodePipeline to use to either perform actions with no actionRoleArn, or to use to assume roles for actions with an actionRoleArn.
AWS::CodePipeline::Pipeline
This means that in this deployment stage, this IAM role will be used to deploy the image to ECS.
Check the trust policy of this IAM role.
The principal is codepipeline.amazonaws.com, which means that CodePipeline can use this IAM role to perform all actions on ECS, for example.
However, the action to deploy the Docker image in this case will be performed by an ECS task.
So give this IAM role the iam:PassRole privilege and allow this IAM role to pass ECS tasks.
This behavior is implemented by setting the Resource to “*” and the Condition to the condition that the pass target is the “ecs-tasks.amazonaws.com” service.
(Reference) ECS Service
Resources:
Service:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref Cluster
DesiredCount: 0
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
SecurityGroups:
- !Ref ContainerSecurityGroup
Subnets:
- !Ref ContainerSubnet
ServiceName: !Sub "${Prefix}-service"
TaskDefinition: !Ref TaskDefinition
Code language: YAML (yaml)No special settings will be made.
There is one point.
In the DesiredCount property, set the number of tasks to be created on the ECS service to 0.
If this value is greater than 1, an attempt will be made to create a task when the CloudFormation stack is created.
However, during the initial build, the task creation will fail because no image has been pushed to the ECR.
Therefore, set this property to 0 to prevent task creation.
(Reference) Application Container
Dockerfile
FROM amazonlinux
RUN yum update -y && yum install python3 python3-pip -y
RUN pip3 install bottle
COPY main.py ./
CMD ["python3", "main.py"]
EXPOSE 8080
Code language: YAML (yaml)This is the same page as the one shown at the beginning of this document.
Install Python and the web framework Bottle based on Amazon Linux 2.
Copy the Python script (main.py) describing the application logic and configure it to run.
The app will listen for HTTP requests at 8080/tcp, so expose this port.
main.py
from bottle import route, run
@route('/')
def hello():
return 'Hello CodePipeline.'
if __name__ == '__main__':
run(host='0.0.0.0', port=8080)
Code language: YAML (yaml)We will use Bottle to build a simple web server.
The simple configuration is to listen for HTTP requests at 8080/tcp and return “Hello CodePipeline.
Architecting
We will use CloudFormation to build this environment and check the actual behavior.
Create CloudFormation stacks and check resources in stacks
Create a CloudFormation stack.
For information on how to create stacks and check each stack, please refer to the following page
After checking the resources in each stack, information on the main resources created this time is as follows
- ECS cluster: fa-076-cluster
- ECS service: fa-076-service
- ECR: fa-076
- CodeCommit: fa-076
- CodeBuild project: fa-076
- CodePipeline: fa-076
- SSM parameter parameter name 1: fa-076-DcokerHubUsername
- SSM parameter parameter name2: fa-076-DcokerHubPassword
- EC2 instance: i-0024a483b8e4a8778
Confirm the created resource from the AWS Management Console.
Check ECS(Fargate).
The ECS Cluster Service task has been successfully created.
The Desired Count is 0, indicating that no tasks have been created.
Check ECR.
It is empty.
The pipeline will be executed and the image will be pushed here.
Check CodeCommit.
This is also empty.
By pushing code here, the pipeline will be executed.
Check CodeBuild.
CodeBuild has been created as specified in the CloudFormation template.
Check CodePipeline.
The pipeline has failed to execute.
This is because the pipeline was triggered by the creation of CodeCommit and its internal branches when the CloudFormation stack was created.
Since we are not pushing code to CodeCommit at this time, this means that an error occurred during the pipeline execution process.
Checking Action
Pipeline execution 1st
Now that we are ready, we push the code to CodeCommit.
First, pull CodeCommit.
$ git clone https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-076
Cloning into 'fa-076'…
warning: You appear to have cloned an empty repository.
Code language: Bash (bash)An empty repository has been pulled.
Add a Dockerfile and main.py to the repository.
$ ls -al
total 8
drwxrwxr-x 3 ec2-user ec2-user 51 Aug 14 11:56 .
drwxrwxr-x 3 ec2-user ec2-user 20 Aug 14 11:56 ..
-rw-rw-r-- 1 ec2-user ec2-user 187 Aug 12 11:01 Dockerfile
drwxrwxr-x 7 ec2-user ec2-user 119 Aug 14 11:56 .git
-rw-rw-r-- 1 ec2-user ec2-user 681 Aug 13 12:30 main.pyCode language: Bash (bash)Push 2 files to CodeCommit.
$ git add .
$ git commit -m "first commit"
[master (root-commit) 738b02d2] first commit
...
2 files changed, 39 insertions(+)
create mode 100644 Dockerfile
create mode 100644 main.py
$ git push
...
* [new branch] master -> master
Code language: Bash (bash)I was able to push successfully.
Check CodeCommit again.
Two files have indeed been pushed.
CodePipeline has started running.
Wait for a while.
The pipeline has successfully completed execution.
Check the ECR.
The image has been pushed.
This means that the image built by CodeBuild has been pushed to ECR.
Change the configuration of the ECS service.
Update Desired Count from 0 to 1.
Check the ECS service.
One task has been created.
Check the details of the created task.
You can see that the automatically assigned private address is “10.0.2.246”.
Access the EC2 instance to make an HTTP request to the container.
Use SSM Session Manager to access the instance.
% aws ssm start-session --target i-0024a483b8e4a8778
Starting session with SessionId: root-00be6118d01682117
sh-4.2$
Code language: Bash (bash)For more information on SSM Session Manager, please refer to the following page
Use the curl command to access the container in the task.
sh-4.2$ curl http://10.0.2.246:8080
Hello CodePipeline.
Code language: Bash (bash)The container responded.
It is indeed the string we set up in the Bottle app.
This indicates that the image was built and the ECS task was generated from this image by executing the pipeline created by CodePipeline.
Pipeline execution 2nd
We will run the pipeline again.
Make a few changes to the code in main.py and push it to CodeComit.
$ cat main.py
from bottle import route, run
@route('/')
def hello():
return 'Hello CodePipeline. Update!'
if __name__ == '__main__':
run(host='0.0.0.0', port=8080)
Code language: Bash (bash)$ git add .
$ git commit -m "second commit"
[master e3dc93e] second commit
Committer: EC2 Default User <ec2-user@ip-172-31-26-46.ap-northeast-1.compute.internal>
...
1 file changed, 1 insertion(+), 1 deletion(-)
$ git push
...
To https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-076
738b02d..e3dc93e master -> master
Code language: Bash (bash)Check CodePipeline.
The pipeline is now running again.
Wait for a while and the pipeline will complete.
Check ECR again.
A new image has been built and pushed.
From now on, ECS tasks should be generated using this image.
Check the behavior of the ECS service during deployment.
During the ECS deployment, two tasks can be checked the running.
One is the task that was originally running. This task was generated from the old image.
The other is a newly created task. This task is generated from the new image.
Thus, when deployment begins, there are times when new tasks are generated and old and new tasks coexist.
After the deployment is complete, check the ECS service again.
Only the new task is in action.
The old task has been stopped.
When the deployment is complete, only the tasks generated from the new image will be in action.
Check the details of the newly generated task.
The private address assigned to the newly generated task was “10.0.2.224”.
Access the ECS task from the EC2 instance again.
sh-4.2$ curl http://10.0.2.224:8080/
Hello CodePipeline. Update!
Code language: Bash (bash)The container responded.
The string reflects the updated code.
This indicates that the latest image has been deployed to ECS (Fargate) by running the pipeline created by CodePipeline.
Summary
In this case, we were able to create a deployment stage in CodePipeline and deploy the built image to Fargate.