Setting up Test Units in CodePipeline
Test units can be added to CodePipeline.
Test units can be created with CodeBuild.
In this case, we will use CodePipeline to build a pipeline to create a Docker image and push it to an ECR repository.
Our goal is to add test units to the process.
Environment
We will configure CodePipeline to link three resources.
The first is CodeCommit.
CodeCommit is responsible for the source stage of CodePipeline.
It is used as a Git repository.
The second is CodeBuild.
CodeBuild is in charge of the test stage of CodePipeline.
This time, the test target will be a Python script, and the tool used for testing will be pytest.
The third is also a CodeBuild.
This CodeBuild is in charge of the build stage of CodePipeline.
It builds a Docker image from the code pushed to CodeCommit.
Push the built image to ECR.
Stores DockerHub account information in the SSM parameter store.
These will be used to pull the base image when generating the image with DockerBuild, after signing in to DockerHub.
The trigger for CodePipeline to be started is conditional on a push to CodeCommit.
Specifically, we will have a rule in EventBridge that satisfies the above.
CloudFormation templates files
The above configuration is built using CloudFormation.
The CloudFormation templates are located at the following URL
https://github.com/awstut-an-r/awstut-fa/tree/main/081
Explanation of key points of the template files
The purpose of this page is to add test units in the pipeline created by CodePipeline.
For basic information about CodePipeline, please refer to the following page.
Use CloudFormation custom resources to automatically delete objects in S3 buckets and images in ECR repositories when deleting CloudFormation stacks.
For more information, please refer to the following page
ReportGroup
Resources:
CodeBuildReportGroup:
Type: AWS::CodeBuild::ReportGroup
Properties:
DeleteReports: true
ExportConfig:
ExportConfigType: NO_EXPORT
Name: !Sub "${CodeBuildProject1}-${ReportName}"
Type: TEST
Code language: YAML (yaml)To run test units in CodeBuild, create a report group.
Set the type of report group in the Type property.
If “CODE_COVERAGE” is specified, a report on code coverage rate will be created.
When “TEST” is specified, a normal test report is created.
The latter is selected this time.
The DeleteReports property is a parameter for deleting reports.
Enabling this will allow you to delete the group and the report together, even if the report remains inside the group.
ExportConfig is a parameter related to exporting report groups.
It allows exporting the raw report data to an S3 bucket.
We will not export it this time.
CodeBuild
Resources:
CodeBuildProject1:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: CODEPIPELINE
Cache:
Type: NO_CACHE
Environment:
ComputeType: !Ref ProjectEnvironmentComputeType
EnvironmentVariables:
- Name: REPORT_NAME
Type: PLAINTEXT
Value: !Ref ReportName
Image: !Ref ProjectEnvironmentImage
ImagePullCredentialsType: CODEBUILD
Type: !Ref ProjectEnvironmentType
PrivilegedMode: true
LogsConfig:
CloudWatchLogs:
GroupName: !Ref LogGroup
Status: ENABLED
S3Logs:
Status: DISABLED
Name: !Sub "${Prefix}-project-01"
ServiceRole: !GetAtt CodeBuildRole.Arn
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.2
phases:
install:
runtime-versions:
python: 3.7
commands:
- pip3 install pytest
- pip3 install bottle
build:
commands:
- python -m pytest --junitxml=reports/pytest_reports.xml
reports:
$REPORT_NAME:
files:
- pytest_reports.xml
base-directory: reports
file-format: JUNITXML
Visibility: PRIVATE
Code language: YAML (yaml)Set the environment in which to run the test in the Environment property.
Specify the specifications of the build environment in the ComputeType property.
In this case, specify “BUILD_GENERAL1_SMALL” for the minimum configuration (4GB memory, 2vCPU, 50GB storage).
https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-compute-types.html
The EnvironmentVariables property allows you to set environment variables that can be used in the build environment.
In this case, we will set the name of the aforementioned report group.
The Image property specifies the Docker image for creating the build environment.
This time, specify “aws/codebuild/amazonlinux2-aarch64-standard:2.0”, which is the ARM version of Amazon Linux 2.
https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html
The ImagePullCredentialsType property sets the credentials for pulling images.
Setting “CODEBUILD” will use your own credentials.
The Type property sets the type of build environment.
In this case, set “ARM_CONTAINER” which means ARM environment.
The PrivilegedMode property is a parameter related to the execution of the Docker daemon.
Enable it this time as shown in the following quotation.
Enables running the Docker daemon inside a Docker container. Set to true only if the build project is used to build Docker images. Otherwise, a build that attempts to interact with the Docker daemon fails.
AWS::CodeBuild::Project Environment
Define the contents of buildspec.yml in the Source property.
Use pytest to test your Python scripts.
In this case, we used the following page as a reference for the configuration.
https://docs.aws.amazon.com/codebuild/latest/userguide/test-report-pytest.html
The following are the IAM roles for CodeBuild.
Resources:
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- codebuild.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser
Policies:
- PolicyName: PipelineExecutionPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- ssm:GetParameters
Resource:
- !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${SSMParameterDockerHubPassword}"
- !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${SSMParameterDockerHubUsername}"
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketAcl
- s3:GetBucketLocation
Resource:
- !Sub "arn:aws:s3:::${BucketName}"
- !Sub "arn:aws:s3:::${BucketName}/*"
- Effect: Allow
Action:
- codebuild:CreateReportGroup
- codebuild:CreateReport
- codebuild:UpdateReport
- codebuild:BatchPutTestCases
- codebuild:BatchPutCodeCoverages
Resource:
- !Sub "arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:report-group/*"
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource:
- !GetAtt LogGroup.Arn
- !Sub
- "${LogGroupArn}:log-stream:*"
- LogGroupArn: !GetAtt LogGroup.Arn
Code language: YAML (yaml)Grant permissions on report groups.
CodePipeline
Resources:
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref BucketName
Type: S3
Name: !Ref Prefix
RoleArn: !GetAtt CodePipelineRole.Arn
Stages:
- Actions:
- ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeCommit
Version: 1
Configuration:
BranchName: !Ref BranchName
OutputArtifactFormat: CODE_ZIP
PollForSourceChanges: false
RepositoryName: !GetAtt CodeCommitRepository.Name
Name: Source
OutputArtifacts:
- Name: !Ref PipelineSourceArtifact
Region: !Ref AWS::Region
RunOrder: 1
Name: Source
- Actions:
- ActionTypeId:
Category: Test
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref CodeBuildProject1
InputArtifacts:
- Name: !Ref PipelineSourceArtifact
Name: Test
OutputArtifacts: []
Region: !Ref AWS::Region
RunOrder: 1
Name: Test
- Actions:
- ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref CodeBuildProject2
InputArtifacts:
- Name: !Ref PipelineSourceArtifact
Name: Build
OutputArtifacts:
- Name: !Ref PipelineBuildArtifact
Region: !Ref AWS::Region
RunOrder: 1
Name: Build
Code language: YAML (yaml)Define test stages in the Stages property.
Within the Configuration property, configure the detailed settings for the test stage.
In the ProjectName property, specify the aforementioned CodeBuild.
In the InputArtifacts property, specify the scripts, etc. to be tested. In this case, specify the artifacts of the Source stage.
(Reference) File for application container
Dockerfile
FROM amazonlinux
RUN yum update -y && yum install python3 python3-pip -y
RUN pip3 install bottle
COPY main.py ./
CMD ["python3", "main.py"]
EXPOSE 8080
Code language: Dockerfile (dockerfile)The image for the app container will be based on Amazon Linux 2.
We will use Bottle, a Python web framework.
So after installing Python and pip, install this.
Copy the Python script (main.py) describing the app logic and set this to run.
As mentioned earlier, the app listens for HTTP requests on 8080/tcp, so expose this port.
main.py
from bottle import route, run
@route('/')
def hello():
return 'Hello CodePipeline.'
if __name__ == '__main__':
run(host='0.0.0.0', port=8080)
Code language: Python (python)We will use Bottle to build a simple web server.
The simple configuration is to listen for HTTP requests at 8080/tcp and return “Hello CodePipeline.
test_main.py
import pytest
from main import hello
@pytest.mark.parametrize(('expected',), [
('Hello CodePipeline.',),
])
def test_hello(expected):
assert hello() == expected
Code language: Python (python)This is a script to test the hello function defined in main.py in pytest.
The hello function is executed and if it returns the string “Hello CodePipeline.
Architecting
Using CloudFormation, we will build this environment and check the actual behavior.
Create CloudFormation stacks and check resources in stacks
Create a CloudFormation stacks.
Please refer to the following pages for information on how to create stacks and check each stack.
After reviewing the resources in each stack, the following is the information on the main resources created in this case.
- ECR: fa-081
- CodeCommit: fa-081
- CodeBuild1: fa-081-project-01
- CodeBuild2: fa-081-project-02
- CodePipeline: fa-081
Check each resource from the AWS Management Console.
Check CodePipeline.
The pipeline has failed to execute.
This is because CodeCommit was created when the CloudFormation stack was created, which triggered the execution of the pipeline.
Since we are not pushing code to CodeCommit at this time, an error occurred during the pipeline execution process.
Note the stage that is being created.
A stage has been created for the test unit, named Test.
When the code is pushed to CodeCommit, the test unit is executed, and if the Operation is checked, the Docker image is built and pushed to the ECR repository.
Check Actions
In case of test succeed
Now that we are ready, we push the code to CodeCommit.
First, pull CodeCommit.
$ git clone https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-081
Cloning into 'fa-081'...
warning: You appear to have cloned an empty repository.
Code language: Bash (bash)An empty repository has been pulled.
Add three files (Dockerfile, main.py, and test_main.py) to the repository.
$ ls -al
total 12
drwxrwxr-x 3 ec2-user ec2-user 71 Aug 21 11:24 .
drwxrwxr-x 6 ec2-user ec2-user 125 Aug 21 11:24 ..
-rw-rw-r-- 1 ec2-user ec2-user 187 Aug 12 11:01 Dockerfile
drwxrwxr-x 7 ec2-user ec2-user 119 Aug 21 11:24 .git
-rw-rw-r-- 1 ec2-user ec2-user 681 Aug 21 08:07 main.py
-rw-rw-r-- 1 ec2-user ec2-user 233 Aug 21 08:34 test_main.py
Code language: Bash (bash)Push the 3 files to CodeCommit.
$ git add .
$ git commit -m 'first commit'
...
3 files changed, 51 insertions(+)
create mode 100644 Dockerfile
create mode 100644 main.py
create mode 100644 test_main.py
$ git push
...
* [new branch] master -> master
Code language: Bash (bash)The push was successful.
After waiting for a while, check CodePipeline again.
Pipeline has been started.
The Source stage has been successfully completed and the Test stage has been reached.
It now says “In progress,” indicating that the test unit is being executed.
Wait for a while.
After the test stage completes successfully, the build stage also completes successfully.
Check the report group of the test.
A report group is created.
A summary of the results of the test unit execution is available.
Check the details of the test executed this time.
Test results can be displayed in graphs and tables.
You can also check the phase transition process during testing.
This time everything was completed successfully.
You can also check the detailed logs of the test execution.
Check the ECR repository.
The image has been pushed. The test has completed successfully, which means that we have moved to the build stage, the Docker image has been generated and pushed to the repository.
In case of test failure
Check the behavior when the test fails for reference.
Modify the string output by the function to make the test fail on a dare.
On CodePipeline, you can see that the test failed.
Check the report group for a summary of failed tests.
Then check the details.
We can see that it failed in the build phase.
The log confirms the cause of the failure.
The log shows that the returned string was not what was expected.
You can see the same details from the test case details page.
Summary
We have shown you how to add a test stage on CodePipeline.