Use TaskCat to automate testing of CloudFormation templates
The following official AWS page introduces a method for automating the testing of CloudFormation templates.
https://aws.amazon.com/solutions/implementations/taskcat-ci/?nc1=h_ls
The significance of this structure is explained as follows
This AWS Solution deploys TaskCat continuous integration and continuous delivery (CI/CD) pipeline for AWS CloudFormation on the Amazon Web Services (AWS) Cloud. It is for users that want to automatically test and deploy CloudFormation templates from a GitHub repository using TaskCat.
TaskCat CI/CD Pipeline for AWS CloudFormation
This page shows a configuration based on the above.
Test the CloudFormation template pushed to a development branch on CodeCommit using TaskCat.
If the test is successful, the pipeline will continue and merge the development branch with the master branch.
Environment
Configure a pipeline in CodePipeline.
The pipeline works with the following flow
The first is CodeCommit.
CodeCommit is responsible for the source stage of CodePipeline.
Use it as a Git repository.
The repository stores CloudFormation template files and other files.
The second is CodeBuild.
CodeBuild is responsible for the build stage of CodePipeline.
TaskCat is used to test template files.
Save the test results to an S3 bucket.
The third is SNS.
SNS is responsible for the approval stage of CodePipeline.
Email notification via SNS.
A user who receives an email notification can visit the pipeline page and press the approve button to resume the pipeline.
The fourth is the Lambda function.
The Lambda function is responsible for the deployment stage of CodePipeline.
The function’s work is to merge the development branch and the master branch with the branch.
CloudFormation template files
The above configuration is built with CloudFormation.
The CloudFormation templates are placed at the following URL
https://github.com/awstut-an-r/awstut-fa/tree/main/137
Explanation of key points of template files
CodePipeline
Resources:
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref ArtifactBucket
Type: S3
Name: !Ref Prefix
RoleArn: !GetAtt CodePipelineRole.Arn
Stages:
- Actions:
- ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeCommit
Version: 1
Configuration:
BranchName: !Ref SourceBranch
OutputArtifactFormat: CODE_ZIP
PollForSourceChanges: false
RepositoryName: !GetAtt CodeCommitRepository.Name
Name: SourceAction
OutputArtifacts:
- Name: !Ref PipelineSourceArtifact
Region: !Ref AWS::Region
RunOrder: 1
Name: Source
- Actions:
- ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref CodeBuildProject
InputArtifacts:
- Name: !Ref PipelineSourceArtifact
Name: Build
OutputArtifacts:
- Name: !Ref PipelineBuildArtifact
Region: !Ref AWS::Region
RunOrder: 2
Name: Build
- Actions:
- ActionTypeId:
Category: Approval
Owner: AWS
Provider: Manual
Version: 1
Configuration:
ExternalEntityLink: !Sub "https://${AWS::Region}.console.aws.amazon.com/s3/buckets/${ArtifactBucket}?region=${AWS::Region}&tab=objects"
NotificationArn: !Ref TopicArn
Name: Approval
Region: !Ref AWS::Region
RunOrder: 3
Name: Approval
- Actions:
- ActionTypeId:
Category: Invoke
Owner: AWS
Provider: Lambda
Version: 1
Configuration:
FunctionName: !Ref GitMergeFunction
Name: GitMerge
Region: !Ref AWS::Region
RunOrder: 4
Name: Deploy
CodePipelineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: PipelinePolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- s3:GetBucketVersioning
- s3:ListBucket
- s3:ListBucketVersions
Resource:
- !Sub "arn:aws:s3:::${ArtifactBucket}"
- Effect: Allow
Action:
- s3:GetObject
- s3:GetObjectVersion
- s3:PutObject
Resource:
- !Sub "arn:aws:s3:::${ArtifactBucket}/*"
- Effect: Allow
Action:
- cloudformation:CreateChangeSet
- cloudformation:CreateStack
- cloudformation:DeleteChangeSet
- cloudformation:DeleteStack
- cloudformation:DescribeChangeSet
- cloudformation:DescribeStacks
- cloudformation:ExecuteChangeSet
- cloudformation:SetStackPolicy
- cloudformation:UpdateStack
- cloudformation:ValidateTemplate
Resource: !Sub "arn:${AWS::Partition}:cloudformation:*:*:*"
- Effect: Allow
Action:
- iam:PassRole
Resource: "*"
Condition:
StringEquals:
iam:PassedToService: cloudformation.amazonaws.com
- Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
Resource: !GetAtt CodeBuildProject.Arn
- Effect: Allow
Action:
- lambda:invokeFunction
Resource:
- !Sub "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:${GitMergeFunction}"
- Effect: Allow
Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetRepository
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
Resource:
- !GetAtt CodeCommitRepository.Arn
- Effect: Allow
Action:
- sns:Publish
Resource:
- !Ref TopicArn
Code language: YAML (yaml)Create a pipeline in CodePipeline.
This pipeline consists of four stages.
Source Stage
In the reference site, the pipeline used Github as the source stage, but this time CodeCommit will be selected.
Specify the development branch in the BranchName property.
This will make the CloudFormation template files, etc. that exist on the branch the output artifact of the source stage.
The following is CodeCommit.
Resources:
CodeCommitRepository:
Type: AWS::CodeCommit::Repository
Properties:
RepositoryName: !Ref Prefix
Code language: YAML (yaml)Specify only the repository name.
No other special settings.
Build Stage
Test CloudFormation templates in CodeBuild.
The templates to test are artifacts generated in the source stage.
Specifically, they are the various files maintained on the development branch.
The following is CodeBuild.
Resources:
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: CODEPIPELINE
Cache:
Type: NO_CACHE
Environment:
ComputeType: !Ref ProjectEnvironmentComputeType
EnvironmentVariables:
- Name: ARTIFACT_BUCKET
Type: PLAINTEXT
Value: !Ref ArtifactBucket
- Name: REGION
Type: PLAINTEXT
Value: !Ref AWS::Region
- Name: REPOSITORY_URL
Type: PLAINTEXT
Value: !GetAtt CodeCommitRepository.CloneUrlHttp
- Name: RELEASE_BRANCH
Type: PLAINTEXT
Value: !Ref ReleaseBranch
- Name: SOURCE_BRANCH
Type: PLAINTEXT
Value: !Ref SourceBranch
Image: !Ref ProjectEnvironmentImage
ImagePullCredentialsType: CODEBUILD
Type: !Ref ProjectEnvironmentType
PrivilegedMode: true
LogsConfig:
CloudWatchLogs:
GroupName: !Ref LogGroup
Status: ENABLED
S3Logs:
Status: DISABLED
Name: !Sub "${Prefix}-project"
ServiceRole: !GetAtt CodeBuildRole.Arn
Source:
Type: CODEPIPELINE
BuildSpec: |
version: 0.2
env:
shell: bash
phases:
install:
runtime-versions:
python: 3.x
commands:
- echo "Entered the install phase..."
- echo "Installing system dependencies..."
- echo "Installing python dependencies..."
- pip3 -q install taskcat
pre_build:
commands:
- echo "Entered the pre_build phase..."
- echo "Current directory is $CODEBUILD_SRC_DIR"
- ls -lA
- dirname=`pwd`
- echo "Directory name $dirname"
- ls -lA
- echo "Verifying TaskCat installation..."
- taskcat
build:
commands:
- echo "Entered the build phase..."
- echo "Running TaskCat tests..."
- taskcat test run
- |
if $(grep -Fq "CREATE_FAILED" taskcat_outputs/index.html)
then
echo "Build failed!"
exit 1
else
echo "Build passed!"
exit 0
fi
finally:
- ls -1 taskcat_outputs
- ls -1 taskcat_outputs | while read LOG; do cat taskcat_outputs/$LOG; done
- >- # Do not remove
echo "Zipping and uploading report to S3 bucket: '$ARTIFACT_BUCKET'..."
- zip -r taskcat_report.zip taskcat_outputs
- aws s3 cp taskcat_report.zip s3://$ARTIFACT_BUCKET/taskcat_reports/$CODEBUILD_BUILD_ID.zip
Visibility: PRIVATE
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- codebuild.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- !Sub "arn:${AWS::Partition}:iam::aws:policy/AdministratorAccess"
Code language: YAML (yaml)For more information on CodeBuild, please see the following pages
The buildspec.yml is the key.
Generally the same as the reference site, but with the following differences
- Change the CodeBuild environment from ubuntu-based to Amazon Linux-based.
- Instead of testing templates pulled from Github, test the ones on CodeCommit as artifacts from the previous stage.
- No pre-commit, etc. is installed or performed.
Save the results of the Taskcat run to an S3 bucket.
Approval Stage
Email notification via SNS.
Resources:
Topic:
Type: AWS::SNS::Topic
Properties:
Subscription:
- Endpoint: !Ref MailAddress
Protocol: email
TopicName: !Ref Prefix
Code language: YAML (yaml)Specify the email address to be notified.
No other special settings are required.
For more information on SNS, please see the following page.
Deployment Stage
Invoke the Lambda function and merge the development branch with the master branch.
Resources:
GitMergeFunction:
Type: AWS::Lambda::Function
Properties:
Environment:
Variables:
REGION: !Ref AWS::Region
RELEASE_BRANCH: !Ref ReleaseBranch
REPOSITORY: !GetAtt CodeCommitRepository.Name
SOURCE_BRANCH: !Ref SourceBranch
Code:
ZipFile: |
import boto3
import json
import os
region = os.environ['REGION']
release_branch = os.environ['RELEASE_BRANCH']
repository = os.environ['REPOSITORY']
source_branch = os.environ['SOURCE_BRANCH']
codecommit_client = boto3.client('codecommit', region_name=region)
codepipeline_client = boto3.client('codepipeline', region_name=region)
def lambda_handler(event, context):
job_id = event['CodePipeline.job']['id']
try:
merge_response = codecommit_client.merge_branches_by_fast_forward(
repositoryName=repository,
sourceCommitSpecifier=source_branch,
destinationCommitSpecifier=release_branch
)
print(merge_response)
codepipeline_client.put_job_success_result(
jobId=job_id
)
return True
except Exception as e:
print(e)
codepipeline_client.put_job_failure_result(
jobId=job_id,
failureDetails={
'type': 'JobFailed',
'message': 'Something happened.'
}
)
FunctionName: !Sub "${Prefix}-GitMergeFunction"
Handler: !Ref LambdaHandler
MemorySize: !Ref LambdaMemory
Runtime: !Ref LambdaRuntime
Role: !GetAtt GitMergeFunctionRole.Arn
Timeout: !Ref LambdaTimeout
GitMergeFunctionRole:
Type: AWS::IAM::Role
DeletionPolicy: Delete
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: GitMergePolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- codecommit:MergeBranchesByFastForward
Resource: !Sub
- "arn:${AWS::Partition}:codecommit:${AWS::Region}:${AWS::AccountId}:${Repository}*"
- Repository: !GetAtt CodeCommitRepository.Name
- Effect: Allow
Action:
- codepipeline:PutJobSuccessResult
- codepipeline:PutJobFailureResult
Resource: "*"
Code language: YAML (yaml)Define the code to be executed by the Lambda function in inline notation.
For more information, please see the following page.
In the reference site, the merge is done by HTTP access to the Github endpoint.
In this case, it is CodeCommit, so we execute the merge_branches_by_fast_forward method of the Boto3 CodeCommit client object to merge both branches.
Please note that to invoke a Lambda function within CodePipeline, you must use the put_job_success_result or put_job_failure_result method of the CodePipeline client object.
Please see the following page for more information.
EventBridge
Resources:
EventsRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.codecommit
detail-type:
- CodeCommit Repository State Change
resources:
- !GetAtt CodeCommitRepository.Arn
detail:
event:
- referenceCreated
- referenceUpdated
referenceType:
- branch
referenceName:
- !Ref SourceBranch
Name: !Ref Prefix
Targets:
- Arn: !Sub "arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}"
Id: !Sub "${Prefix}-CodePipeline-CodeCommit"
RoleArn: !GetAtt EventsRuleRole.Arn
EventsRuleRole:
Type: AWS::IAM::Role
DeletionPolicy: Delete
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: PipelineExecutionPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- codepipeline:StartPipelineExecution
Resource:
- !Sub "arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}"
Code language: YAML (yaml)Use EventBridge to trigger the start of a CodePipeline.
Create EventBridge rules and IAM roles by referring to the following page.
Specifically, it triggers when it detects a creation or update event on a development branch.
(Reference) Test template
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Handler:
Type: String
Memory:
Type: Number
Prefix:
Type: String
Runtime:
Type: String
Resources:
Function:
Type: AWS::Lambda::Function
Properties:
Environment:
Variables:
REGION: !Ref AWS::Region
MEMORY: !Ref Memory
Code:
ZipFile: |
import json
import os
region = os.environ['REGION']
memory = os.environ['MEMORY']
def lambda_handler(event, context):
data = {
'region': region,
'memory': memory
}
return {
'statusCode': 200,
'body': json.dumps(data, indent=2)
}
FunctionName: !Sub "${Prefix}-function"
Handler: !Ref Handler
MemorySize: !Ref Memory
Runtime: !Ref Runtime
Role: !GetAtt FunctionRole.Arn
Function2:
Type: AWS::Lambda::Function
Properties:
Environment:
Variables:
REGION: !Ref AWS::Region
MEMORY: !Ref Memory
Code:
ZipFile: |
import json
import os
region = os.environ['REGION']
memory = os.environ['MEMORY']
def lambda_handler(event, context):
data = {
'region': region,
'memory': memory
}
return {
'statusCode': 200,
'body': json.dumps(data, indent=2)
}
FunctionName: !Sub "${Prefix}-function2"
Handler: !Ref Handler
MemorySize: !Ref Memory
Runtime: !Ref Runtime
Role: !GetAtt FunctionRole.Arn
FunctionRole:
Type: AWS::IAM::Role
DeletionPolicy: Delete
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Code language: YAML (yaml)Sample CloudFormation template.
This is the target to test with Taskcat.
This section describes the creation of Lambda functions and IAM roles.
.taskcat.yml
project:
name: taskcat-ci
regions:
- ap-northeast-1
tests:
default:
parameters:
Handler: index.lambda_handler
Memory: 128
Prefix: taskcat-ci
Runtime: python3.8
template: ./test-template.yaml
Code language: YAML (yaml)Taskcat configuration file.
Specify the aforementioned nameplate as the test target.
The test stack is created in the ap-northeast-1 region.
Specify parameters for test stack creation in parameters.
In this case, four parameters are passed.
Architecting
Use CloudFormation to build this environment and check its actual behavior.
Create CloudFormation stacks and check the resources in the stacks
Create CloudFormation stacks.
For information on how to create stacks and check each stack, please see the following page.
After reviewing the resources in each stack, information on the main resources created in this case is as follows
- CodePipeline: fa-137
- CodeCommit repository: fa-137
- CodeBuild project: fa-137
- S3 bucket: fa-137
- Lambda function: fa-137-GitMergeFunction
- SNS Topic: fa-137
Email Address Authentication
If you specify an email address as a subscriber to an SNS topic, you must authenticate that email address.
The following authentication email will be sent to the specified email address.
Press “Confirm subscription” to proceed with the authentication.
The above page will appear, indicating that authentication has been completed.
Resource Acknowledgement
Check the created resource from the AWS Management Console.
Check the SNS.
Indeed, SNS topic has been created.
Check CodeCommit.
Indeed, a repository has been created.
Check CodeBuild.
Indeed, a CodeBuild project has been created.
Check the Lambda function.
Created successfully.
Check CodePipeline.
The pipeline has failed to execute.
This is because the pipeline was triggered by the creation of CodeCommit when the CloudFormation stack was created.
An error occurred during the pipeline execution process because we are not pushing any code to CodeCommit at this time.
Operation Check
Now that you are ready, push the code to CodeCommit.
First pull the CodeCommit repository.
$ git clone https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-137
Cloning into 'fa-137'...
warning: You appear to have cloned an empty repository.Code language: Bash (bash)An empty repository has been pulled.
Commit and push the master branch.
(master) $ git commit --allow-empty -m "initial commit"
[master (root-commit) 3aee162] initial commit
...
(master) $ git push
...
To https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-137
* [new branch] master -> master
...Code language: Bash (bash)The file does not exist, but has been committed and pushed.
This procedure is necessary to create a development branch and merge both branches as described below.
Create a development branch.
(master) $ git branch dev
(master) $ git branch -a
dev
* master
remotes/origin/masterCode language: Bash (bash)Switch to the development branch.
(master) $ git checkout dev
Switched to branch 'dev'Code language: Bash (bash)Place templates for testing and files for Taskcat.
(dev) $ ls -al
total 8
drwxrwxr-x 3 ec2-user ec2-user 64 Jul 7 11:56 .
drwxrwxr-x 3 ec2-user ec2-user 20 Jul 7 11:43 ..
drwxrwxr-x 8 ec2-user ec2-user 166 Jul 7 11:55 .git
-rw-rw-r-- 1 ec2-user ec2-user 264 Jul 4 23:12 .taskcat.yml
-rw-rw-r-- 1 ec2-user ec2-user 3426 Jul 7 11:26 test-template.yamlCode language: Bash (bash)Push the two files to the development branch.
(dev) $ git add .
(dev) $ git commit -m 'first commit'
[dev 8fee41a] first commit
...
2 files changed, 146 insertions(+)
create mode 100644 .taskcat.yml
create mode 100644 test-template.yaml
(dev) $ git push --set-upstream origin dev
...
To https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/fa-137
* [new branch] dev -> dev
branch 'dev' set up to track 'origin/dev'.Code language: Bash (bash)I was able to push successfully.
Check CodeCommit again.
Indeed, two files are being pushed.
Check CodePipeline.
CodePipeline has been launched.
This was triggered by an updated development branch in CodeCommit.
The Source stage is complete.
Then the Build stage is started.
After a short wait, CodeBuild completes successfully and the Build stage also ends.
Pause at the Approval stage.
The following message was sent to the email address specified for the SNS topic.
Review for approval.
Access the “Content to review” URL.
The ZIP file is saved.
Check the contents.
Here are the results of the test by Taskcat.
CloudFormation successfully generated a Lambda function and IAM role in the ap-northeast-1 region.
Now that we have verified that the template is OK, we perform the approval action.
This will restart the pipeline.
After a short wait, the Deploy stage is also completed.
Check the execution log of the Lambda function.
You can see that it was successfully executed.
This means that the merge between the development branch and the master branch was successful.
Finally, check CodeCommit.
In addition to the development branch, there is evidence of commits on the master branch.
The two files are stored in the master branch.
This means that the two branches have been merged so that the files that were in the development branch are now also in the master branch.
Summary
CloudFormation templates pushed to a development branch on CodeCommit were tested using TaskCat.
If the test was successful, the pipeline continued and the development and master branches were merged.