AWS_EN

Enable Container Insights in Fargate and set alarms with traffic

スポンサーリンク
Enable Container Insights in Fargate and set alarms with traffic. AWS_EN
スポンサーリンク
スポンサーリンク

Enable Container Insights in Fargate and set alarms with traffic

In the following page, we introduced a configuration for Fargate type ECS that uses CloudWatch alarms to send email notifications when CPU usage exceeds a threshold value.

You can get more detailed metrics by enabling the Container Insight feature for ECS clusters.

This time, we aim to use the traffic information obtained by Container Insight to notify via CloudWatch alarm via email.

Environment

Diagram of enable Container Insights in Fargate and set alarms with traffic.

Create a Fargate type ECS on a private subnet.
Enable Container Insight for the ECS cluster.

Create a VPC endpoint for metrics on the container subnet to deliver metrics from the private subnet to CloudWatch.

Set thresholds in CloudWatch alarms for network send/receive volume.
Set up SNS to publish a message when communication volume exceeds 100Bytes/Sec.

Set the SNS as a subscriber with an email address.

Create an EC2 instance.
Use it as a client to access the container.

Create a NAT gateway for two purposes.
The first is to get an official Nginx image from DockerHub in order to create an ECS container.
The second is to install Apache Bench on the EC2 instance, which will run against the ECS container and generate a large number of requests, which will increase CPU usage and raise alarms.

Once the above uses are completed, the NAT gateway is no longer needed.
In this case, we will use a CloudFormation custom resource and configure it to eventually remove the NAT gateway, etc.

CloudFormation Template Files

The above configuration is built using CloudFormation.
The CloudFormation templates are located at the following URL

awstut-fa/072 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Explanation of key points of the template files

This configuration is basically the same as the page introduced at the beginning of this document.

This page focuses on how to enable Container Insight and set alarms from the metrics that can be obtained from it.

Enabling Container Insight

Resources: Cluster: Type: AWS::ECS::Cluster Properties: ClusterName: !Sub "${Prefix}-cluster" ClusterSettings: - Name: containerInsights Value: enabled
Code language: YAML (yaml)

Container Insight is enabled on the cluster resource.

In the ClusterSettings property, set “containerInsights” to Name and “enabled” to Value to enable Container Insight.

CloudWatch Alarm

Resources: Alarm1: Type: AWS::CloudWatch::Alarm Properties: AlarmActions: - !Ref TopicArn ComparisonOperator: GreaterThanThreshold Dimensions: - Name: ClusterName Value: !Ref ClusterName - Name: ServiceName Value: !Ref ServiceName EvaluationPeriods: 1 MetricName: NetworkRxBytes Namespace: ECS/ContainerInsights Period: 60 Statistic: Average Threshold: 100 Unit: Bytes/Second Alarm2: Type: AWS::CloudWatch::Alarm Properties: AlarmActions: - !Ref TopicArn ComparisonOperator: GreaterThanThreshold Dimensions: - Name: ClusterName Value: !Ref ClusterName - Name: ServiceName Value: !Ref ServiceName EvaluationPeriods: 1 MetricName: NetworkTxBytes Namespace: ECS/ContainerInsights Period: 60 Statistic: Average Threshold: 100 Unit: Bytes/Second
Code language: YAML (yaml)

To set CloudWatch alarms using metrics available from Container Insight, set the Namespace property to “ECS/ContainerInsights”.

To condition the metrics to be retrieved on the cluster service level, specify “ClusterName” and “ServiceName” for the Dimensions property.

Network-related information that can be retrieved from Container Insight includes “NetworkRxBytes” and “NetworkTxBytes”.
These are the amount of data received and sent, respectively.
Set the MetricName properties of the two alarms, respectively.

Set the alarm conditions in the ComparisonOperator, EvaluationPeriods, Period, Statistic, and Threshold properties.
In summary, the system evaluates the amount of data received/sent every 60 seconds, and if the average value of the metrics for the past one time exceeds 100Bytes/Sec, the alarm condition is satisfied.

Architecting

Using CloudFormation, we will build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • SNS topic: fa-072
  • ECS cluster: fa-072-cluster
  • ECS service: fa-072-service
  • EC2 instance: i-06b8f677825287fba

Authentication of email address

If you have specified an email address as a subscriber to an SNS topic, you must authenticate that email address.

For more information, please refer to the following page.

Resource Confirmation

Check each resource from the AWS Management Console.

Check the SNS topic.

Detail of SNS.

You can see that the SNS topic has been successfully created.

In addition, you can see that the email address registered as a subscriber is registered.
The Status value of the email address is “Confirmed,” indicating that the authentication has been completed.

Next, check ECS (Fargate).

Detail of ECS(Fargate) 1.
Detail of ECS(Fargate) 2.

The ECS cluster service task has been successfully created.
The latest version of Nginx image is obtained from Dockerhub and a container is created from it.
You can also see that the private address assigned to the task is “10.0.3.230”.

Check the CloudWatch alarms.
First is the alarm for the amount received.

Detail of CloudWatch Alarms 1.
Detail of CloudWatch Alarms 2.

The alarm is successfully created.
Since it is currently below the threshold value of 100 Bytes/Sec, the status is “OK” and it is not in an alarm state.

Next is the alarm for the transmission volume.

Detail of CloudWatch Alarms 3.
Detail of CloudWatch Alarms 4.

This alarm is also created normally.
Since the value is currently below the threshold value of 100 Bytes/Sec, the status is “OK” and no alarm is generated.

Checking Action

Now that everything is ready, access the EC2 instance.
Use SSM Session Manager to access the instance.

% aws ssm start-session --target i-06b8f677825287fba Starting session with SessionId: root-089994be40366b049 sh-4.2$
Code language: Bash (bash)

For more information on SSM Session Manager, please refer to the following page

Access the container in the task using the curl command.

sh-4.2$ curl http://10.0.3.230/ <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> html { color-scheme: light dark; } body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
Code language: Bash (bash)

We were able to access the site successfully.
We can see that the Nginx container is in action on Fargate.

We will now increase Fargate’s send/receive volume to trigger an action on the CloudWatch alarm.
To increase the volume of incoming and outgoing traffic, we will use Apache Bench to generate a large number of requests to load the server.

First, make sure Apache is installed.

sh-4.2$ sudo yum list installed | grep httpd generic-logos-httpd.noarch 18.0.0-4.amzn2 @amzn2-core httpd.aarch64 2.4.54-1.amzn2 @amzn2-core httpd-filesystem.noarch 2.4.54-1.amzn2 @amzn2-core httpd-tools.aarch64 2.4.54-1.amzn2 @amzn2-core
Code language: Bash (bash)

It appears that Apache has been successfully installed by the user data.

Run Apache Bench.
Generate 100,000 requests for a container (task) on Fargate.

sh-4.2$ ab -n 100000 http://10.0.3.230/ This is ApacheBench, Version 2.3 <$Revision: 1901567 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 10.0.3.230 (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: nginx/1.23.1 Server Hostname: 10.0.3.230 Server Port: 80 Document Path: / Document Length: 615 bytes Concurrency Level: 1 Time taken for tests: 35.863 seconds Complete requests: 100000 Failed requests: 0 Total transferred: 84800000 bytes HTML transferred: 61500000 bytes Requests per second: 2788.38 [#/sec] (mean) Time per request: 0.359 [ms] (mean) Time per request: 0.359 [ms] (mean, across all concurrent requests) Transfer rate: 2309.13 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 1 Processing: 0 0 0.0 0 1 Waiting: 0 0 0.0 0 1 Total: 0 0 0.1 0 1 Percentage of the requests served within a certain time (ms) 50% 0 66% 0 75% 0 80% 0 90% 0 95% 1 98% 1 99% 1 100% 1 (longest request)
Code language: Bash (bash)

The load caused by the Apache Bench should have increased the amount of data sent and received.

Check CloudWatch alarms again.

Detail of CloudWatch Alarms 5.
Detail of CloudWatch Alarms 6.
Detail of CloudWatch Alarms 7.
Detail of CloudWatch Alarms 8.

The amount of data sent/received has exceeded 100 Bytes/Sec.
Since the threshold has been exceeded, the two alarm states have been changed from “OK” to “In alarm”.
The history shows that a message was sent to the SNS as an Action due to the alarm status.

You will immediately receive the following two emails to your registered address.

Detail of CloudWatch Alarms 9.
Detail of CloudWatch Alarms 10.

The body of the email is the content of the CloudWatch alarm.
In this way, we were able to use the metrics obtained by Container Insight to send email notifications via CloudWatch alarms.

Summary

We have confirmed how to use the metrics obtained by Container Insight to send email notifications via CloudWatch alarms.

タイトルとURLをコピーしました