AWS_EN

Deliver Logs of Fargate containers in Private Subnets to CloudWatch Logs

スポンサーリンク
Deliver logs of Fargate Containers in Private Subnets to CloudWatch Logs AWS_EN
スポンサーリンク
スポンサーリンク

Deliver Logs of ECS(Fargate) containers in Private Subnets to CloudWatch Logs

One way to collect logs of containers running in an ECS task is to use CloudWatch Logs.

In this article, we will check how to distribute logs to CloudWatch Logs.

Environment

Diagram of deliver logs of Fargate containers in private subnets to CloudWatch Logs.

Create a Fargate type ECS on a private subnet.

Create a VPC endpoint for Logs on the container subnet to distribute logs to CloudWatch Logs.

Place a NAT gateway on the public subnet to retrieve the official Nginx image from DockerHub.

Create an EC2 instance.
Use it as a client to access the container.

CloudFormation template files

Build the above configuration with CloudFormation.
The CloudFormation templates are located at the following URL

https://github.com/awstut-an-r/awstut-fa/tree/main/065

Explanation of key points of template files

This page focuses on how to distribute logs to CloudWatch Logs.

For basic information on ECS (Fargate), please refer to the following page

For information on how to deploy Fargate on a private subnet, please see the following page

VPC Endpoints for CloudWatch Logs

Resources:
  CloudWatchLogsEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref EndpointSecurityGroup2
      ServiceName: !Sub "com.amazonaws.${AWS::Region}.logs"
      SubnetIds:
        - !Ref ContainerSubnet
      VpcEndpointType: Interface
      VpcId: !Ref VPC
Code language: YAML (yaml)

In order to deliver Fargate logs located on a private subnet to CloudWatch Logs, create a VPC endpoint for CloudWatch Logs.

Below is the security group to be applied to this endpoint.

Resources:
  EndpointSecurityGroup2:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub "${Prefix}-EndpointSecurityGroup2"
      GroupDescription: Allow HTTPS from ContainerSecurityGroup.
      VpcId: !Ref VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: !Ref HTTPSPort
          ToPort: !Ref HTTPSPort
          SourceSecurityGroupId: !Ref ContainerSecurityGroup
Code language: YAML (yaml)

The contents allow inbound communication of 443/tcp with the security group applied to Fargate as the source.

CloudWatch Logs Log Group

Resources:
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties: 
      LogGroupName: !Sub "${Prefix}-LogGroup"
Code language: YAML (yaml)

Create a simple log group.
No special configuration is required to distribute logs for the Fargate container.

Log settings for ECS task

Resources:
  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      ContainerDefinitions:
        - Name: !Sub "${Prefix}-container"
          Image: nginx:latest
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !Ref LogGroup
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: !Sub "${Prefix}-container"
      Cpu: !Ref TaskCpu
      ExecutionRoleArn: !Ref FargateTaskExecutionRole
      Memory: !Ref TaskMemory
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      TaskRoleArn: !Ref TaskRole
Code language: YAML (yaml)

The key point is the LogsConfiguration property.

Specify the log driver with the LogDriver property.
To distribute logs to CloudWatch Logs, specify “awslogs”.

Set the parameters regarding the destination CloudWatch Logs in the Options property.
Set at least three parameters.
The first is awslogs-group, which specifies the destination log group.
The second, awslogs-region, specifies the region where the destination log group is created.
The third is awslogs-stream-prefix, which specifies the prefix of the stream name to be created in the log group.

Architecting

Use CloudFormation to build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • ECS cluster: fa-065-cluster
  • ECS service: fa-065-service
  • CloudWatch Logs log group: fa-065-LogGroup
  • EC2 instance: i-0be518a618c126bbf

Check the created resource from the AWS Management Console.
First, check the ECS cluster.

Detail of Fargate Cluster 1.

We can see that the cluster has been successfully created and the ECS service is in active.

Check the details of the task created in the service.

Detail of Fargate Cluster 2.

We can see that a container has been created in the task, and that it is working properly.
You can also see that a private address has been assigned to the task.
The address assigned this time is “10.0.3.77”.

Checking Action

Now that everything is ready, access the EC2 instance.

% aws ssm start-session --target i-0be518a618c126bbf

Starting session with SessionId: root-0ba9574fb6aac6aaf
sh-4.2$
Code language: Bash (bash)

For more information on SSM Session Manager, please refer to the following page

Access the container in the task using the curl command.

sh-4.2$ curl http://10.0.3.77
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Code language: Bash (bash)

We were able to access it successfully.

Then check CloudWatch Logs.

Detail of CloudWatch Logs 1.

We can see that the log group has been successfully created and a log stream has been created inside.

Check the log stream.

Detail of CloudWatch Logs 2.

You can see that the log from the EC2 instance accessed earlier has been written.
From the above, we can see that by using the VPC endpoint for CloudWatch Logs, we can distribute the logs of the Fargate container on the private subnet to CloudWatch Logs.

Summary

We have seen how to distribute logs of Fargate containers on private subnets to CloudWatch Logs.

タイトルとURLをコピーしました