AWS_EN

Four ways to initialize Linux instance

スポンサーリンク
Four ways to initialize Linux instance with CloudFormation. AWS_EN
スポンサーリンク
スポンサーリンク

Four ways to initialize a Linux instance

This article introduces the configurations that are frequently used in this site.

We will cover the following four methods of initializing an EC2 instance at build time.

  1. The method that uses user data
  2. Using cfn-init
  3. Running the Ansible Playbook in SSM State Manager
  4. Running a shell script in SSM State Manager

Environment

Diagram of four ways to initialize Linux instance.

We will execute the initialization process for the four EC2 instances using the above methods.

The following are common to all of them.

  1. Update yum.
  2. Install Apache with yum, and start and enable it.
  3. Write the instance ID in index.html and make it the root page of Apache.

Each instance should be configured to be accessible via SSH.

CloudFormation template files

We will build the above configuration using CloudFormation.

Place the CloudFormation template at the following URL.

awstut-fa/004 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Template file points

We will cover the key points of each template file to configure this environment.

Define the initialization process using user data

Define instance1 in fa-004-ec2-01.yaml.

Resources: Instance1: Type: AWS::EC2::Instance Properties: ImageId: !Ref ImageId InstanceType: !Ref InstanceType KeyName: !Ref KeyName NetworkInterfaces: - AssociatePublicIpAddress: true DeviceIndex: 0 SubnetId: !Ref PublicSubnet GroupSet: - !Ref InstanceSecurityGroup UserData: !Base64 | #!/bin/bash -xe yum update -y yum install -y httpd systemctl start httpd systemctl enable httpd ec2-metadata -i > /var/www/html/index.html
Code language: YAML (yaml)

By using user data, you can define the initialization process of the instance.

When you launch an instance in Amazon EC2, you have the option of passing user data to the instance that can be used to perform common automated configuration tasks and even run scripts after the instance starts.

Run commands on your Linux instance at launch

The UserData property allows you to define user data. The content specified in the user data must not be plain text, but must be encoded in Base64.

User data must be base64-encoded. The Amazon EC2 console can perform the base64-encoding for you or accept base64-encoded input.

Work with instance user data

This time, we will use CloudFormation’s built-in function Fn::Base64 to encode and pass the initialization process.

Use cfn-init to define the initialization process

Define instance2 in fa-004-ec2-02.yaml.

Resources: Instance2: Type: AWS::EC2::Instance Metadata: AWS::CloudFormation::Init: config: packages: yum: httpd: [] services: sysvinit: httpd: enabled: 'true' ensureRunning: 'true' commands: 001make-index.html: command: ec2-metadata -i > /var/www/html/index.html Properties: ImageId: !Ref ImageId InstanceType: !Ref InstanceType KeyName: !Ref KeyName NetworkInterfaces: - AssociatePublicIpAddress: true DeviceIndex: 0 SubnetId: !Ref PublicSubnet GroupSet: - !Ref InstanceSecurityGroup UserData: Fn::Base64: !Sub | #!/bin/bash -xe yum install -y aws-cfn-bootstrap /opt/aws/bin/cfn-init -v \ --stack ${AWS::StackName} \ --resource Instance2 \ --region ${AWS::Region}
Code language: YAML (yaml)

By using cfn-init, a kind of CloudFormation helper script, you can define the instance initialization process.

The cfn-init helper script reads template metadata from the AWS::CloudFormation::Init key and acts accordingly to:
Fetch and parse metadata from CloudFormation
Install packages
Write files to disk
Enable/disable and start/stop services

cfn-init

Define the execution of the script in user data. Define the process to be executed by the script as metadata.

Use the AWS::CloudFormation::Init type to include metadata on an Amazon EC2 instance for the cfn-init helper script. If your template calls the cfn-init script, the script looks for resource metadata rooted in the AWS::CloudFormation::Init metadata key.

AWS::CloudFormation::Init

Under the config property, define the process to be executed in the corresponding section.

The configuration is separated into sections…
The cfn-init helper script processes these configuration sections in the following order: packages, groups, users, sources, files, commands, and then services.

AWS::CloudFormation::Init

In this case, we will define it as follows.

  • packages section: Installing Apache.
  • services section: Start and activate Apache.
  • commands section: Write the instance ID into index.html to make it the root page of Apache.

Run the Ansible Playbook in SSM State Manager to initialize

Define instance3 in fa-004-ec2-03.yaml.

Resources: ApplyAnsiblePlaybooksAssociation: Type: AWS::SSM::Association Properties: AssociationName: !Sub ${Prefix}-playbook-association Name: AWS-ApplyAnsiblePlaybooks OutputLocation: S3Location: OutputS3BucketName: !Ref PlaybookBucketName OutputS3KeyPrefix: !Sub "${Prefix}/playbook-association-log" Parameters: Check: - "False" ExtraVariables: - SSM=True InstallDependencies: - "True" PlaybookFile: - !Ref PlaybookFileName SourceInfo: - !Sub '{"path": "https://${PlaybookBucketName}.s3.${AWS::Region}.amazonaws.com/${Prefix}/${PlaybookPackageName}"}' SourceType: - S3 Verbose: - -v Targets: - Key: InstanceIds Values: - !Ref Instance3 WaitForSuccessTimeoutSeconds: !Ref WaitForSuccessTimeoutSeconds
Code language: YAML (yaml)

By using the SSM State Manager, you can define the initialization process of the instance.

State Manager, a capability of AWS Systems Manager, is a secure and scalable configuration management service that automates the process of keeping your Amazon Elastic Compute Cloud (Amazon EC2) and hybrid infrastructure in a state that you define.

AWS Systems Manager State Manager

AWS recommends using SSM State Manager for initialization rather than CloudFormation’s cfn-init.

There are many patterns that you can explore between CloudFormation and State Manager, but we recommend using CloudFormation to define your AWS Resources and Systems Manager to perform configuration management… start to think about using State Manager instead of cfn-init.

Using State Manager over cfn-init in CloudFormation and its benefits

In order to initialize an instance by SSM State Manager, a State Manager association needs to be defined. The key parameter in creating the association is the SSM documentation.

An AWS Systems Manager document (SSM document) defines the actions that Systems Manager performs on your managed instances. Systems Manager includes more than 100 pre-configured documents that you can use by specifying parameters at runtime.

AWS Systems Manager documents

In instance ③, we will use Ansible Playbook to perform the initialization. Therefore, specify “AWS-ApplyAnsiblePlaybooks” in the Name property.

Also, in the OutputLocation property, specify the location where the log will be saved during processing. In this case, we will set it so that the log will be output to the same location as the Playbook file described below.

Specify the target instance to be associated with in the Targets property. In this case, we will specify it using the instance ID.

In the Parameters property, specify the file name of the Playbook file and the location of the file. In this case, I referred to Create an association that runs Ansible playbooks (CLI) for the settings.

Define the IAM role for the instance.

Resources: InstanceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: sts:AssumeRole Principal: Service: - ec2.amazonaws.com - ssm.amazonaws.com Policies: - PolicyName: SSMStateManagerPolicy PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - s3:GetObject - s3:PutObject - s3:PutObjectAcl - s3:ListBucket Resource: - !Sub "arn:aws:s3:::${PlaybookBucketName}" - !Sub "arn:aws:s3:::${PlaybookBucketName}/*" ManagedPolicyArns: - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
Code language: YAML (yaml)

In order to initialize using the SSM State Manager, you need to grant two permissions to the instance.

The first is access to the S3 bucket where the Ansible Playbook and logs are located. You can define the required permissions in the inline policy.

The second is the permissions to meet the requirements of SSM managed instances, which are the instances that can be targeted by SSM State Manager.

A managed node is any machine configured for AWS Systems Manager. You can configure Amazon Elastic Compute Cloud (Amazon EC2) instances; AWS IoT Greengrass core devices; and on-premises servers, edge devices, and virtual machines (VMs) in a hybrid environment as managed nodes.

Managed nodes

Amazon Linux 2-based instances can be treated as SSM managed instances by granting them permissions under the AWS managed policy AmazonSSMManagedInstanceCore.

Run a shell script in SSM State Manager to initialize

Define instance4 in fa-004-ec2-04.yaml.

Resources: RunShellScriptAssociation: Type: AWS::SSM::Association Properties: AssociationName: !Sub ${Prefix}-shellscript-association Name: AWS-RunShellScript OutputLocation: S3Location: OutputS3BucketName: !Ref LogBucketName OutputS3KeyPrefix: !Sub "${Prefix}/shellscript-association-log" Parameters: commands: - "sudo yum update -y" - "sudo yum install -y httpd" - "sudo systemctl start httpd" - "sudo systemctl enabled httpd" - "ec2-metadata -i > /var/www/html/index.html" Targets: - Key: InstanceIds Values: - !Ref Instance4 WaitForSuccessTimeoutSeconds: !Ref WaitForSuccessTimeoutSeconds
Code language: YAML (yaml)

Basically, it is the same as the case of the Ansible Playbook described earlier.

This time, we will simply specify “AWS-RunShellScript” in the SSM document for the purpose of executing a shell script.

You can define the command to be executed in the commands property in the Parameters property.

Architecting

Using CloudFormation, we will build this environment and check its actual behavior.

Place the Playbook in an S3 bucket

The first step is to install the Ansible Playbook file that will be executed when initializing the instance ③ in the same S3 bucket.

This time, after zipping the file, we will run it using the AWS CLI.

$ zip -r playbook.zip playbook.yml $ aws s3 cp playbook.zip s3://[bucket-name]/
Code language: Bash (bash)

Create CloudFormation stack and check resources in stacks

Create a CloudFormation stack.

For information on how to create a CloudFormation stack from the AWS CLI, please refer to the following page.

After checking the resources for each stack, the following information is available for the main resources created this time.

  • ID of instance1: i-0a9b66cd7cedd7c32
  • ID of instance2: i-0e809df5ce3202e36
  • ID of instance3: i-0972c3d05d89efeba
  • ID of instance4: i-02fb072859880b2d8

You can also check the details of each instance with the following command.

$ aws ec2 describe-instances \ --instance-ids <meta charset="utf-8">i-0a9b66cd7cedd7c32 i-0e809df5ce3202e36 i-0972c3d05d89efeba i-02fb072859880b2d8
Code language: Bash (bash)

I checked the public DNS name of each instance from the above, and this time it is as follows

  • Public DNS name for instance1: ec2-18-183-148-192.ap-northeast-1.compute.amazonaws.com
  • Public DNS name for instance2: ec2-46-51-227-221.ap-northeast-1.compute.amazonaws.com
  • Public DNS name for instance3: ec2-35-72-10-175.ap-northeast-1.compute.amazonaws.com
  • Public DNS name for instance4: ec2-35-72-10-35.ap-northeast-1.compute.amazonaws.com

Behavior check 1: Initialize the instance with user data

Now that we are ready, let’s check the actual behavior.

First, make sure that instance1 has been initialized properly.

$ curl http://ec2-18-183-148-192.ap-northeast-1.compute.amazonaws.com instance-id: i-0a9b66cd7cedd7c32
Code language: Bash (bash)

Next, access it via SSH and check the log during initialization.

$ ssh -i MyKeyPair.pem ec2-18-183-148-192.ap-northeast-1.compute.amazonaws.com __| __|_ ) _| ( / Amazon Linux 2 AMI ___|\___|___| Amazon Linux 2aws.amazon.com [ec2-user@ip-10-0-1-253 ~]$ [ec2-user@ip-10-0-1-253 ~]$ sudo cat /var/log/cloud-init-output.log ... + yum update -y ... Complete! + yum install -y httpd ... Installed: httpd.x86_64 0:2.4.48-2.amzn2 ... Complete! + systemctl start httpd + systemctl enable httpd ... + ec2-metadata -i
Code language: Bash (bash)

In /var/log/cloud-init-output.log, we were able to check the log during initialization.

When initializing with user data in this way, the log will be placed locally on the instance to be processed, unless configured otherwise.

Behavior check 2: Initialize the instance using cfn-init

First, make sure that instance2 has been initialized properly.

$ curl http://ec2-46-51-227-221.ap-northeast-1.compute.amazonaws.com instance-id: i-0e809df5ce3202e36
Code language: Bash (bash)

Next, access it via SSH and check the log during initialization.

$ ssh -i MyKeyPair.pem ec2-46-51-227-221.ap-northeast-1.compute.amazonaws.com __| __|_ ) _| ( / Amazon Linux 2 AMI ___|\___|___| Amazon Linux 2aws.amazon.com [ec2-user@ip-10-0-1-82 ~]$
Code language: Bash (bash)
[ec2-user@ip-10-0-1-82 ~]$ sudo cat /var/log/cfn-init.log 2021-10-03 05:02:50,379 [INFO] -----------------------Starting build----------------------- 2021-10-03 05:02:50,380 [DEBUG] Not setting a reboot trigger as scheduling support is not available 2021-10-03 05:02:50,381 [INFO] Running configSets: default 2021-10-03 05:02:50,382 [INFO] Running configSet default 2021-10-03 05:02:50,384 [INFO] Running config config 2021-10-03 05:02:53,990 [DEBUG] Installing/updating ['httpd'] via yum 2021-10-03 05:02:57,144 [INFO] Yum installed ['httpd'] 2021-10-03 05:02:57,144 [DEBUG] No groups specified 2021-10-03 05:02:57,144 [DEBUG] No users specified 2021-10-03 05:02:57,144 [DEBUG] No sources specified 2021-10-03 05:02:57,144 [DEBUG] No files specified 2021-10-03 05:02:57,144 [DEBUG] Running command 001make-index.html 2021-10-03 05:02:57,144 [DEBUG] No test for command 001make-index.html 2021-10-03 05:02:57,182 [INFO] Command 001make-index.html succeeded 2021-10-03 05:02:57,182 [DEBUG] Command 001make-index.html output: 2021-10-03 05:02:57,182 [DEBUG] Using service modifier: /sbin/chkconfig 2021-10-03 05:02:57,182 [DEBUG] Setting service httpd to enabled 2021-10-03 05:02:57,278 [INFO] enabled service httpd 2021-10-03 05:02:57,278 [DEBUG] Using service runner: /sbin/service 2021-10-03 05:02:57,297 [DEBUG] Starting service httpd as it is not running 2021-10-03 05:02:57,370 [INFO] Started httpd successfully 2021-10-03 05:02:57,371 [INFO] ConfigSets completed 2021-10-03 05:02:57,371 [DEBUG] Not clearing reboot trigger as scheduling support is not available 2021-10-03 05:02:57,371 [INFO] -----------------------Build complete-----------------------
Code language: Bash (bash)
[ec2-user@ip-10-0-1-82 ~]$ sudo cat /var/log/cfn-init-cmd.log 2021-10-03 05:02:50,382 P2445 [INFO] ************************************************************ 2021-10-03 05:02:50,382 P2445 [INFO] ConfigSet default 2021-10-03 05:02:50,384 P2445 [INFO] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2021-10-03 05:02:50,384 P2445 [INFO] Config config 2021-10-03 05:02:53,991 P2445 [INFO] ============================================================ 2021-10-03 05:02:53,991 P2445 [INFO] yum install httpd 2021-10-03 05:02:57,135 P2445 [INFO] -----------------------Command Output----------------------- ... 2021-10-03 05:02:57,142 P2445 [INFO] Installed: 2021-10-03 05:02:57,142 P2445 [INFO] httpd.x86_64 0:2.4.48-2.amzn2 ... 2021-10-03 05:02:57,143 P2445 [INFO] Complete! 2021-10-03 05:02:57,144 P2445 [INFO] ------------------------------------------------------------ 2021-10-03 05:02:57,144 P2445 [INFO] Completed successfully. 2021-10-03 05:02:57,144 P2445 [INFO] ============================================================ 2021-10-03 05:02:57,145 P2445 [INFO] Command 001make-index.html 2021-10-03 05:02:57,181 P2445 [INFO] Completed successfully.
Code language: Bash (bash)

We were able to check the logs during initialization in /var/log/cfn-init.log and /var/log/cfn-init-cmd.log.

As you can see, when you initialize with cfn-init, the logs will be placed locally on the instance to be processed, unless you configure otherwise.

Behavior check 3: Run Ansible Playbook in SSM State Manager to initialize the instance

First, make sure that the instance3 has been initialized correctly.

$ curl http://ec2-35-72-10-175.ap-northeast-1.compute.amazonaws.com instance-id: i-01014c1daa17be41b
Code language: Bash (bash)

Next, check the execution status of the SSM Run Command from the AWS Management Console.

From the results of the execution of ApplyAnsiblePlaybooks in SSM Run Commands, we can see that it was successful.

You can see that the Run Command has been executed for Instance3 and it has finished successfully.

You can also check the details of the execution result.

In the detail page of the result of running ApplyAnsiblePlaybooks, we can see that the process was divided into two steps.

You can see that the command was executed in two separate steps to run the Ansible Playbook.

If you enable the option, you can output the log to the S3 bucket.

s3://[bucket-name]/[folder-name]/[command-id]/[instance-id]/awsrunShellScript/runShellScript/stdout

The contents are as follows.

$ cat stdout ... Installed: ansible.noarch 0:2.9.25-1.el7 ... Running Ansible in /var/lib/amazon/ssm/i-079d3fa8b1503d6b2/document/orchestration/4649a395-5815-400d-bf21-596b1c098a20/downloads Archive: ./playbook.zip inflating: playbook.yml Using /etc/ansible/ansible.cfg as config file PLAY [all] ********************************************************************* TASK [update yum] ************************************************************** ... TASK [install the latest version of Apache] ************************************ ... TASK [start and enable Apache] ************************************************* ... TASK [make index.html] ****************************************** ... PLAY RECAP ********************************************************************* localhost : ok=4 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Code language: Bash (bash)

If you check the log, you can see that after installing Ansible, the Playbook is downloaded and the Tasks are executed in order.

When initialized with SSM State Manager in this way, the logs can be output to the S3 bucket.

Behavior Check 4: Initialize the instance by running a shell script in SSM State Manager

First, make sure that the instance4 has been initialized correctly.

$ curl http://ec2-35-72-10-35.ap-northeast-1.compute.amazonaws.com instance-id: i-01014c1daa17be41b
Code language: Bash (bash)

Next, check the execution status of the SSM Run Command from the AWS Management Console.

From the results of the execution of AWS-RunShellScript in SSM Run Commands, we can see that it was successful.

You can see that it was executed against Instance4 and was successful.

The detailed page is as follows

On the AWS-RunShellScript execution result detail page, you can also check the log.

You can see that it was executed in one step.

This one is also configured to save the execution results to an S3 bucket. The log will be placed in the following location

s3://[bucket-name]/[folder-name]/[command-id]/[instance-id]/awsrunShellScript/0.awsrunShellScript/stdout

The contents are as follows.

$ cat stdout Loaded plugins: extras_suggestions, langpacks, priorities, update-motd ... Installed: httpd.x86_64 0:2.4.48-2.amzn2 Complete!
Code language: Bash (bash)

Checking the log, we can see that the standard output of the specified command has been written.

Thus, when initialized with SSM State Manager, the log can be output to the S3 bucket.

Summary

We have identified four ways to initialize an EC2 instance.

The method using user data and cfn-init will place the logs locally on the instance to be processed.

With SSM State Manager, you can initialize it with Ansible Playbook or shell scripts, and output the logs to S3 buckets.

タイトルとURLをコピーしました