Four ways to attach EBS to EC2 instance

Four ways to attach EBS to EC2 instance
Environment
CloudFormation template files
Template file points
Architecting
Summary

Four ways to attach EBS to EC2 instance

We will check the configuration of attaching EBS (Elastic Block Store) to a Linux instance.

In this page, we will check the steps to attach EBS to a Linux instance and make it available for storage.
The procedures after attaching will follow those described in the AWS official website Make an Amazon EBS volume available for use on Linux.

Specifically, the following three patterns are identified

if it is not an instance built with Nitro System
for instances built with Nitro System
how to execute shell scripts from SSM documentation

Also, please refer to the following page for the procedure to attach EBS to a Windows instance.

Environment

Diagram of Attaching EBS to Linux instance.

Create three EC2 instances.
The type of the first instance will be T2.nano, which is not built with Nitro System, and the second and third instances will be T4g.nano, which is built with Nitro System.

CloudFormation template files

We will build the above configuration using CloudFormation.
The CloudFormation template is placed at the following URL

https://github.com/awstut-an-r/awstut-fa/tree/main/008

Template file points

Create EC2 instances

Check the EC2 to which the EBS will be attached.
The point is that you need to set the AMI according to the architecture of the instance you are creating.

Parameters:
  ImageId1:
    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
    
  ImageId2:
    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
    
  InstanceType1:
    Type: String
    
  InstanceType2:
    Type: String
    

Resources:
  Instance1:
    Type: AWS::EC2::Instance
    Properties:
      IamInstanceProfile: !Ref InstanceProfile
      ImageId: !Ref ImageId1
      InstanceType: !Ref InstanceType1
      NetworkInterfaces:
        - DeviceIndex: 0
          SubnetId: !Ref PrivateSubnet
          GroupSet:
            - !Ref InstanceSecurityGroup

  Instance2:
    Type: AWS::EC2::Instance
    Properties:
      IamInstanceProfile: !Ref InstanceProfile
      ImageId: !Ref ImageId2
      InstanceType: !Ref InstanceType2
      NetworkInterfaces:
        - DeviceIndex: 0
          SubnetId: !Ref PrivateSubnet
          GroupSet:
            - !Ref InstanceSecurityGroup
Code language: YAML (yaml)

The key settings in this configuration are the InstanceType and ImageId properties.
As mentioned earlier, the instance types are specified for the two instances this time as follows.

Instance 1: T2.nano
Instance 2: T4g.nano

Use the built-in function Fn::Ref to refer to the value in the Parameters section and set the InstanceType property of the two instances.

The two instances share the same OS and run Amazon Linux 2, but it is important to note that the two instances run Amazon Linux 2.
That is, the above instance types have different architectures, so you cannot specify the same AMI.

T2 is an x86_64 based instance.

Features:
High frequency Intel Xeon processors
Amazon EC2 Instance Types

T4g, on the other hand, is an Arm-based instance.

Features:
Custom built AWS Graviton2 Processor with 64-bit Arm Neoverse cores
Powered by the AWS Nitro System, a combination of dedicated hardware and lightweight hypervisor
Amazon EC2 Instance Types

The SSM Parameter Store can be used to obtain the latest AMI IDs for your architecture.

Want a simpler way to query for the latest Amazon Linux AMI? AWS Systems Manager Parameter Store already allows for querying the latest Windows AMI. Now, support has been expanded to include the latest Amazon Linux AMI. Each Amazon Linux AMI now has its own Parameter Store namespace that is public and describable. Upon querying, an AMI namespace returns only its regional ImageID value.
Query for the latest Amazon Linux AMI IDs using AWS Systems Manager Parameter Store

In CloudFormation, the values stored in the SSM Parameter Store can be retrieved by specifying “AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>” in the Type property of the Parameters section.

You can use the existing Parameters section of your CloudFormation template to define Systems Manager parameters, along with other parameters. Systems Manager parameters are a unique type that is different from existing parameters because they refer to actual values in the Parameter Store. The value for this type of parameter would be the Systems Manager (SSM) parameter key instead of a string or other value. CloudFormation will fetch values stored against these keys in Systems Manager in your account and use them for the current stack operation.
Integrating AWS CloudFormation with AWS Systems Manager Parameter Store

This time, the latest AMI ID for each architecture is obtained by referencing values from the SSM Parameter Store under the following conditions.

Instance 1: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2
Instance 2: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-arm64-gp2

As of October 2021, the AMI IDs that can be obtained when accessing the SSM Parameter Store under the above conditions are as follows

Instance 1
- AMI ID: ami-0701e21c502689c31
- Virtualization type: hvm
Instance 2
- AMI ID: ami-0427ff21031d224a8
- Virtualization type: hvm

The virtualization type is information related to the root device name described below.

Attaching EBS to instances

Find out how to attach EBS to an EC2 instance.

Resources:
  EBS1:
    Type: AWS::EC2::Volume
    Properties:
      AvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone}" 
      Size: !Ref VolumeSize
      VolumeType: !Ref VolumeType
      
  EBSAttachment1:
    Type: AWS::EC2::VolumeAttachment
    Properties:
      Device: !Sub "/dev/sd${DeviceNameSuffix}"
      InstanceId: !Ref Instance1
      VolumeId: !Ref EBS1
Code language: YAML (yaml)

First, define the EBS resource.
Specify the AZ in which to place the EBS in the AvailabilityZone property; the EBS must be placed in the same AZ as the EC2 instance to which it is attached.

You can attach an available EBS volume to one or more of your instances that is in the same Availability Zone as the volume.
Attach an Amazon EBS volume to an instance

Follow the above and use the built-in function Fn::Sub to specify that it should be placed in the same AZ as the instance.
Specify the volume type of the EBS with the VolumeType property. The following five volume types can be specified.

General Purpose SSD
Provisioned IOPS SSD
Throughput Optimized HDD
Cold HDD
Magnetic (older generation)

In this case, a general-purpose SSD is selected.

There are two types of General Purpose SSD.
The first is gp2. It has two indicators (baseline and burst) for performance (IOPS), and features a mechanism that increases the baseline in proportion to the reserved capacity.

These volumes deliver single-digit millisecond latencies and the ability to burst to 3,000 IOPS for extended periods of time. Between a minimum of 100 IOPS (at 33.33 GiB and below) and a maximum of 16,000 IOPS (at 5,334 GiB and above), baseline performance scales linearly at 3 IOPS per GiB of volume size.
General Purpose SSD volumes (gp2)

The second is gp3. It provides constant performance regardless of the volume capacity.

These volumes deliver a consistent baseline rate of 3,000 IOPS and 125 MiB/s, included with the price of storage.
General Purpose SSD volumes (gp3)

From a performance standpoint, gp3 is more effective if the amount of space to be reserved is less than 1024 GiB, where 3000 IOPS can be expected at any given time.
On the other hand, if you are seeking higher performance from a general-purpose SSD, select gp2 and secure more capacity, which can be expected to improve up to 16,000 IOPS.
In this case, we specify that gp3 should be selected.

Specify the capacity to be reserved in the Size property.
For general-purpose SSDs, it is possible to specify in the range of 1 to 16384 (GiB).
In this case, we will reserve the minimum amount of 1 GiB.

Next, define the resources for attaching the EBS to the EC2 instance.

Resources:
  EBSAttachment1:
    Type: AWS::EC2::VolumeAttachment
    Properties:
      Device: !Sub "/dev/sd${DeviceNameSuffix}"
      InstanceId: !Ref Instance1
      VolumeId: !Ref EBS1
Code language: YAML (yaml)

Specify the instance and the EBS to attach in the InstanceId and VolumeId properties.
Specify the device name to be assigned to the EBS in the Device property; the possible values for the device name in the Linux AMI depend on the virtualization type. As mentioned earlier, both AMIs for the two instances are hvm type. hvm root or recommended device names are as follows

Reserved for root: /dev/sda1 or /dev/xvda
Recommended for EBS volumes: /dev/sd[f-p] *
Device names on Linux instances

Therefore, this time, “/dev/sdf” is specified as the device name for 2EBS.

No key pair configuration required if accessed using SSM Session Manager

The most common way to access a Linux instance is SSH using a key pair, but this time we will use SSM Session Manager.

Session Manager is a fully managed AWS Systems Manager capability that lets you manage your Amazon EC2 instances through an interactive one-click browser-based shell or through the AWS CLI. You can use Session Manager to start a session with an instance in your account. After the session is started, you can run bash commands as you would through any other connection type.
Connect to your Linux instance using Session Manager

In this configuration, the instance is located on a private subnet, so SSM is accessed through the VPC endpoint for SSM.
For more information, please see the following page.

Enabling EBS with SSM Documentation

We will show you the commands to enable EBS later, but in addition to executing the commands manually, you can also execute them through the SSM documentation.

Resources:
  RunShellScriptAssociation:
    Type: AWS::SSM::Association
    Properties:
      AssociationName: !Sub "${Prefix}-run-shellscript-association"
      Name: AWS-RunShellScript
      Parameters:
        commands:
          - !Sub |
            for d in $(lsblk -n -r -p | grep disk | awk '{ print $1 }' )
            do
              if  [ "$(sudo file -s -b $d)" == data ]; then
                sudo mkfs -t xfs $d
                sudo mkdir ${MountPoint}
                sudo mount $d ${MountPoint}
              fi
            done
      Targets:
        - Key: InstanceIds
          Values:
            - !Ref Instance
      WaitForSuccessTimeoutSeconds: !Ref WaitForSuccessTimeoutSeconds
Code language: YAML (yaml)

The SSM document AWS-RunShellScript can be used to execute the specified command against a Linux instance (Amazon Linux 2).
Details will be confirmed later.

Architecting

Using CloudFormation, we will build this environment and check its actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stacks.
For more information on how to create stacks and check each stack, please refer to the following page.

After checking the resources for each stack, the following information is available for the main resources created this time.

ID of Instance 1：i-064e80196c9a8065c
ID of Instance 2：i-09cfd456e6c89ea01
ID of Instance 3：i-0bbda6e4d4e4bcdc3
ID of EBS to be attached to instance 1：vol-06904bb289e054d68
ID of EBS to be attached to instance 2：vol-0972ea603af9f8b34
ID of EBS to be attached to instance 3：vol-09d7b2304f910940e

Check the EBS attach status

Check the status of each instance from the AWS Management Console.
Instances 1 and 2 will be taken up as representative.

EBS is attached to the Block Device Mapping of the instance 1.

EBS is attached to the Block Device Mapping of the instance 2.

Looking at the Block devices items of both instances, we can see that two EBSs are attached to each instance.
The “dev/xvda” is the root device. As mentioned earlier, this device name is reserved for the root device.
The other “/dev/sdf” is the EBS created this time.

Checking the EBS Attachment Status 2: For Instances Not on Nitro System

Next, we will actually access the instances to check the EBS attach status.
First, we will access Instance1. We will use the SSM Session Manager to access the instance.

$ aws ssm start-session \
--target i-00f95cd8419e73461

Starting session with SessionId: root-0e49c3789cbebe2c9
sh-4.2$
Code language: Bash (bash)

You can use the lsblk command to check the status of block devices.

sh-4.2$ lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0   8G  0 disk 
└─xvda1 202:1    0   8G  0 part /
xvdf    202:80   0   1G  0 disk 
Code language: Bash (bash)

You can see that two block devices, “xvda” and “xvdf”, are recognized.
As we saw earlier, the former is the root device.
The latter is the EBS that we attached this time. The device name has been changed from the one specified in the CloudFormation template (/dev/sdf). This is due to the specification of EBS.

Depending on the block device driver of the kernel, the device could be attached with a different name than you specified. For example, if you specify a device name of /dev/sdh, your device could be renamed /dev/xvdh or /dev/hdh. In most cases, the trailing letter remains the same.
Device names on Linux instances

From the above, we found out that the device name specified when attaching EBS and the device name recognized by the instance may differ.

Checking the EBS Attachment Status 3: For instances on Nitro System

Next, access Instance2 to check the EBS attach status.

$ aws ssm start-session \
--target i-03f376aac4fcc98ab

Starting session with SessionId: root-07ebbd9797a52a2de

sh-4.2$
Code language: Bash (bash)

sh-4.2$ lsblk
NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1       259:0    0   8G  0 disk 
├─nvme0n1p1   259:1    0   8G  0 part /
└─nvme0n1p128 259:2    0  10M  0 part /boot/efi
nvme1n1       259:3    0   1G  0 disk
Code language: Bash (bash)

You can see that two block devices, “nvme0n1” and “nvme1n1”, are recognized. This is due to the specification of the instance on the Nitro System.

EBS volumes are exposed as NVMe block devices on instances built on the Nitro System. The device names are /dev/nvme0n1, /dev/nvme1n1, and so on. The device names that you specify in a block device mapping are renamed using NVMe device names (/dev/nvme[0-26]n1).
Amazon EBS and NVMe on Linux instances

The instance type specified for Instance2 is T4g, which is the type that runs on the Nitro System.

The following instances are built on the Nitro System:
Virtualized: A1, C5, C5a, C5ad, C5d, C5n, C6g, C6gd, C6gn, C6i, D3, D3en, DL1, G4, G4ad, G5, G5g, I3en, Im4gn, Inf1, Is4gen, M5, M5a, M5ad, M5d, M5dn, M5n, M5zn, M6a, M6g, M6gd, M6i, p3dn.24xlarge, P4, R5, R5a, R5ad, R5b, R5d, R5dn, R5n, R6g, R6gd, R6i, T3, T3a, T4g, high memory (u-*), VT1, X2gd, and z1d
Instance types

From the above, “nvme0n1” is the root device and “nvme1n1” is the attached EBS.

Steps to make the attached EBS ready for use

We will continue to check the subsequent steps on Instance2.

The steps after attaching are covered in the AWS official website, Make an Amazon EBS volume available for use on Linux, and can be summarized as follows

Create a file system
Create a mount point and mount the volume.

Let’s actually do it.
First, create a file system with mkfs command.

sh-4.2$ sudo mkfs -t xfs /dev/nvme1n1
meta-data=/dev/nvme1n1           isize=512    agcount=4, agsize=65536 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=0
data     =                       bsize=4096   blocks=262144, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
Code language: Bash (bash)

We specified XFS as the file system to be created.

Next, create a mount point with the mkdir command, and then mount the volume with the mount command.

sh-4.2$ sudo mkdir /data
sh-4.2$ sudo mount /dev/nvme1n1 /data
Code language: Bash (bash)

Once again, check the status with the lsblk command.

sh-4.2$ lsblk -f
NAME          FSTYPE LABEL UUID                                 MOUNTPOINT
nvme0n1                                                         
├─nvme0n1p1   xfs    /     9e2f3182-7352-4382-adbd-c67f6be433ed /
└─nvme0n1p128 vfat         2EE0-5AE9                            /boot/efi
nvme1n1       xfs          5c851eca-35be-4d6a-a4e8-e10c1efed050 /data
Code language: Bash (bash)

You can check the file system and mount point.

Write a file to try it out.

sh-4.2$ cd /data
sh-4.2$ sudo touch test.txt
sh-4.2$ ls
test.txt
Code language: Bash (bash)

We were able to access the mount point and install the files.

Thus, after attaching the EBS, we found that by creating and mounting the file system, we could use it just like normal storage.

How to automatically mount EBS on reboot

We will see how to automatically mount the EBS volume.
On the official website, you can find instructions on how to edit /etc/fstab. Open the file with an editor such as vim, and edit it as follows.

sh-4.2$ sudo vim /etc/fstab
sh-4.2$ cat /etc/fstab 
#
UUID=9e2f3182-7352-4382-adbd-c67f6be433ed     /           xfs    defaults,noatime  1   1
UUID=2EE0-5AE9        /boot/efi       vfat    defaults,noatime,uid=0,gid=0,umask=0077,shortname=winnt 0 0
UUID=5c851eca-35be-4d6a-a4e8-e10c1efed050     /data       xfs    defaults,nofail   0   2
Code language: Bash (bash)

The third line is the relevant part: specify the UUID confirmed by the lsblk command and write it so that it is automatically mounted at the mount point (/data).

To confirm the correctness of the configuration, you can follow the steps below.

Unmount the EBS volume
Mount all volumes according to /etc/fstab
Access the mount point.

If you get to the end without any errors, you can call it a success.

sh-4.2$ sudo umount /data
sh-4.2$ sudo mount -a
sh-4.2$ ls /data
test.txt
Code language: Bash (bash)

We were able to proceed to the end successfully.
Now the EBS volume will be mounted automatically after reboot.

SSM documentation to make EBS ready for use

Finally, we will show you how to use the SSM document AWS-RunShellScript to automatically configure EBS to be ready for use on a Linux instance (Amazon Linux 2).
Once again, the shell script to run is described below.

- !Sub |
  for d in $(lsblk -n -r -p | grep disk | awk '{ print $1 }' )
  do
    if  [ "$(sudo file -s -b $d)" == data ]; then
      sudo mkfs -t ${FileSystem} $d
      sudo mkdir ${MountPoint}
      sudo mount $d ${MountPoint}
      
      uuid=$(sudo blkid $d -o export | grep ^UUID)
      echo -e "$uuid\t/data\t${FileSystem}\tdefaults,nofail\t0\t2" >> /etc/fstab
    fi
  done
Code language: YAML (yaml)

The contents of the script are as described above, but in case you are interested, here is a brief explanation.

run the lsblk command to obtain the device name whose TYPE is disk.
use the file command to check the information of each device, and regard the one with “data” as an EBS and an unprocessed device, and proceed with the processing.
after creating the file system, mount the EBS.
obtain UUID and add it to /etc/fstab for automatic mount setting.

This script will be automatically executed when the CloudFormation stack is created.

Access instance 3 to verify that the script worked successfully.

% aws ssm start-session \
--target i-0bbda6e4d4e4bcdc3

Starting session with SessionId: root-03c8972378759f257
sh-4.2$ 

sh-4.2$ lsblk
NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1       259:0    0   8G  0 disk 
├─nvme0n1p1   259:1    0   8G  0 part /
└─nvme0n1p128 259:2    0  10M  0 part /boot/efi
nvme1n1       259:3    0   1G  0 disk /data
Code language: Bash (bash)

You can see that the file system has been created on the EBS and mounted successfully.

Check the settings related to automount.

sh-4.2$ cat /etc/fstab
#
UUID=b2d2af18-472e-4a07-a7f5-313592c907b9     /           xfs    defaults,noatime  1   1
UUID=7882-6D24        /boot/efi       vfat    defaults,noatime,uid=0,gid=0,umask=0077,shortname=winnt 0 0
UUID=a38658f8-ad51-417d-b7a7-4d865bd500b9	/data	xfs	defaults,nofail	0	2

sh-4.2$ sudo umount /data

sh-4.2$ lsblk
NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1       259:0    0   8G  0 disk 
├─nvme0n1p1   259:1    0   8G  0 part /
└─nvme0n1p128 259:2    0  10M  0 part /boot/efi
nvme1n1       259:3    0   1G  0 disk 

sh-4.2$ sudo mount -a

sh-4.2$ lsblk
NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1       259:0    0   8G  0 disk 
├─nvme0n1p1   259:1    0   8G  0 part /
└─nvme0n1p128 259:2    0  10M  0 part /boot/efi
nvme1n1       259:3    0   1G  0 disk /data
Code language: Bash (bash)

We have confirmed that the EBS information is written to /etc/fstab and mounted accordingly.

Summary

We have confirmed how to attach EBS to an EC2 instance using CloudFormation.
We confirmed the steps to make the attached EBS available for use and the steps to mount it automatically.
We have shown you how to use SSM documentation to automatically set your EBS to a usable state.