AWS_EN

Introduction to AWS IoT Core using CloudFormation

Introduction to AWS IoT Core using CloudFormation

This page deals with AWS IoT Core.

AWS IoT Core services connect IoT devices to AWS IoT services and other AWS services. AWS IoT Core includes the device gateway and the message broker, which connect and process messages between your IoT devices and the cloud.

Getting started with AWS IoT Core

Since this is an introduction to AWS IoT Core, we will proceed with the following official tutorial.

Create a virtual device with Amazon EC2 - AWS IoT Core
In this tutorial, you'll create an Amazon EC2 instance to serve as your virtual device in the cloud.
Use your Windows or Linux PC or Mac as an AWS IoT device - AWS IoT Core
In this tutorial, you'll configure a personal computer for use with AWS IoT. These instructions support Windows and Linux PCs and Macs. To accomplish this, you ...

Specifically, we will see how to use an EC2 instance as an IoT device to send MQTT messages to AWS IoT Core.

AWS IoT Core is one of the most central services among the various IoT-related services offered by AWS, and a better understanding of AWS IoT Core will help you build IoT applications smoothly.

Environment

Diagram of Introduction to AWS IoT Core using CloudFormation.

Create a public subnet within the VPC.
Place an EC2 instance in the same subnet.
Look at this instance as an IoT device (Thing).
This instance is Amazon Linux 2023.

Connect to AWS IoT Core from this device and send MQTT messages.
Use the AWS IoT Device SDK for Python to connect to AWS IoT Core and send messages.

Scenarios

This page confirms the following

  • Thing objects
  • AWS IoT Policy
  • Client Certificate
  • Connect to AWS IoT Core using AWS IoT Device SDK

CloudFormation template files

The above configuration is built with CloudFormation.
The CloudFormation template is placed at the following URL

awstut-fa/151 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Explanation of key points of template files

AWS IoT Thing means IoT Device

An object of things means a device to be managed by AWS IoT.

Devices connected to AWS IoT Core are represented by thing objects in the AWS IoT registry. A thing object represents a specific device or logical entity. It can be a physical device or sensor (for example, a light bulb or a light switch on the wall).

Create a thing object
Resources:
  Thing:
    Type: AWS::IoT::Thing
    Properties:
      ThingName: !Ref Prefix
Code language: YAML (yaml)

In this case, we will consider an EC2 instance as an IoT device, so the object of this thing will mean an instance.

This time we will simply set only the object name of the thing.
The name is “fa-151”.

AWS IoT policies control the permissions of things

AWS IoT policies are tied to the certificates used by the device and define the actions that the device can perform for AWS IoT.

Devices use an X.509 certificate to authenticate with AWS IoT Core. The certificate has AWS IoT policies attached to it. These policies determine which AWS IoT operations, such as subscribing or publishing to MQTT topics, the device is permitted to perform. Your device presents its certificate when it connects and sends messages to AWS IoT Core.

Create an AWS IoT policy
Resources:
  Policy:
    Type: AWS::IoT::Policy
    Properties:
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action:
              - iot:Publish
              - iot:Receive
            Resource:
              - !Sub "arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topic/${TopicName}"
          - Effect: Allow
            Action:
              - iot:Subscribe
            Resource:
              - !Sub "arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topicfilter/${TopicName}"
          - Effect: Allow
            Action:
              - iot:Connect
            Resource:
              - !Sub "arn:aws:iot:${AWS::Region}:${AWS::AccountId}:client/${Thing}"
      PolicyName: !Ref Prefix
Code language: YAML (yaml)

As you can see, AWS IoT policies are similar to IAM policies.

In this case, since this is a verification, we created it based on what is described in the following official tutorial.

Use your Windows or Linux PC or Mac as an AWS IoT device - AWS IoT Core
In this tutorial, you'll configure a personal computer for use with AWS IoT. These instructions support Windows and Linux PCs and Macs. To accomplish this, you ...

The name of this policy is “fa-151”.

Set up an EC2 instance as an IoT device

As mentioned earlier, the role of the EC2 instance in this case is an IoT device.
User data is used to configure the various settings required on the IoT device side.

For basic information on user data, please refer to the following pages.

Resources:
  Instance:
    Type: AWS::EC2::Instance
    Properties:
      IamInstanceProfile: !Ref InstanceProfile
      ImageId: !Ref ImageId
      InstanceType: !Ref InstanceType
      NetworkInterfaces:
        - AssociatePublicIpAddress: true
          DeviceIndex: 0
          GroupSet:
            - !Ref InstanceSecurityGroup
          SubnetId: !Ref PublicSubnet1
      UserData: !Base64
        Fn::Sub: |
          #!/bin/bash -xe
          dnf update -y
          dnf install python3.11-pip -y
          dnf install -y git
          
          mkdir ~/certs
          
          curl -o ~/certs/Amazon-root-CA-1.pem \
            https://www.amazontrust.com/repository/AmazonRootCA1.pem
          
          aws iot create-keys-and-certificate \
            --set-as-active \
            --certificate-pem-outfile "~/certs/device.pem.crt" \
            --public-key-outfile "~/certs/public.pem.key" \
            --private-key-outfile "~/certs/private.pem.key" > ~/output.txt
          
          certificate_arn=`cat ~/output.txt | jq -r .certificateArn`
          
          aws iot attach-thing-principal \
            --thing-name "${Thing}" \
            --principal "$certificate_arn"
  
          aws iot attach-policy \
            --policy-name "${Policy}" \
            --target "$certificate_arn"
          
          aws iot describe-endpoint \
            --endpoint-type iot:Data-ATS > ~/endpoint.txt
          
          endpoint=`cat ~/endpoint.txt| jq -r .endpointAddress`
          
          python3.11 -m pip install awsiotsdk
          cd ~ && git clone https://github.com/aws/aws-iot-device-sdk-python-v2.git

          cd ~/aws-iot-device-sdk-python-v2/samples && python3.11 pubsub.py \
            --endpoint "$endpoint" \
            --ca_file ~/certs/Amazon-root-CA-1.pem \
            --cert ~/certs/device.pem.crt \
            --key ~/certs/private.pem.key \
            --client_id "${Thing}" \
            --topic "${TopicName}" \
            --count 5
Code language: YAML (yaml)

We will focus on the UserData property.

First, after updating dnf, use dnf to install Python 3.11 and Git.
These are used to get and run the AWS IoT Device SDK for Python.

Then use the curl command to obtain the Amazon CA certificate (AmazonRootCA1.pem).
This certificate is used for server proof to verify that the destination is correct when sending MQTT messages from the IoT device to AWS IoT Core.

Create the certificate with the aws iot create-keys-and-certificate command.
Generate the client certificate (device.pem.crt) that the thing will use to communicate with the AWS IoT Core.
The certificate is generated on the EC2 instance and also registered on the AWS IoT side.

The command will also retrieve the public key (public.pem.key) and private key (private.pem.key) used to create the certificate.
The private key in particular is used to send MQTT messages to AWS IoT Core.

Save the result of this command execution to a text file (output.txt).
This is to retrieve the ARN assigned to the certificate when it is registered with AWS IoT.
Use the jq command to retrieve this and store it in a variable.
By the way, the execution result of this command is returned in the following format.

{
    "certificateArn": "arn:aws:iot:[region]:[account-id]:cert/[cert-id]",
    "certificateId": "[cert-id]",
    "certificatePem": "-----BEGIN CERTIFICATE-----\n[cert-pem]\n-----END CERTIFICATE-----\n",
    "keyPair": {
        "PublicKey": "-----BEGIN PUBLIC KEY-----\n[public-key]\n-----END PUBLIC KEY-----\n",
        "PrivateKey": "-----BEGIN RSA PRIVATE KEY-----\n[private-key]\n-----END RSA PRIVATE KEY-----\n"
    }
}
Code language: JSON / JSON with Comments (json)

Execute the aws iot attach-thing-principal command to associate the thing with the certificate.
Specify the resource name of the aforementioned thing and the ARN of the certificate, respectively.

Execute the aws iot attach-policy command to associate the AWS IoT policy with the certificate.
Note that the object to be associated with the AWS IoT policy is the certificate, not the thing.
Specify the aforementioned policy name and the ARN of the certificate.

Run the aws iot describe-endpoint command to get the endpoints to which MQTT messages are sent.
There are two types of endpoints as follows.

Every customer has an iot:Data-ATS and an iot:Data endpoint. Each endpoint uses an X.509 certificate to authenticate the client. We strongly recommend that customers use the newer iot:Data-ATS endpoint type to avoid issues related to the widespread distrust of Symantec certificate authorities.

Connecting devices to AWS IoT

Specify “iot:Data-ATS” for the endpoint-type option according to the above.

Save the result of this command execution to a text file (endpoint.txt) and store the endpoint name in a variable.
Again, use the jq command to extract the endpoint names from the following JSON-formatted data.

{
    "endpointAddress": "[endpoint-id]-ats.iot.[region].amazonaws.com"
}
Code language: JSON / JSON with Comments (json)

Install the AWS IoT Device SDK for Python (awsiotsdk) using pip.
And use Git to also get the sample script for this SDK.

Finally, run the sample script pubsub.py.
This sample script sends a test message (Hello World!) to AWS IoT Core.

To run this script, we pass the endpoints we have obtained so far, the Amazon CA certificate, the client certificate, and the private key for the client certificate.

In addition, two parameters are specified.

The first is the client ID.
This is explained by the AWS official as follows

A typical device use case involves the use of the thing name as the default MQTT client ID. Although we don’t enforce a mapping between a thing’s registry name and its use of MQTT client IDs, certificates, or shadow state, we recommend you choose a thing name and use it as the MQTT client ID for both the registry and the Device Shadow service.

Managing devices with AWS IoT

Specify the name of the thing for this parameter according to the above.

The second is the topic name.
This is explained by the AWS official as follows

MQTT topics identify AWS IoT messages. AWS IoT clients identify the messages they publish by giving the messages topic names. Clients identify the messages to which they want to subscribe (receive) by registering a topic filter with AWS IoT Core. The message broker uses topic names and topic filters to route messages from publishing clients to subscribing clients.

MQTT topics

For this one, specify “test/topic”.

IAM roles for EC2 instance

Resources:
  InstanceRole:
    Type: AWS::IAM::Role
    DeletionPolicy: Delete
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: sts:AssumeRole
            Principal:
              Service:
                - ec2.amazonaws.com
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
      Policies:
        - PolicyName: InstancePolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - iot:AttachPolicy
                  - iot:AttachThingPrincipal
                  - iot:CreateKeysAndCertificate
                  - iot:DescribeEndpoint
                Resource:
                  - "*"
Code language: YAML (yaml)

As we have just confirmed, this EC2 instance is intended to operate as an IoT device.
To this end, we have described the necessary processing in the definition of the initialization process with user data.
This IAM role grants the instance the necessary permissions to perform those processes.

Architecting

Use CloudFormation to build this environment and check its actual behavior.

Create CloudFormation stacks and check resources

Create CloudFormation stacks.
For information on how to create stacks and check each stack, please see the following page.

Check the various resources of AWS IoT.

Check AWS IoT Thing.

Detail of AWS IoT 01.

Certainly things are created.

Check the AWS IoT policy.

Detail of AWS IoT 02.

The necessary permissions are granted to run the sample script.
The topic name is “test/topic” and the client ID is “fa-151”.
These are the values that the EC2 instance is supposed to specify when executing the sample script.

Check the certificate.

Detail of AWS IoT 03.
Detail of AWS IoT 04.

The certificate has indeed been created.
This means that the aws iot create-keys-and-certificate command was successfully executed on the EC2 instance.

Also, if we look at the resources associated with this certificate, we can see the policy and thing we just checked.
This means that the aws iot attach-thing-principal and aws iot attach-policy commands were successfully executed on the same instance.

Check the endpoint name.

Detail of AWS IoT 05.

This is the AWS IoT endpoint assigned to my account.
The EC2 instance will send MQTT messages here.

Operation Check

Now that we are ready, we access the EC2 instance (i-045ca338a33b06229).
The SSM Session Manager is used to access the instance.

% aws ssm start-session --target i-045ca338a33b06229
...
sh-5.2$
Code language: Bash (bash)

For more information on SSM Session Manager, please refer to the following page.

Check the execution status of processes defined in user data.

Use the tree command to check the contents of the /root directory.

sh-5.2$ sudo tree /root
/root
├── aws-iot-device-sdk-python-v2
│    ├── ...
│    ├── samples
│   │    │   ├── ...
│   │    │   ├── pubsub.py
│   │    │   └── ...
│    └── ...
├── certs
│    ├── Amazon-root-CA-1.pem
│    ├── device.pem.crt
│    ├── private.pem.key
│    └── public.pem.key
├── endpoint.txt
└── output.txt
Code language: Bash (bash)

Indeed, the process defined in the user data is successfully executed and various files are located.

Run the journalctl command to check the instance logs.

sh-5.2$ sudo journalctl --no-pager
...
Feb 24 06:15:19 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: + python3.11 pubsub.py --endpoint a2oxckhng7gmur-ats.iot.ap-northeast-1.amazonaws.com --ca_file /root/certs/Amazon-root-CA-1.pem --cert /root/certs/device.pem.crt --key /root/certs/private.pem.key --client_id fa-151 --topic test/topic --count 5
...
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Connecting to a2oxckhng7gmur-ats.iot.ap-northeast-1.amazonaws.com with client ID 'fa-151'...
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Connected!
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Subscribing to topic 'test/topic'...
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Connection Successful with return code: 0 session present: False
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Subscribed with 1
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Sending 5 message(s)
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Publishing message to topic 'test/topic': Hello World!  [1]
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Received message from topic 'test/topic': b'"Hello World!  [1]"'
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Publishing message to topic 'test/topic': Hello World!  [2]
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Received message from topic 'test/topic': b'"Hello World!  [2]"'
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Publishing message to topic 'test/topic': Hello World!  [3]
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Received message from topic 'test/topic': b'"Hello World!  [3]"'
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Publishing message to topic 'test/topic': Hello World!  [4]
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Received message from topic 'test/topic': b'"Hello World!  [4]"'
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Publishing message to topic 'test/topic': Hello World!  [5]
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Received message from topic 'test/topic': b'"Hello World!  [5]"'
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: 5 message(s) received.
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Disconnecting...
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Connection closed
Feb 24 06:15:25 ip-10-0-1-45.ap-northeast-1.compute.internal cloud-init[1672]: Disconnected!
...
Code language: Bash (bash)

We have extracted the part of the script that sends MQTT messages to the AWS IoT endpoint.
We can see that the sample script is executed to connect to AWS IoT and send 5 messages.

Finally, check the messages sent from the EC2 instance on the AWS IoT side.
To confirm, use the AWS IoT MQTT client in the AWS Management Console.

Detail of AWS IoT 06.

In the Subscribe to a topic tab, enter the topic name in the Topic filter and press Subscribe.
Since the EC2 instance sends messages to “test/topic”, specify this.

Detail of AWS IoT 07.

Message displayed.
It was indeed sent by the EC2 instance just now.

Detach the certificates before deleting the objects and AWS IoT policies!

After confirmation, you will probably want to delete the CloudFormation stacks, but there is a caveat.
That is, you need to unattach the Things and AW IoT policy resources from the certificate before deleting them.

Detail of AWS IoT 08.
Detail of AWS IoT 09.

Press “Detach” on both pages.

Please note that if you try to delete both resources before this is done, the deletion will fail.

Summary

We have identified how to use an EC2 instance as an IoT device to send MQTT messages to AWS IoT Core.

AWS IoT Core is one of the most central services among the various IoT-related services offered by AWS.
Better understanding of AWS IoT Core will help you build IoT apps smoothly.

タイトルとURLをコピーしました