AWS_EN

Introduction to OpenSearch with CFN

スポンサーリンク
Introduction to OpenSearch with CloudFormation AWS_EN
スポンサーリンク
スポンサーリンク

Configuration Getting Started with OpenSearch using CloudFormation

OpenSearch is a search and analysis suite forked from Elasticsearch.
In this introduction, we will build a basic OpenSearch environment using CloudFormation.

Environment

Diagram of introduction to OpenSearch with CloudFormation.

Create an OpenSearch domain.
Replicate the configuration described in the official AWS tutorial.

Amazon OpenSearch Service

CloudFormation template files

Build the above configuration with CloudFormation.
The CloudFormation template is located at the following URL

https://github.com/awstut-an-r/awstut-fa/tree/main/042

Explanation of key points of template files

Template for OpenSearch domain

Resources:
  Domain:
    Type: AWS::OpenSearchService::Domain
    Properties:
      AccessPolicies:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              AWS: "*"
            Action: es:*
            Resource: !Sub "arn:aws:es:${AWS::Region}:${AWS::AccountId}:domain/${DomainName}/*"
      AdvancedSecurityOptions:
        Enabled: true
        InternalUserDatabaseEnabled: true
        MasterUserOptions:
          MasterUserName: !Ref MasterUserName
          MasterUserPassword: !Ref MasterUserPassword
      ClusterConfig:
        DedicatedMasterEnabled: false
        InstanceCount: !Ref InstanceCount
        InstanceType: !Ref InstanceType
        WarmEnabled: false
        ZoneAwarenessEnabled: false
      CognitoOptions:
        Enabled: false
      DomainEndpointOptions:
        CustomEndpointEnabled: false
        EnforceHTTPS: true
        TLSSecurityPolicy: Policy-Min-TLS-1-0-2019-07
      DomainName: !Ref DomainName
      EBSOptions:
        EBSEnabled: true
        VolumeSize: !Ref VolumeSize
        VolumeType: gp2
      EncryptionAtRestOptions:
        Enabled: true
        KmsKeyId: !Ref Key
      EngineVersion: !Ref EngineVersion
      NodeToNodeEncryptionOptions:
        Enabled: true
Code language: YAML (yaml)

Check the parameters.

The AccessPolicies property allows you to set restrictions on access to the OpenSearch domain.
Restrictions are set in the form of IAM policies.
In this case, we will create a master user and use the user information for authentication.
Therefore, this property will allow all operations on the resource to be created.

The AdvancedSecurityOptions property allows you to set fine-grained access control for security.
The tutorial shows how to create a master user in OpenSearch and use that user’s information for authentication.
This property is set in a similar manner.
Specifically, activate the Internal User Database and set the user name and password for the master user to be created.

The ClusterConfig property allows you to configure settings related to the specifications of the OpenSearch domain to be created.
You can set the type and number of instances that make up the OpenSearch infrastructure, and whether or not there is a dedicated master node.
Basically, configure as described in the tutorial, but this time the number of instances is set to one.

The DomainEndpointOptions property allows you to configure settings related to the OpenSearch domain to be created.
If you want to enable authentication, or fine-grained access control, using master user information, as in this case, HTTPS for the domain is a prerequisite.

Fine-grained access control requires OpenSearch or Elasticsearch 6.7 or later. It also requires HTTPS for all traffic to the domain, Encryption of data at rest, and node-to-node encryption.

Enabling fine-grained access control

Fine-grained access control requires OpenSearch or Elasticsearch 6.7 or later. It also requires HTTPS for all traffic to the domain, encryption of data in storage, and encryption between nodes.
https://docs.aws.amazon.com/ja_jp/opensearch-service/latest/developerguide/fgac.html#fgac-enabling
Enabling Granular Access Control

Configure the settings in a manner similar to the above requirements.
Specifically, enforce HTTPS communication with this property and enable encryption with the EncryptionAtRestOptions and NoDeToNodeEncryptionOptions properties.
Custom endpoints are not enabled according to the tutorial policy.

Set the name of the OpenSearch domain to be created in the DomainName property.

The EBSOptions property allows you to configure the storage settings used by the instance described above.
Follow the tutorial to set up 10GB of gp2 type EBS.

Set the version of OpenSearch to be created in the EngineVersion property.
This time, we set the version to “OpenSearch_1.2,” which is the latest version.

Set the KMS key used for encryption as follows.

Resources:
  Key:
    Type: AWS::KMS::Key
    Properties:
      Enabled: true
      KeyPolicy:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              AWS: "*"
            Action:
              - kms:Encrypt
              - kms:Decrypt
              - kms:ReEncrypt*
              - kms:GenerateDataKey*
              - kms:CreateGrant
              - kms:DescribeKey
            Resource: "*"
            Condition:
              StringEquals:
                kms:ViaService: !Sub "es.${AWS::Region}.amazonaws.com"
                kms:CallerAccount: !Ref AWS::AccountId
          - Effect: Allow
            Principal:
              AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
            Action: "*"
            Resource: "*"
          - Effect: Allow
            Principal:
              Service: es.amazonaws.com
            Action:
              - kms:Describe*
              - kms:Get*
              - kms:List*
            Resource: "*"
Code language: YAML (yaml)

Created with reference to the AWS managed key used by default for OpenSearch.

Architecting

Use CloudFormation to build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stack.
For information on how to create stacks and check each stack, please refer to the following page

The following is information on the main resources created in this case

  • OpenSearch domain name: fa-042
  • OpenSearch dashboard URL: https://search-fa-042-m2yc6j6kr63hqyhvqp6jwfim7a.ap-northeast-1.es.amazonaws.com/_dashboards
  • OpenSearch domain endpoint URL: https://search-fa-042-m2yc6j6kr63hqyhvqp6jwfim7a.ap-northeast-1.es.amazonaws.com
  • Master user name: test
  • Master user password: p@ssw0rd

Check OpenSearch from the AWS Management Console as well.

OpenSearch Details 1
OpenSearch Details 2
OpenSearch Details 3

It has been created as described in the CloudFormation template file.

Confirmation of operation

Follow the tutorial.

First, upload the sample data.

Amazon OpenSearch Service

Using the data from the tutorial as is, upload the data using the curl command.
The following command is used for the configuration we have built this time.

$ curl -XPUT -u 'test:P@ssw0rd' 'https://search-fa-042-m2yc6j6kr63hqyhvqp6jwfim7a.ap-northeast-1.es.amazonaws.com/fa-042/_doc/1' -d '{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi"], "year": 1996, "actor": ["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}' -H 'Content-Type: application/json'
{"_index":"fa-042","_type":"_doc","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
Code language: Bash (bash)

Uploaded successfully.

Next is the data search.

Amazon OpenSearch Service

Following the tutorial, execute the search using the curl command.
The following command is applied to the configuration we have built this time.

$ curl -XGET -u 'test:P@ssw0rd' 'https://search-fa-042-m2yc6j6kr63hqyhvqp6jwfim7a.ap-northeast-1.es.amazonaws.com/fa-042/_search?q=mars&pretty=true'
{
  "took" : 45,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 0.2876821,
    "hits" : [
      {
        "_index" : "fa-042",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 0.2876821,
        "_source" : {
          "director" : "Burton, Tim",
          "genre" : [
            "Comedy",
            "Sci-Fi"
          ],
          "year" : 1996,
          "actor" : [
            "Jack Nicholson",
            "Pierce Brosnan",
            "Sarah Jessica Parker"
          ],
          "title" : "Mars Attacks!"
        }
      }
    ]
  }
}
Code language: Bash (bash)

The search was successfully executed.
A search for the word “mars” yielded the string “Mars Attacks!” with a _score of “0.2876821”.

Summary

As an introduction to OpenSearch, we built a basic environment using CloudFormation.

タイトルとURLをコピーしました