AWS_EN

Introduction to OpenSearch with CFN

スポンサーリンク
Introduction to OpenSearch with CloudFormation AWS_EN
スポンサーリンク
スポンサーリンク

Configuration Getting Started with OpenSearch using CloudFormation

OpenSearch is a search and analysis suite forked from Elasticsearch.
In this introduction, we will build a basic OpenSearch environment using CloudFormation.

Environment

Diagram of introduction to OpenSearch with CloudFormation.

Create an OpenSearch domain.
Replicate the configuration described in the official AWS tutorial.

Step 1: Create an Amazon OpenSearch Service domain - Amazon OpenSearch Service
An OpenSearch Service domain is synonymous with an OpenSearch cluster. Domains are clusters with the settings, instance types, instance counts, and storage reso...

CloudFormation template files

Build the above configuration with CloudFormation.
The CloudFormation template is located at the following URL

awstut-fa/042 at main · awstut-an-r/awstut-fa
Contribute to awstut-an-r/awstut-fa development by creating an account on GitHub.

Explanation of key points of template files

Template for OpenSearch domain

Resources: Domain: Type: AWS::OpenSearchService::Domain Properties: AccessPolicies: Version: 2012-10-17 Statement: - Effect: Allow Principal: AWS: "*" Action: es:* Resource: !Sub "arn:aws:es:${AWS::Region}:${AWS::AccountId}:domain/${DomainName}/*" AdvancedSecurityOptions: Enabled: true InternalUserDatabaseEnabled: true MasterUserOptions: MasterUserName: !Ref MasterUserName MasterUserPassword: !Ref MasterUserPassword ClusterConfig: DedicatedMasterEnabled: false InstanceCount: !Ref InstanceCount InstanceType: !Ref InstanceType WarmEnabled: false ZoneAwarenessEnabled: false CognitoOptions: Enabled: false DomainEndpointOptions: CustomEndpointEnabled: false EnforceHTTPS: true TLSSecurityPolicy: Policy-Min-TLS-1-0-2019-07 DomainName: !Ref DomainName EBSOptions: EBSEnabled: true VolumeSize: !Ref VolumeSize VolumeType: gp2 EncryptionAtRestOptions: Enabled: true KmsKeyId: !Ref Key EngineVersion: !Ref EngineVersion NodeToNodeEncryptionOptions: Enabled: true
Code language: YAML (yaml)

Check the parameters.

The AccessPolicies property allows you to set restrictions on access to the OpenSearch domain.
Restrictions are set in the form of IAM policies.
In this case, we will create a master user and use the user information for authentication.
Therefore, this property will allow all operations on the resource to be created.

The AdvancedSecurityOptions property allows you to set fine-grained access control for security.
The tutorial shows how to create a master user in OpenSearch and use that user’s information for authentication.
This property is set in a similar manner.
Specifically, activate the Internal User Database and set the user name and password for the master user to be created.

The ClusterConfig property allows you to configure settings related to the specifications of the OpenSearch domain to be created.
You can set the type and number of instances that make up the OpenSearch infrastructure, and whether or not there is a dedicated master node.
Basically, configure as described in the tutorial, but this time the number of instances is set to one.

The DomainEndpointOptions property allows you to configure settings related to the OpenSearch domain to be created.
If you want to enable authentication, or fine-grained access control, using master user information, as in this case, HTTPS for the domain is a prerequisite.

Fine-grained access control requires OpenSearch or Elasticsearch 6.7 or later. It also requires HTTPS for all traffic to the domain, Encryption of data at rest, and node-to-node encryption.

Enabling fine-grained access control

Fine-grained access control requires OpenSearch or Elasticsearch 6.7 or later. It also requires HTTPS for all traffic to the domain, encryption of data in storage, and encryption between nodes.
https://docs.aws.amazon.com/ja_jp/opensearch-service/latest/developerguide/fgac.html#fgac-enabling
Enabling Granular Access Control

Configure the settings in a manner similar to the above requirements.
Specifically, enforce HTTPS communication with this property and enable encryption with the EncryptionAtRestOptions and NoDeToNodeEncryptionOptions properties.
Custom endpoints are not enabled according to the tutorial policy.

Set the name of the OpenSearch domain to be created in the DomainName property.

The EBSOptions property allows you to configure the storage settings used by the instance described above.
Follow the tutorial to set up 10GB of gp2 type EBS.

Set the version of OpenSearch to be created in the EngineVersion property.
This time, we set the version to “OpenSearch_1.2,” which is the latest version.

Set the KMS key used for encryption as follows.

Resources: Key: Type: AWS::KMS::Key Properties: Enabled: true KeyPolicy: Version: 2012-10-17 Statement: - Effect: Allow Principal: AWS: "*" Action: - kms:Encrypt - kms:Decrypt - kms:ReEncrypt* - kms:GenerateDataKey* - kms:CreateGrant - kms:DescribeKey Resource: "*" Condition: StringEquals: kms:ViaService: !Sub "es.${AWS::Region}.amazonaws.com" kms:CallerAccount: !Ref AWS::AccountId - Effect: Allow Principal: AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" Action: "*" Resource: "*" - Effect: Allow Principal: Service: es.amazonaws.com Action: - kms:Describe* - kms:Get* - kms:List* Resource: "*"
Code language: YAML (yaml)

Created with reference to the AWS managed key used by default for OpenSearch.

Architecting

Use CloudFormation to build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stack.
For information on how to create stacks and check each stack, please refer to the following page

The following is information on the main resources created in this case

  • OpenSearch domain name: fa-042
  • OpenSearch dashboard URL: https://search-fa-042-m2yc6j6kr63hqyhvqp6jwfim7a.ap-northeast-1.es.amazonaws.com/_dashboards
  • OpenSearch domain endpoint URL: https://search-fa-042-m2yc6j6kr63hqyhvqp6jwfim7a.ap-northeast-1.es.amazonaws.com
  • Master user name: test
  • Master user password: p@ssw0rd

Check OpenSearch from the AWS Management Console as well.

OpenSearch Details 1
OpenSearch Details 2
OpenSearch Details 3

It has been created as described in the CloudFormation template file.

Confirmation of operation

Follow the tutorial.

First, upload the sample data.

ステップ 2: インデックス作成のために Amazon OpenSearch Service にデータをアップロードする - Amazon OpenSearch Service
コマンドラインまたはほとんどのプログラミング言語を使用して、OpenSearch Service ドメインにデータをアップロードできます。

Using the data from the tutorial as is, upload the data using the curl command.
The following command is used for the configuration we have built this time.

$ curl -XPUT -u 'test:P@ssw0rd' 'https://search-fa-042-m2yc6j6kr63hqyhvqp6jwfim7a.ap-northeast-1.es.amazonaws.com/fa-042/_doc/1' -d '{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi"], "year": 1996, "actor": ["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}' -H 'Content-Type: application/json' {"_index":"fa-042","_type":"_doc","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
Code language: Bash (bash)

Uploaded successfully.

Next is the data search.

ステップ 3: Amazon OpenSearch Service でドキュメントを検索する - Amazon OpenSearch Service
Amazon OpenSearch Service ドメインでドキュメントを検索するには、OpenSearch 検索 API を使用します。または、 OpenSearch Dashboards を使用してドメインのドキュメントを検索することもできます。

Following the tutorial, execute the search using the curl command.
The following command is applied to the configuration we have built this time.

$ curl -XGET -u 'test:P@ssw0rd' 'https://search-fa-042-m2yc6j6kr63hqyhvqp6jwfim7a.ap-northeast-1.es.amazonaws.com/fa-042/_search?q=mars&pretty=true' { "took" : 45, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 1, "relation" : "eq" }, "max_score" : 0.2876821, "hits" : [ { "_index" : "fa-042", "_type" : "_doc", "_id" : "1", "_score" : 0.2876821, "_source" : { "director" : "Burton, Tim", "genre" : [ "Comedy", "Sci-Fi" ], "year" : 1996, "actor" : [ "Jack Nicholson", "Pierce Brosnan", "Sarah Jessica Parker" ], "title" : "Mars Attacks!" } } ] } }
Code language: Bash (bash)

The search was successfully executed.
A search for the word “mars” yielded the string “Mars Attacks!” with a _score of “0.2876821”.

Summary

As an introduction to OpenSearch, we built a basic environment using CloudFormation.

タイトルとURLをコピーしました