DVA_EN

Introduction to ElastiCache with CFN – Redis Cluster Enabled

スポンサーリンク
Introduction to ElastiCache with CFN - Redis Cluster Enabled DVA_EN
スポンサーリンク
スポンサーリンク

Introduction to ElastiCache with CFN – Redis Cluster Mode Enabled

The following page covers Redis (cluster mode disabled version) as an introduction to ElastiCache.

In this page, we will create ElastiCache for Redis (cluster mode enabled).

Environment

Detail of Introduction to ElastiCache with CFN - Redis Cluster Enabled

The basic configuration is the same as in the page introduced at the beginning of this document.

The difference is ElastiCache.
This time, we will build ElastiCache as a cluster mode enabled version of Redis.

CloudFormation template files

The above configuration is built using CloudFormation.
The CloudFormation templates are located at the following URL

awstut-dva/03/004 at main · awstut-an-r/awstut-dva
Contribute to awstut-an-r/awstut-dva development by creating an account on GitHub.

Explanation of key points of the template files

This page focuses on the key points of creating a cluster mode enabled version of ElastiCache for Redis.

Please refer to the following page for the basic matters of ElastiCache for Redis.

ElastiCache for Redis (cluster mode enable)

Resources: ReplicationGroup: Type: AWS::ElastiCache::ReplicationGroup Properties: AutomaticFailoverEnabled: true AutoMinorVersionUpgrade: true CacheNodeType: !Ref CacheNodeType CacheParameterGroupName: !Ref CacheParameterGroupName CacheSubnetGroupName: !Ref SubnetGroup Engine: !Ref CacheEngine EngineVersion: !Ref CacheEngineVersion MultiAZEnabled: true NodeGroupConfiguration: - PrimaryAvailabilityZone: !Sub "${AWS::Region}${AvailabilityZone1}" ReplicaAvailabilityZones: - !Sub "${AWS::Region}${AvailabilityZone2}" ReplicaCount: !Ref ReplicaCount NumNodeGroups: !Ref NumNodeGroups Port: !Ref RedisPort ReplicationGroupDescription: !Sub "${Prefix}-ReplicationGroup" ReplicationGroupId: !Sub "${Prefix}-replicationgroup" SecurityGroupIds: - !Ref CacheSecurityGroup
Code language: YAML (yaml)

Create points in creating a cluster-enabled version of ElastiCache for Redis.

The first is the AutomaticFailoverEnabled property.
This property must be enabled when building a cluster mode enabled version of Redis with ElastiCache.

AutomaticFailoverEnabled must be enabled for Redis (cluster mode enabled) replication groups.

AWS::ElastiCache::ReplicationGroup

The second is the CacheParameterGroupName property.
In this case, the latest version (6.2) of Redis is selected, but the value set for this property depends on whether cluster mode is enabled or disabled.

  • Cluster mode enabled: default.redis6.x.cluster.on
  • Cluster mode disabled: default.redis6.x

According to the above, the former is specified in this case.

The third is the NumNodeGroups property.
It sets the number of nodes (instances) to be included in one shard.
Two instances are created, but they are included in one shard.
Therefore, the value of this property is “1”.

(Reference) Lambda function

Resources: Function: Type: AWS::Lambda::Function Properties: Architectures: - !Ref Architecture Environment: Variables: CONFIGURATION_ENDPOINT_ADDRESS: !Ref ConfigurationEndPointAddress REDIS_PORT: !Ref RedisPort Code: ZipFile: | import json import os import redis configuration_endpoint_address = os.environ['CONFIGURATION_ENDPOINT_ADDRESS'] redis_port = int(os.environ['REDIS_PORT']) def lambda_handler(event, context): pool = redis.ConnectionPool( host=configuration_endpoint_address, port=redis_port ) client = redis.StrictRedis(connection_pool=pool) client.set('hogehoge', 'fugafuga') return { 'statusCode': 200, 'body': json.dumps( { 'redis_result': client.get('hogehoge').decode() }, indent=2 ) } FunctionName: !Sub "${Prefix}-function" Handler: !Ref Handler Layers: - !Ref LambdaLayer Runtime: !Ref Runtime Role: !GetAtt FunctionRole.Arn VpcConfig: SecurityGroupIds: - !Ref FunctionSecurityGroup SubnetIds: - !Ref LambdaSubnet
Code language: YAML (yaml)

The contents are to be retrieved after connecting to Redis from Python and storing the values for testing.

Architecting

Using CloudFormation, build this environment and check the actual behavior.

Create CloudFormation stacks and check resources in stacks

Create a CloudFormation stack.
For information on how to create stacks and check each stack, please refer to the following page

After checking the resources in each stack, information on the main resources created this time is as follows

  • ElastiCache: dva-03-004-replicationgroup
  • ElastiCache configuration endpoint: dva-03-004-replicationgroup.ryxnym.clustercfg.apne1.cache.amazonaws.com
  • EC2 instance: i-0e58df81a218038e1
  • Function URL for the Lambda function: https://m2mcg2dyjisl7ycp5ar7ttdzwu0hchwd.lambda-url.ap-northeast-1.on.aws/

Check each resource from the AWS Management Console.
First, check ElastiCache itself.

Detail of Redis 1.

ElastiCache has been successfully created.
We can see that the engine is Redis and cluster mode is enabled.
One shard has been created in the cluster, and two nodes (instances) have been created inside the shard.

You can see that a configuration endpoint has been created in the cluster.
The disabled version of cluster mode creates three types of endpoints: primary endpoints, read endpoints, and node endpoints, while the enabled version is different.

Redis (cluster mode enabled) clusters with replicas, because they have multiple shards (API/CLI: node groups), which mean they also have multiple primary nodes, have a different endpoint structure than Redis (cluster mode disabled) clusters. Redis (cluster mode enabled) has a configuration endpoint which “knows” all the primary and node endpoints in the cluster. Your application connects to the configuration endpoint. Whenever your application writes to or reads from the cluster’s configuration endpoint, Redis, behind the scenes, determines which shard the key belongs to and which endpoint in that shard to use. It is all quite transparent to your application.

Finding replication group endpoints
Detail of Redis 2.
Detail of Redis 3.

You can see that there are two nodes in one shard.
Indeed, we can see that endpoints have also been created for each node.

Check Action

Now that everything is ready, let’s check the Operation.

Connecting to ElastiCache for Redis from EC2 instance

After accessing the EC2 instance, connect to ElastiCache.
Use SSM Session Manager to access the instance.

% aws ssm start-session --target i-0e58df81a218038e1 sh-4.2$
Code language: Bash (bash)

For more information on SSM Session Manager, please see the following page

Check the execution status of the initialization process of the EC2 instance by user data.

Connect to ElastiCache using redis-cli.
First, specify the primary endpoint and connect to the primary node.

sh-4.2$ redis-cli -h dva-03-004-replicationgroup.ryxnym.clustercfg.apne1.cache.amazonaws.com -p 6379 dva-03-004-replicationgroup.ryxnym.clustercfg.apne1.cache.amazonaws.com:6379>
Code language: Bash (bash)

Successfully connected.

Save the values for testing.

dva-03-004-replicationgroup.ryxnym.clustercfg.apne1.cache.amazonaws.com:6379> set hoge foo OK
Code language: Bash (bash)

Saved successfully.

Retrieve the saved value.

dva-03-004-replicationgroup.ryxnym.clustercfg.apne1.cache.amazonaws.com:6379> get hoge "foo"
Code language: Bash (bash)

For reference, check the IP address of the configuration endpoint.

sh-4.2$ nslookup dva-03-004-replicationgroup.ryxnym.clustercfg.apne1.cache.amazonaws.com Server: 10.0.0.2 Address: 10.0.0.2#53 Non-authoritative answer: Name: dva-03-004-replicationgroup.ryxnym.clustercfg.apne1.cache.amazonaws.com Address: 10.0.4.114 Name: dva-03-004-replicationgroup.ryxnym.clustercfg.apne1.cache.amazonaws.com Address: 10.0.3.146
Code language: Bash (bash)

Two addresses were returned.

Continue with the name resolution of the node endpoints.

sh-4.2$ nslookup dva-03-004-replicationgroup-0001-001.ryxnym.0001.apne1.cache.amazonaws.com Server: 10.0.0.2 Address: 10.0.0.2#53 Non-authoritative answer: Name: dva-03-004-replicationgroup-0001-001.ryxnym.0001.apne1.cache.amazonaws.com Address: 10.0.3.146
Code language: Bash (bash)
sh-4.2$ nslookup dva-03-004-replicationgroup-0001-002.ryxnym.0001.apne1.cache.amazonaws.com Server: 10.0.0.2 Address: 10.0.0.2#53 Non-authoritative answer: Name: dva-03-004-replicationgroup-0001-002.ryxnym.0001.apne1.cache.amazonaws.com Address: 10.0.4.114
Code language: Bash (bash)

We can see that the addresses associated with the configuration endpoints are for two nodes.

For further reference, here is the behavior when connecting to each node endpoint

sh-4.2$ redis-cli -h dva-03-004-replicationgroup-0001-001.ryxnym.0001.apne1.cache.amazonaws.com -p 6379 dva-03-004-replicationgroup-0001-001.ryxnym.0001.apne1.cache.amazonaws.com:6379> get hoge "foo"
Code language: Bash (bash)
sh-4.2$ redis-cli -h dva-03-004-replicationgroup-0001-002.ryxnym.0001.apne1.cache.amazonaws.com -p 6379 dva-03-004-replicationgroup-0001-002.ryxnym.0001.apne1.cache.amazonaws.com:6379> get hoge (error) MOVED 1525 10.0.3.146:6379
Code language: Bash (bash)

I tried to retrieve the value I just stored on two nodes, but failed on one of them.
This was due to the fact that the node being accessed and the node actually storing the data were different.

Connecting to ElastiCache for Redis from Lambda function

Connect to ElastiCache from a Lambda function.

Access the Function URL of this function.
For more information about Function URL, please refer to the following page

Detail of Redis 4.

The value stored in the function is successfully retrieved.
In this way, the Lambda function can also connect to ElastiCache for Redis with cluster mode enabled.

Summary

We have confirmed how to create ElastiCache for Redis (cluster mode enabled).
We have confirmed that the created ElastiCache can be connected to from EC2 instances and Lambda functions.

タイトルとURLをコピーしました