Loop in Step Functions until the condition is satisfied

TOC

Loop in Step Functions until the condition is satisfied

The following pages cover conditional branching (Choice) in Step Functions.

あわせて読みたい
Step Functions – Conditional branching using Choice State 【Step Functions - Conditional branching using Choice State】 This section is about refactoring, which is the scope of the AWS DBA. The following page covers...

This time, we will use Choice to create a state machine that iterates until a condition is met.

For example, consider creating and copying a snapshot from RDS using Step Functions and Lambda functions.
Immediately after performing the snapshot creation, the status of the snapshot will be “creating” and cannot be copied immediately.
In other words, you must wait until the status becomes “available” before performing the copy.

The point we want to achieve using Choice this time is the “wait until the status changes” part.

The Step Functions state machine we are creating will not proceed to the next step until the status changes, and will stop temporarily.
After stopping, it will perform another status check and only proceed to the next step if the status meets the condition.

Environment

Diagram of Loop in Step Functions until the condition is satisfied.

Define four states in the Step Functions state machine.

The two states are tasks that invokes the following Lambda functions

  • Lambda function 1: Obtains the number of seconds in the current date/time and determines whether this is an even or odd number.
  • Lambda function 2: Returns a string indicating that the state machine has completed.

The runtime environment for the function is Python 3.8.

The functions of the remaining two states are as follows

  • Choice: Conditional branching of states. If the number of seconds is even, proceeds to the Lambda function 2 state.
    If the number of seconds is odd, it proceeds to the Wait state.
  • Wait: Wait for 3 seconds. After waiting, returns to the state of Lambda function 1.

CloudFormation template files

The above configuration is built with CloudFormation.
The CloudFormation templates are placed at the following URL

https://github.com/awstut-an-r/awstut-fa/tree/main/140

Explanation of key points of template files

Lambda Functions

Resources:
  Function1:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          import datetime

          def lambda_handler(event, context):
            now = datetime.datetime.now()

            if now.second % 2 == 0:
              return True
            return False
      FunctionName: !Sub "${Prefix}-function-01"
      Handler: !Ref Handler
      Runtime: !Ref Runtime
      Role: !GetAtt FunctionRole.Arn

  Function2:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          def lambda_handler(event, context):
            return 'finish!'
      FunctionName: !Sub "${Prefix}-function-02"
      Handler: !Ref Handler
      Runtime: !Ref Runtime
      Role: !GetAtt FunctionRole.Arn
Code language: YAML (yaml)

Create two functions.

The first is a function that is imagined as a status determination.
Originally, as in the RDS example at the beginning of this section, the content is to check the status of the snapshot.
In this case, since we are verifying the operation, we will focus on the number of seconds in the current date and time.
We will check if this value is even, that is, divisible by 2.
Returns True if it is even, False if it is odd.

The second function signals the completion of the state machine.
Simply returns the string “finish!

The following are the IAM roles for both functions.

Resources:
  FunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: sts:AssumeRole
            Principal:
              Service:
                - lambda.amazonaws.com
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Code language: YAML (yaml)

A simple IAM role with an attached AWS management policy for Lambda.

Step Functions

Resources:
  StateMachine:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      Definition:
        Comment: !Sub "${Prefix}-StateMachine"
        StartAt: FirstState
        States:
          FirstState:
            Type: Task
            Resource: !Ref FunctionArn1
            Next: ChoiceState
          ChoiceState:
            Type: Choice
            Choices:
              - Variable: $
                BooleanEquals: false
                Next: WaitState
              - Variable: $
                BooleanEquals: true
                Next: LastState
          WaitState:
            Type: Wait
            Seconds: !Ref WaitSeconds
            Next: FirstState
          LastState:
            Type: Task
            Resource: !Ref FunctionArn2
            End: true
      LoggingConfiguration:
        Destinations:
          - CloudWatchLogsLogGroup:
              LogGroupArn: !GetAtt LogGroup.Arn
        IncludeExecutionData: true
        Level: ALL
      RoleArn: !GetAtt StateMachineRole.Arn
      StateMachineName: !Ref Prefix
      StateMachineType: STANDARD
Code language: YAML (yaml)

Check the overall structure of the state machine.

The StartAt property shows that the process starts in FirstState.
In FirstState, function 1 is executed.

It then moves to ChoiceState, where a conditional decision is made by referencing the value returned from FirstState.
By specifying “$” in the Variables property, the result of Function 1 can be referenced.
If the value returned from function 1 is True, we move to LastState, function 2 is executed, and the state machine exits.
If the value is False, it moves to WaitState, waits 3 seconds, and then executes FirstState again.

The point is the destination after this WaitState.
By returning to FirstState here, we can reproduce the behavior of checking the status again.
In other words, the process can be repeated until it changes to a specific status.

The following are the IAM roles for this state machine.

Resources:
  StateMachineRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: sts:AssumeRole
            Principal:
              Service:
                - states.amazonaws.com
      Policies:
        - PolicyName: !Sub "${Prefix}-InvokeTaskFunctions"
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - lambda:InvokeFunction
                Resource:
                  - !Ref FunctionArn1
                  - !Ref FunctionArn2
        - PolicyName: !Sub "${Prefix}-DeliverToCloudWatchLogPolicy"
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogDelivery
                  - logs:GetLogDelivery
                  - logs:UpdateLogDelivery
                  - logs:DeleteLogDelivery
                  - logs:ListLogDeliveries
                  - logs:PutLogEvents
                  - logs:PutResourcePolicy
                  - logs:DescribeResourcePolicies
                  - logs:DescribeLogGroups
                Resource: "*"
Code language: YAML (yaml)

Authorization to execute the aforementioned Lambda functions.

Architecting

Use CloudFormation to build this environment and check its actual behavior.

Create CloudFormation stacks and check the resources in the stacks

Create CloudFormation stacks.
For information on how to create stacks and check each stack, please see the following page.

あわせて読みたい
CloudFormation’s nested stack 【How to build an environment with a nested CloudFormation stack】 Examine nested stacks in CloudFormation. CloudFormation allows you to nest stacks. Nested ...

After reviewing the resources in each stack, information on the main resources created in this case is as follows

  • Step Functions state machine: fa-140
  • Lambda function 1: fa-140-function-01
  • Lambda function 2: fa-140-function-02

Check the creation status of Step Functions from the AWS Management Console.

Detail of Step Functions 1.

The state machine has been successfully created.
If you look at the diagram on the right, you will indeed see an arrow going from WaitState back to FirstState.
This allows us to achieve the behavior of repeating the process over and over until the condition is met.

Operation Check

1st

Now that you are ready, execute the state machine.
To execute from the AWS Management Console, press “Start execution”.

Detail of Step Functions 2.

I was able to reach LastState on the first attempt.
I see in the FirstState execution result that the value of output is true.
That is, since the number of seconds was an even number when the Lambda function was executed, it arrived at LastState without proceeding to WaitState.

2nd

Run the state machine again.

Detail of Step Functions 3.

Now we have moved to WaitState.
I see the result of the FirstState execution, the value of output is false.
That means it did not proceed to LastState because the number of seconds was odd when the Lambda function was executed.

After waiting 3 seconds, it moves to FirstState again.

Detail of Step Functions 4.

The second time was true.
This means that the second time the Lambda function was executed, the number of seconds was even, so the Choice state condition was met, and we were able to move to LastState.

In this way, Choice can be used to implement a configuration that repeats a process over and over until the condition is satisfied.

Summary

Choice was used to create a state machine that iterates until a condition is met.

TOC