Hey Reader!

Hope you're having an awesome week! This week's edition is about implementing Step Function Workflows. One major use-case that Step Functions enable is the Wait for Callback pattern.

Code Included 👨🏽‍💻

This edition comes with a special bonus - for the first time, we're including a GitHub repository just for you! Clone the repo and deploy it in your AWS Account. ⚡

Step Functions Orchestrate Workflows

What are even Step Functions?

Step Functions orchestrate workflows with different AWS Services. They are all about making different AWS Services work together in harmony.

You can make use of direct integrations like:

• 💾 Scanning items in DynamoDB
• ⚡️ Invoking Lambda functions
• 📤 Sending WebSocket Messages
• 📊 Getting data from external APIs

You can have dedicated error & retry mechanisms for each step within a Step Function, making it a powerful orchestrator.

Today, we will build such a workflow together.

The Workflow Designer

AWS offers you a great tool to get started with Step Functions: the workflow designer.

It's a bit like playing with digital LEGO on your screen, where you can drag and drop pieces to build your workflow. The workflow designer is a low-code tool. Meaning that you don't need to code your workflow. Instead, you'll design it.

Here is an example from the workflow designer. It is a demo workflow by AWS:

However, in the end, you need to have some sort of code to include it in your Infrastructure as Code (IAC) provider. From the workflow designer, you can export a .json file in the Amazon State Language (ASL).

Or with an IAC framework like the CDK (which we use in the GitHub repository), you can build up your whole Step Function with its own syntax.

We prefer the latter one. It can be quite cumbersome to export a huge .json file into your code base every time you update one option in your state.

To get started with Step Functions we recommend using the workflow designer first. It shows you what step functions are capable of. But if you want to have it in production, translate it into IAC.

Callback Patterns Let You Wait for User Input

Today, we're diving into the magic of the callback pattern with Step Functions, a feature that's like putting your workflow on pause. Think of it as a "Hold, please! 👋" button for your tasks. The callback pattern lets a part of your workflow wait quietly until it's given the green light to go again.

Picture this: you grab some data from DynamoDB, send it over to a Lambda function, and then—here's where the magic happens—you tell the next Lambda task to hang tight until it gets a special signal, a callback.

This setup is perfect when you need a human interaction, like a manual check or approval, before moving forward. Want to see how we put this together? Check out the Step Function Definition on GitHub and follow along with us.

Real-world Application: Content Moderation

Our use case for implementing the callback pattern is a Content Moderation System. Imagine hosting a blogging platform such as Hashnode or Medium. You don't want to enable everybody to publish anything. This is why you put some safeguards into place.

We've created a streamlined model for this concept. Here's the scenario: when someone submits an article that includes a specific no-go word (yep, "forbidden" is our trigger word), we don't just push it live. Instead, we pause and call in a human – a manual reviewer – to give it the once-over.

Is it okay to publish, or does it cross the line?

This is how the end state will look like:

​

1. An article title is submitted.
2. Our Step Function kicks into gear.
3. If the title includes "forbidden", everything pauses, waiting for a special nod – a task token.
4. Over in the admin dashboard, there's a new article flagged for review.
5. Upon approval, the Step Function springs back into action, moving the process forward.

It's a simple but effective way to keep tabs on content.

Diving Into the Architecture

The architecture consists of several parts:

• Frontend - a Next.JS/React frontend that handles the submissions and admin dashboard
• Lambda API - a REST API based on a Lambda Function URL that handles all business logic
• Step Function - a Step Function to handle the moderation workflow
• DynamoDB - all workflow executions that need to be reviewed including all URLs
• Lambda Approval Receiver - another API to approve or reject the Step Function

This is a birds-eye view. Let's dive into it.

Triggering Step Functions

​

Once the user submits an article we start a new Step Function with the given title.

• Content Check: First up, the Step Function scans the title. If the "forbidden" word is present, we'll pause the Step Function.
• Requesting a Review: Once paused a Lambda function is executed that sets the workflow into its waiting state. The execution ARN and some other information are saved in DynamoDB.

The Wait for Callback Explained

This waiting game means the Step Function halts until a Task Token is received from the Step Function API. This token is essential; it's the green light our Lambda function needs to proceed.

In the workflow designer, the Lambda Function looks like this:

We've configured the input of the task token (see taskToken.$: $$.Task.Token), and we set the configuration Wait for callback, which pauses the whole workflow.

In CDK the Lambda Task looks like this:

Once a Step Function has the word forbidden in its title the Step Function will halt and look like this:

​

Our Lambda Function Ask For Manual Approval, standing by for a callback, also creates URLs for approving or rejecting the article in question.

These URLs, along with the details of the waiting Step Function, are stored in DynamoDB. The following item is stored in DynamoDB:

With this system in place, retrieving the list of pending reviews and their respective decision-making URLs becomes a breeze.

Decision Time: Approve or Reject?

​

When our Step Function waits, admins jump in. They head to a simple dashboard at the /admin route in our frontend. Here, they see articles waiting for a thumbs up or down, each with its link.

Clicking approve or reject sends their choice straight to a special function. This function tells the Step Function what to do next:

Once this API call is sent the Step Function continues and stores a boolean value for decision in the output of the Lambda function.

​

​

Recap & Resources

Step Functions are a great service in AWS to orchestrate and handle complex workflows. Using the Callback Pattern is one of the best use cases for Step Functions. While the input and output transformations of the data can be quite complex, it is a very good way to build observable workflows.

Key Takeaways:

• Step Functions are your workflow orchestrators, ensuring each step of your process harmonizes with the next.
• The Workflow Designer is a good way to get started, but you need IAC in the end.
• Callback Patterns offer the flexibility to pause your workflows, awaiting human input or external triggers to proceed.
• Through our Content Moderation System example, you've seen how these concepts apply in real-world scenarios, ensuring quality and control.

Go ahead, deploy the GitHub Repository to your AWS account, and change a few things. For example:

• store the article in a DynamoDB with a direct integration
• check the input for more words than just forbidden
• send the user a notification based on WebSockets, Emails, or SMS

Resources to Explore:

• ​AWS Workshop for Step Functions​
• ​GitHub Code​
• ​Wait for Callback Documentation​
• ​Getting Started with Step Functions​

​

Thanks & Question for you

Thank you for reading so far!

Do you like these hands-on editions with included & deployable code?

They cost much more time but we think the value readers get out of that is great. Instead of covering one service in all its details like in the first year of this newsletter, we want to do more deep-dives with code included.

Let us know what you think!

See you soon ✌🏽
Sandro & Tobi ⛅

​

Tobi & Sandro our goal is to simplify AWS & Cloud Learning for everybody. You don't need expensive certifications to build on AWS! AWS Fundamentals Blog​

Dr.-Otto-Bößner-Weg 7a, Ottobrunn, Bavaria 85521 · Unsubscribe · Preferences​