Exploring CloudWatch Synthetic Canaries: Hands-On Learning with SST ION ⚡️


  • ⌛ Reading time: 7 minutes
  • 🎓 Main Learning: Visual Regression Testing with AWS CloudWatch Synthetics
  • 👨🏽‍💻 GitHub Code
  • 📝 Blog Post

Hey Reader

Keeping your website running smoothly is important.
It's also important to keep an eye on its visual appearance and that it doesn't change due to an unwanted change.

AWS CloudWatch Synthetic Canaries is a tool that helps you check on your websites (and APIs!) to make sure it’s working well and look as expected.

In this newsletter, we’re going to show you how to set up a simple website and how to use canaries to keep an eye on it.

What You Will Learn In This Issue

  • 🏗️ Application Deployment: How to set up and deploy a web application (featuring Astro) using SST Ion (the latest and greatest kid on the IaC block! 👶). This will even include a few Pulumi components.
  • 🕵️ Monitoring with Canaries: Using Synthetic Canaries to ensure our visual regression on our sample website.
  • 👩‍💻 Hands-On Repository: Access to a GitHub repository to actually deploy what you've learned in your own AWS account.

💡 The application's full repository can be found on our Github organization. You can deploy it with SST Ion to your AWS account or just run a local development version.

The Application Setup

The application is pretty straightforward. It’s a simple web app built with Astro.

We’re using SST Ion, a flexible infrastructure as code (IaC) framework that’s perfect for deploying modern web application frameworks.

SST Ion simplifies deploying cloud applications, allowing developers to focus more on code and less on infrastructure management.

SST vs. SST Ion: The new engine of SST is Ion. It’s built on top of Pulumi and Terraform. This makes it easy for us to extend the functionality and build our own “stacks”. You can read more here.

For our example app, which showcases how Canaries work at AWS, you can check out our GitHub repository.

The only thing you need to do is to install SST Ion beforehand:

Afterward, you can already deploy the application to your AWS account via sst deploy .

The deployment process will do all the necessary steps to build and deploy the application to AWS by using S3 and CloudFront. Besides that, our script will deploy and start the monitoring of canaries in your AWS account.

After deploying, SST will provide a URL where your site is hosted. This site serves as a perfect candidate for our monitoring tutorial using AWS Synthetic Canaries.

Monitoring with AWS Synthetic Canaries

Now, let’s talk about monitoring. AWS Synthetic Canaries are not just for checking APIs but are incredibly effective for monitoring websites.

This is because canaries can collect screenshots and compare them with each other.

The script is very simple:

The only thing the script needs to do is:

  1. Enable the visual comparison mode by setting the necessary configuration flags. (visualCompareWithBaseRun and visualVarianceThresholdPercentage).
  2. Go to the site passed via the environment variable SITE_URL.
  3. Wait for it to load fully (waitUntil: networkidle0).
  4. Take a screenshot.

If you jump to the CloudWatch console, you’ll find our new canary in the section Application Signals > Synthetic Canaries.

The first run will collect a baseline and store it in S3. Further runs will then compare this initial screenshot to the current one.

By default, after deploying with SST, the canaries will start to run. The first run will be triggered immediately, but you can always start or stop the canary in the console via the Actions dropdown.

If a run has been completed, we’ll see this in the Availability tab.

There we’ll also find the Screenshots tab. This will show us the screenshots (in our case just one) that the canary has collected.

If this was the first run, this first screenshot will be set as the baseline. Each of our further runs will be compared with this screenshot to detect any (unwanted) changes.

Let’s test exactly that by changing our frontends content. We can do this for example by changing our content’s max-width to a much smaller value:

With this, we’ll trigger the comparison to fail. So let’s deploy our changes with sst deploy to AWS and wait for our next run.

After the run has finished, we should see the failure.

CloudWatch has detected the change. Another look into the screenshots tab shows us the expected result. CloudWatch also provides us with another screenshot, showing the actual differences highlighted in yellow:

In our example change, the actual difference was 13.21%. The threshold at which CloudWatch starts to fail the check can be set via the configuration property visualVarianceThresholdPercentage. In our example, we went with 0 to fail at any change. In a real-world scenario, this value should be higher as very small layout adjustments shouldn’t result in a failed regression test.

Continuing with our failed check: what if this is exactly what we want it to be?

If we don’t return our page to the look and content it had in the first screenshot that CloudWatch took, we’ll get a failure for every run that has to come.

But we can also tell CloudWatch, that this should now be the new baseline to compare against.

Let’s click on Actions > Edit and scroll down to the Visual Monitoring section.

Here, we can enable CloudWatch to use the next run’s screenshot to overwrite the existing base image.

By that, future runs won’t end in failures and the next runs should be green again.

Conclusion

By using AWS Synthetic Canaries, you’re not just just simply monitoring the uptime of your site. You can actively check for the expected look and feel. This helps to quickly spot and address visual issues in any web application.

This visual regression feature even comes with a blueprint, making it easy to get started.


This was all for this week!

Thank you for reading! 💛

Tobi & Sandro

our goal is to simplify AWS & Cloud Learning for everybody. You don't need expensive certifications to build on AWS!

Dr.-Otto-Bößner-Weg 7a, Ottobrunn, Bavaria 85521 · Unsubscribe · Preferences

AWS for the Real World

Join our community of over 8,800 readers delving into AWS. We highlight real-world best practices through easy-to-understand visualizations and one-pagers. Expect a fresh newsletter edition every two weeks.

Read more from AWS for the Real World

⌛ Reading time: 14 minutes 🎓 Main Learning: Feature Flags with AWS AppConfig 👾 GitHub Repository ✍️ Read the Full Post Online 🔗 Hey Reader 👋🏽 There's no other field where it's so common to have "a small side-project" like in the software industry. Even though it's possible to build things as quickly as ever before due to cloud providers, tools, platforms, and AI, many indie founders (and also large enterprises) tend to fall into the same trap: they tend to build features that users do not...

⌛ Reading time: 17 minutes 🎓 Main Learning: Observability at Scale with Open-Source 👾 GitHub Repository ✍️ Read the Full Post Online 🔗 Hey Reader 👋🏽 Welcome to this edition of the AWS Fundamentals newsletter! In this issue, we're focusing on observability with open-source tools on AWS. As most of you already know, we can use Amazon CloudWatch and X-Ray to monitor our application from every angle. But what if we want to hybrid setup where we run certain parts of our ecosystem outside of AWS?...

⌛ Reading time: 9 minutes 🎓 Main Learning: Polling or WebSockets: Choosing with Amazon API Gateway 👾 GitHub Repository ✍️ Read the Full Post Online 🔗 Hey Reader 👋🏽 What would you use for quick and regular data updates inside your web app? Or let's phrase it another way: how would you choose between Polling and WebSockets? 💭 Understanding the nuances between these two communication methods is important, as they both come with their own advantages, gotchas, and side effects that are not very...