Join our community of over 8,800 readers delving into AWS. We highlight real-world best practices through easy-to-understand visualizations and one-pagers. Expect a fresh newsletter edition every two weeks.
π Update #3: Deep Dive into CloudWatch with Our GitHub Tracker Tutorial, Video Included
Hey Reader, amazing that you're interested in creating the CloudWatch Book together with us. This is the third CloudWatch Update. Coincidentally, Sandro (I'm writing this update) turned 30 last week π₯³ In this update, we'll introduce you to our newly created web application: The GitHub Repository Tracker. While it is not the fanciest name it, the project will help you a lot in learning & understanding CloudWatch for the real world. We thought it would be much easier to show it in a video so here we go π½οΈ
Learning the basics and theories of an AWS service like CloudWatch can be tough. In a real-world scenario, you rarely try to use CloudWatch by just reading through the documentation. You will always have some application or workload you need to debug, develop, or fix some incident that is currently going on. This is what our example application is there for. We want to give you a deployable application that shows you a real usage of CloudWatch. What is the Application about?As the name suggests the example application lets you track GitHub repositories. You can add repositories and we will show you some stats such as
While this information is not necessarily the most interesting one we want to show you how different AWS services interact and how you can observe applications with CloudWatch. You will especially learn how to:
β ArchitectureLet's go through the architecture of the application. β The web application consists of mainly three parts:
FrontendThe front-end code is not the core focus of this application. β However, if you build full-stack applications on AWS you will see clients and web applications a lot. This is why we will also give you full access to the frontend code. The front end is built with Next.JS and is therefore based on React. For styling, we have used Tailwind CSS. All files are exported as static files and saved in an S3 bucket. We use CloudFront to serve the front end to all end users. Backend (Part 1 - Rest API)The backend gets more interesting. Everything is built in Infrastructure-as-Code. We use Terraform and CDK. Why both? It's pretty simple:
We want to give you the best out of both worlds and show you how it's done in a real-world setting! ποΈ This pushes back the timeline quite a bit but we are sure it is worth it! β β Let's go through the rest of the architecture. The main services we use in the backend are:
Everything is serverless we don't use any virtual machines or containers. Mainly because we want everybody can deploy our application without any costs. But we also believe that such applications should be serverless by default in the future (another discussion). S3 and CloudFront are there to serve the front end to users. The main part of the backend consists of an API. We use API Gateway with a Lambda integration to handle all synchronous workloads. Every time you read, add, or delete a repository you will hit API Gateway. API Gateway forwards the request to Lambda, and Lambda gets the data from DynamoDB or calls the GitHub API and adds the repository to DynamoDB. This brings us to our external API -> The GitHub API. For us, it was very important to add an external API call. We want to show you traces, logs, and network calls within CloudWatch. Adding an external dependency makes it much more interesting. This is just one part of the backend. Backend Part 2 - Asynchronous Workloads with Step FunctionsThe last part of the architecture is our asynchronous workload. All APIs we have used until this point are synchronous. Adding, removing, or reading repositories from a database are synchronous operations. Asynchronous operations are the ones that run in the background. We have implemented one asynchronous schedule that updates all repositories that you have saved. Once every 5 minutes we will update your repositories. This happens with the following services:
β WebSocket APIβ β Once you open the frontend a WebSocket connection is opened to API Gateway. The connection ID is saved in DynamoDB. This is meant to give you an interaction with our asynchronous workloads. β Step Function in the BackgroundThe background task checks if there is any update in the repository. β The StepFunction iterates through all saved repositories, checks if there is any update. If there is an update you will get notified via a WebSocket directly in the UI. Additionally, you will receive an email (if you have configured your email). All of that runs in the background and is asynchronous. CloudWatch IntegrationsWhat has all of that now to do with CloudWatch? Every part of this application is observable and helps you to understand CloudWatch. User Traces with X-RayEach user request that happens is traceable via X-Ray. β Some users experience a bad performance. Have a look at the service map and identify what is going wrong. Feature Flagging with EvidentlyModern software development cycles use continuous deployment. Developers want fast feedback loops and we don't want to wait for month-long releases just to figure out we broke something. This is where feature flagging comes into play. Evidently (see the last newsletter) allows you to do exactly that! You can toggle features. In our example application, we want to toggle multiple things:
To make it even better, you can toggle the feature flags directly from the front end without using the AWS console. Google Analytics from AWS with RUMCloudWatch doesn't only help you with debugging and observability, it also helps you understand what your users are doing. And this for a super little price. CloudWatch Real User Monitoring is like a little Google Analytics alternative, just much easier to use. We've integrated it with our GitHub tracker and now we can see various data such as:
...and much more! All of that within AWS, within your region for a very low price. Synthetic Monitoring with CanariesWe want to make sure our application and all of its features are working at all times. With CloudWatch Synthetics, we've integrated continuous monitoring of our application by simulating user behavior and interactions through canaries - configurable scripts that run at specified intervals. These canaries navigate and interact with the application, e.g. by adding or removing repositories, which allows us to verify its availability and latency, and to ensure that critical workflows are functioning as expected. By leveraging CloudWatch Synthetics, we gain insights into the performance of our application endpoints and APIs, even when there is no traffic by ourselves. This proactive approach helps us to detect issues early. β The entire book is written around this applicationWe have built the entire book around this application. Each CloudWatch feature is integrated to show you its capabilities. If we say for the real world we want to show it with real-world examples. We could go on and give you examples for each CloudWatch feature that is integrated as well, such as Logs, Metrics, Live Tail, Logs Insights, Anomaly Detection, etc., but we believe you get the point. β Thanks πThis newsletter gave an update and an overall overview of our example application. This is the heart of the book and the whole project. The source code will be available once we launch the book. It is built so that you can deploy it, extend it, and try it out in your environment. Without paying anything for infrastructure costs. Thanks for being part of that amazing journey. If you have any other ideas for additional things to add or if you miss any services please let us know! We read every reply. See you in two weeks Sandro & Tobi βπ½ We appreciate your interest in the CloudWatch Book! If you only want to receive the AWS Newsletter and not these updates anymore, please update your preferences here. Best βπ½ |
Dr.-Otto-BΓΆΓner-Weg 7a, Ottobrunn, Bavaria 85521 Β· Unsubscribe Β· Preferencesβ |
AWS for the Real World
Join our community of over 8,800 readers delving into AWS. We highlight real-world best practices through easy-to-understand visualizations and one-pagers. Expect a fresh newsletter edition every two weeks.
β Reading time: 12 minutes π Main Learning: CloudWatch Launches re:invent 2024 βοΈ Read the Full Post Online π Hey Reader ππ½ re:invent happened already two weeks ago and there were some amazing launches π CloudWatch got a lot of love at that re:invent. This is why we are showing you our top CloudWatch launches for this year. We've worked through all of them, tried to get them working with our example application of the CloudWatch Book, and are now busy updating the book βπ½. Let's dive into...
β Reading time: 14 minutes π Main Learning: Feature Flags with AWS AppConfig πΎ GitHub Repository βοΈ Read the Full Post Online π Hey Reader ππ½ There's no other field where it's so common to have "a small side-project" like in the software industry. Even though it's possible to build things as quickly as ever before due to cloud providers, tools, platforms, and AI, many indie founders (and also large enterprises) tend to fall into the same trap: they tend to build features that users do not...
β Reading time: 17 minutes π Main Learning: Observability at Scale with Open-Source πΎ GitHub Repository βοΈ Read the Full Post Online π Hey Reader ππ½ Welcome to this edition of the AWS Fundamentals newsletter! In this issue, we're focusing on observability with open-source tools on AWS. As most of you already know, we can use Amazon CloudWatch and X-Ray to monitor our application from every angle. But what if we want to hybrid setup where we run certain parts of our ecosystem outside of AWS?...