Hey Reader 👋🏽

Ever tried setting up an AWS Landing Zone? If you have, you know it's not easy. AWS recommends using a separate account just for monitoring all your log data.

We're here to introduce the AWS Observability Access Manager (OAM), designed to make this task easier.

Previously, we couldn't use OAM effectively due to a major limitation, but that's changed.

Interested in diving straight into coding? Visit our repo to deploy your setup using Terraform or CDK.

Why Use Multiple Accounts?

Using several AWS accounts helps to isolate applications, manage finances better, and use resources more efficiently.

A multi-account setup could look like this:

But, it's challenging to oversee operations across these accounts. That's where a dedicated monitoring account comes in, gathering all logs, traces, and metrics in one place.

Aggregating Log Data

Manually aggregating logs across accounts and regions can get complicated. Our own Eduardo Rabelo has an amazing blog post on different strategies for Cross-Account and Cross-Region Log Aggregation.

One common approach is to build a manual solution with Kinesis to ingest all logs. See this architecture:

This can be very error-prone and development-intensive.

OAM offers a simpler solution, allowing you to view all application components in one place, analyzing data across accounts.

What is the Observability Access Manager (OAM)?

The Observability Access Manager (OAM) is a tool and set of APIs built by AWS. It helps you set up a dedicated monitoring account in a multi-account environment.

OAM allows you to monitor all the components of your applications from a centralized view.

It creates cross-account observability to search, analyze, and correlate data stored in CloudWatch.

We want to get observability over data such as:

• CloudWatch Logs
• CloudWatch Metrics
• X-Ray Traces

from multiple accounts into one dedicated Monitoring account. This tool is a managed solution of what we wanted to build in the first step.

Sinks Point to the Monitoring Account and Links Send Application Data

OAM uses three main components:

• Sink: The monitoring account destination.
• Link: Connects your application's account to the monitoring account.
• Sink Policy: Allows connections from source accounts to the monitoring account.

We see three AWS accounts in the previous picture:

1. Source Account A
2. Source Account B
3. Monitoring Account

Imagine both your Source accounts are running two different applications, such as a REST API in A and User Authentication in B.

One user request touches the authentication account and creates logs. It also touches the REST API and creates logs. We need to have one central place to correlate the logs. This is possible with a monitoring account.

OAM helps us with that. You can create a single Sink in us-east-1 in the Monitoring account. You create a Link in the AWS accounts running workloads where logs, metrics, traces, and insights are generated.

The Limit That Made Us Not Working with OAM in the Past

Before, OAM had a one-sink-per-account limit, making it difficult to monitor applications across multiple regions.

Imagine having our account structure again like in the following image:

In this setup, we have two regions (us-east-1 and ap-southeast-2). But since we could only create one sink per account we can only aggregate all logs from us-east-1.

This doesn't help us when our actual application is scattered across regions, which it often is.

​Steve Weldon (thanks!) found out that it is actually possible to create multiple sinks per account. That means we can achieve the following setup:

The official quota still states that only one Sink per account is possible, so we're eagerly awaiting some feedback from AWS:

Deploying Your Setup!

We don't come without code 😎

​Eduardo Rabelo built the whole flow in CDK (SST/ION) and Terraform. Check out the GitHub repository for that.

This repository assumes that you use AWS Organizations and know how to handle the trust relationship of the role OrganizationAccountAccessRole.

If you need help with that, please comment or DM us in our Discord server 😉

After you have all three accounts, you can supply the account IDs after you hit terraform apply.

Logs from Both Accounts

We've deployed a Lambda function in both source accounts to see it in action. We've executed the Lambda a few times to generate some Logs.

We see both Log Groups from both accounts (OAM-Source and OAM-Source-2). The logs are arriving in near-real-time. The same thing is happening with Metrics:

Summary

OAM simplifies setting up cross-account observability, saving you the trouble and cost of manual solutions.

Struggling with CloudWatch? We're working on a CloudWatch Book, aiming for a Q2 release.

Tobi & Sandro our goal is to simplify AWS & Cloud Learning for everybody. You don't need expensive certifications to build on AWS! AWS Fundamentals Blog​

Grünwalder Strasse 145a, Munich, Bavaria 81547 · Unsubscribe · Preferences​