profile

AWS for the Real World

AWS analytics feels slow by default

Published about 10 hours ago • 1 min read
⏱️
Reading time: 21 minutes
🎯
Main Learning: Same CloudFront logs: 12.5s through Kinesis, S3, Glue and Athena, about 1s through Tinybird's managed ClickHouse.
📝

Hey Reader 👋🏽

We've been running Plausible for the analytics on awsfundamentals.com. Good software, no complaints. We still wanted the data to be ours: our retention, our schema, nobody's script in the visitor's browser.

So we built the dashboard ourselves. Twice, on two different backends, fed by the same CloudFront logs.

Then we measured both instead of arguing about them.

In this issue: what AWS-native analytics actually costs in seconds and in dollars, and where it holds up fine.

Sponsored
Tinybird — ship fast over a managed ClickHouse

SQL in, REST API out

Stream events into Tinybird, write a SQL pipe, publish it. That's your API: ClickHouse underneath, no serving layer to write, nothing to keep warm. Schemas and pipes are files in your repo, and you test a schema change on a branch of real production data before it ships.

Sponsored by Tinybird. This issue's deep dive was done in collaboration with them, and every number in it is measured from our own setup.

We built the same analytics dashboard on AWS-native and Tinybird

📚 This Week's Deep Dive

One dashboard, two backends, the same CloudFront real-time logs feeding both.

The AWS-native side: Kinesis, a writer Lambda, JSON on S3, a Glue crawler keeping the catalog current, Athena on top. The other side: the same Kinesis stream, a forwarder Lambda, a Tinybird Data Source, and four SQL pipes published as endpoints. One frontend queries both and renders them next to each other, so every number came out of the same UI.

Both pipelines side by side: AWS-native on top, Tinybird below, both feeding the same dashboard

Then we stopped guessing at Athena's reputation and measured it: the same four dashboard queries against both backends, every ~80 seconds for 15 minutes. 48 calls each side, nothing warmed up, nothing cached.

Athena never answered in under 11 seconds. Not on the first call, not on the 48th. The band sat between 11.3s and 16.1s the entire run and averaged 12.5s. Tinybird stayed under 2s throughout, averaging about 1.0s, on the free tier.

Part of that Athena number is our own file layout, not Athena, and we break that down honestly on the blog. Along with the identical schema change run on both sides with a stopwatch, what a Glue crawler costs once you want fresh data, and the one thing that made AWS-native competitive again.

The honest takeaway: AWS-native isn't broken, it's just cold every single time. You can fix that yourself with a cache layer and Partition Projection, and then you own a cache layer.

See you next week! 🙏

Tobi & Sandro

​Sandro Volpicella & Tobias Schmidt from AWS Fundamentals​
​Cloud Engineers • Fullstack Developers • Educators

Manage your email preferences 📨

AWS for the Real World

by Tobi & Sandro

We teach AWS for the real world - not for certifications. Join more than 10,500 developers learning how to build real-world applications on AWS.

Read more from AWS for the Real World

AWS FOR THE REAL WORLD ⏱️ Reading time: 11 minutes 🎯 Main Learning: A Karpenter NodePool is a placement policy, not an instance preference. Pin one instance family, and a Spot shortage moves your fleet across availability zones, where every internal call starts costing $0.01 per GB. 📝 Blog Post Hey Reader 👋🏽 Important off topic things first: Sandro got married! 🎉We were in Munich for it and it was a fantastic day! ☀️Highly recommend a wedding over sprint planning or fighting with AWS...

14 days ago • 3 min read

AWS FOR THE REAL WORLD ⏱️ Reading time: 6 minutes 🎯 Main Learning: One stack, reused for every project. Hono on Lambda, Postgres with Drizzle, a TanStack SPA on S3 and CloudFront, and Better Auth for login. 📝 Blog Post Hey Reader 👋🏽we've build a lot of fullstack applications so far: client projects side projects (this one, shopify apps, etc.) example tutorial apps Over the past few years we switched up tech stacks a lot. That taught us what actually matters in a stack and what is just...

about 1 month ago • 2 min read

AWS FOR THE REAL WORLD ⏱️ Reading time: 10 minutes 🎯 Main Learning: Wildcards come from the tooling, not from laziness. Put least privilege at the account level and let an agent write the policies. 📝 Blog Post Hey Reader 👋🏽 I have shipped my share of s3:* at unusual hours and told myself I would refactor it later - which obviously never happened 😅 So when someone on r/aws asked why developers can't write least privilege policies and put it down to laziness, I was excited to read through all...

about 1 month ago • 2 min read