profile

AWS for the Real World

Networking is still hard in 2026

Published 2 months ago • 3 min read
AWS FOR THE REAL WORLD
⏱️
Reading time: 11 minutes
🎯
Main Learning: A Karpenter NodePool is a placement policy, not an instance preference. Pin one instance family, and a Spot shortage moves your fleet across availability zones, where every internal call starts costing $0.01 per GB.
📝

Hey Reader 👋🏽

Important off topic things first: Sandro got married! 🎉
We were in Munich for it and it was a fantastic day! ☀️
Highly recommend a wedding over sprint planning or fighting with AWS shenanigans!

Now to something less fun: A story I keep hearing, and it always starts the same way.

A cluster gets noticeably more expensive over two days.
1. Nothing was deployed and traffic is flat.
2. All dashboards stay green: pods Running, nodes Ready, no alerts and no errors.
3. But the bill still goes up... significantly 😅

The only thing that changed is where the nodes are running. That's enough.

In this issue: how a Spot shortage quietly rearranges a cluster across three availability zones, and why that makes internal traffic a costly liability!

Sponsored
Typesense
Search without the cluster
Typesense — the open source alternative to Algolia
Terminal: docker run typesense, then a curl search for the misspelled kubernets returning a Kubernetes result

Typo-tolerant search from a single binary. No shard count to pick, no JVM heap to tune, no cluster to keep alive — and natural language search is built in.

Smaller learning curve than Elasticsearch  •  Django & Laravel Scout integrations

Check it out on GitHub →

Sponsored by Typesense, a tool we run ourselves.

Cross-AZ traffic on EKS: the cost nobody watches

📚 This Week's Deep Dive

Three things are true about a lot of EKS clusters using Karpenter (great choice!) as their scheduler:
1. One instance family, benchmarked and standardized across teams
2. Spot preferred, with On-Demand as the fallback
3. No zone-aware routing, because while all the capacity happens to live in one zone, "route anywhere" and "route locally" are the same instruction.

Then Spot capacity for that family runs out in the region. Karpenter does exactly what it was told: it falls back to On-Demand, and it places new nodes wherever it can still get instances. Within a day, a fleet that had effectively lived in one availability zone is spread across three.

Nothing about that is wrong. Spreading across zones is what you are supposed to want. Except the services still talk to each other constantly, and now roughly two thirds of every internal call crosses a zone boundary at $0.01 per GB in each direction.

The traffic pattern never changed. Only the placement did. And there is no Kubernetes signal for placement: no restarts, no failing probes, zero errors. In Cost Explorer it hides inside EC2-Other.

The full story is on the blog: how a NodePool quietly became a placement policy, why one instance family is the thing that set it up, and the four changes that stop it from happening again.

👋 Recommended reading — not a sponsor

10 Most Common AWS Savings Opportunities

Not a sponsor and not paid, we just know the team and rate the work. No affiliate link either. They went through more than 100 AWS accounts and wrote up what they actually found, with how often each thing shows up and the median yearly waste attached to it.

Oversized EC2 instances in 46% of accounts. Redundant NAT gateways in another 46%. Unused EKS clusters at a median $876 a year. Every item comes with the CLI commands to check your own account, which is what makes it worth the 18 minutes.

Read the breakdown →

📰 This Week in AWS

🩺EC2 application status checks

EC2 now calls your app over HTTP or HTTPS and fails the check when it does not answer. You pick protocol, port, path, and the healthy response codes, and Auto Scaling can replace the instance on that signal. One less custom health Lambda. Read More →

🔐The AWS sign-in page is changing

Unified email entry for root users, sign-in with Google, GitHub, Apple or Amazon, and a session picker when you have several accounts open. Rolling out gradually. If you script or automate console login anywhere, test it now. Read More →

💰Managed dashboards in Billing and Cost Management

Five preconfigured read-only dashboards for compute, databases, reservations, and Savings Plans, with no setup at all. Fitting timing for this issue: go look at yours right after you finish reading. Read More →

Go read your own NodePool with this in mind!
If it names exactly one instance family, you already know what to do.

And if you have had a cost surprise that no dashboard warned you about, hit reply and tell me.
I collect these!

See you next week! 🙏

Tobi & Sandro

AWS for the Real World

by Tobi & Sandro

We teach AWS for the real world - not for certifications. Join more than 10,500 developers learning how to build real-world applications on AWS.

Read more from AWS for the Real World

⏱️ Reading time: 15 minutes 🎯 Main Learning: A public container on AWS for $7 a month, with no VPC to own. 📝 Blog Post 🤝 Sponsored by AWS. What we built, what it cost and what we think of it are our own. Hey Reader 👋🏽 Your first container on ECS comes with homework: a VPC, subnets in two Availability Zones, a load balancer, security groups, a certificate and a task role. Express Mode creates most of it for you now (App Runner's successor, so ECS abstraction number three), but it all lands in...

5 days ago • 2 min read

AWS FOR THE REAL WORLD ⏱️ Reading time: 6 minutes 🎯 Main Learning: You cannot skip a level of AWS observability. Structured logs come first, then the AI agent that reads them for you. 📝 Blog Post 📂 Claude Code agent (free gist) 🎬 Watch on YouTube Hey Reader 👋🏽 This week I'm bringing back one of our most-read posts: the 3 levels of AWS observability. A lot of you joined after it went out in May, and the topic fits this week perfectly. Because the real question isn't "do I have logs?" anymore....

18 days ago • 2 min read

AWS FOR THE REAL WORLD ⏱️ Reading time: 10 minutes 🎯 Main Learning: Every convenience layer AWS built on ECS is dead or dying, while ECS itself has not changed since 2014. Build on the primitive, and make the layer above it prove itself first. 📝 Blog Post Hey Reader 👋🏽 Getting started with an AWS service often feels more complicated than it should. Four concepts must be understood before a single container runs. A VPC is required before a single request. So, the fancy CLI that promises to do...

about 1 month ago • 3 min read