profile

AWS for the Real World

Networking is still hard in 2026

Published about 1 hour agoΒ β€’Β 3 min read
AWS FOR THE REAL WORLD
⏱️
Reading time: 11 minutes
🎯
Main Learning: A Karpenter NodePool is a placement policy, not an instance preference. Pin one instance family, and a Spot shortage moves your fleet across availability zones, where every internal call starts costing $0.01 per GB.
πŸ“

Hey Reader πŸ‘‹πŸ½

Important off topic things first: Sandro got married! πŸŽ‰
We were in Munich for it and it was a fantastic day! β˜€οΈ
Highly recommend a wedding over sprint planning or fighting with AWS shenanigans!

Now to something less fun: A story I keep hearing, and it always starts the same way.

A cluster gets noticeably more expensive over two days.
1. Nothing was deployed and traffic is flat.
2. All dashboards stay green: pods Running, nodes Ready, no alerts and no errors.
3. But the bill still goes up... significantly πŸ˜…

The only thing that changed is where the nodes are running. That's enough.

In this issue: how a Spot shortage quietly rearranges a cluster across three availability zones, and why that makes internal traffic a costly liability!

Sponsored
Typesense
Search without the cluster
Typesense β€” the open source alternative to Algolia
Terminal: docker run typesense, then a curl search for the misspelled kubernets returning a Kubernetes result

Typo-tolerant search from a single binary. No shard count to pick, no JVM heap to tune, no cluster to keep alive β€” and natural language search is built in.

Smaller learning curve than Elasticsearch  β€’  Django & Laravel Scout integrations

Check it out on GitHub β†’

Sponsored by Typesense, a tool we run ourselves.

Cross-AZ traffic on EKS: the cost nobody watches

πŸ“š This Week's Deep Dive

Three things are true about a lot of EKS clusters using Karpenter (great choice!) as their scheduler:
1. One instance family, benchmarked and standardized across teams
2. Spot preferred, with On-Demand as the fallback
3. No zone-aware routing, because while all the capacity happens to live in one zone, "route anywhere" and "route locally" are the same instruction.

Then Spot capacity for that family runs out in the region. Karpenter does exactly what it was told: it falls back to On-Demand, and it places new nodes wherever it can still get instances. Within a day, a fleet that had effectively lived in one availability zone is spread across three.

Nothing about that is wrong. Spreading across zones is what you are supposed to want. Except the services still talk to each other constantly, and now roughly two thirds of every internal call crosses a zone boundary at $0.01 per GB in each direction.

The traffic pattern never changed. Only the placement did. And there is no Kubernetes signal for placement: no restarts, no failing probes, zero errors. In Cost Explorer it hides inside EC2-Other.

The full story is on the blog: how a NodePool quietly became a placement policy, why one instance family is the thing that set it up, and the four changes that stop it from happening again.

πŸ‘‹ Recommended reading β€” not a sponsor

10 Most Common AWS Savings Opportunities

Not a sponsor and not paid, we just know the team and rate the work. No affiliate link either. They went through more than 100 AWS accounts and wrote up what they actually found, with how often each thing shows up and the median yearly waste attached to it.

Oversized EC2 instances in 46% of accounts. Redundant NAT gateways in another 46%. Unused EKS clusters at a median $876 a year. Every item comes with the CLI commands to check your own account, which is what makes it worth the 18 minutes.

Read the breakdown β†’

πŸ“° This Week in AWS

🩺EC2 application status checks

EC2 now calls your app over HTTP or HTTPS and fails the check when it does not answer. You pick protocol, port, path, and the healthy response codes, and Auto Scaling can replace the instance on that signal. One less custom health Lambda. Read More β†’

πŸ”The AWS sign-in page is changing

Unified email entry for root users, sign-in with Google, GitHub, Apple or Amazon, and a session picker when you have several accounts open. Rolling out gradually. If you script or automate console login anywhere, test it now. Read More β†’

πŸ’°Managed dashboards in Billing and Cost Management

Five preconfigured read-only dashboards for compute, databases, reservations, and Savings Plans, with no setup at all. Fitting timing for this issue: go look at yours right after you finish reading. Read More β†’

Go read your own NodePool with this in mind!
If it names exactly one instance family, you already know what to do.

And if you have had a cost surprise that no dashboard warned you about, hit reply and tell me.
I collect these!

See you next week! πŸ™

Tobi & Sandro

AWS for the Real World

by Tobi & Sandro

We teach AWS for the real world - not for certifications. Join more than 10,500 developers learning how to build real-world applications on AWS.

Read more from AWS for the Real World

AWS FOR THE REAL WORLD ⏱️ Reading time: 6 minutes 🎯 Main Learning: One stack, reused for every project. Hono on Lambda, Postgres with Drizzle, a TanStack SPA on S3 and CloudFront, and Better Auth for login. πŸ“ Blog Post Hey Reader πŸ‘‹πŸ½we've build a lot of fullstack applications so far: client projects side projects (this one, shopify apps, etc.) example tutorial apps Over the past few years we switched up tech stacks a lot. That taught us what actually matters in a stack and what is just...

21 days agoΒ β€’Β 2 min read

AWS FOR THE REAL WORLD ⏱️ Reading time: 10 minutes 🎯 Main Learning: Wildcards come from the tooling, not from laziness. Put least privilege at the account level and let an agent write the policies. πŸ“ Blog Post Hey Reader πŸ‘‹πŸ½ I have shipped my share of s3:* at unusual hours and told myself I would refactor it later - which obviously never happened πŸ˜… So when someone on r/aws asked why developers can't write least privilege policies and put it down to laziness, I was excited to read through all...

27 days agoΒ β€’Β 2 min read

AWS FOR THE REAL WORLD ⏱️ Reading time: 11 minutes 🎯 Main Learning: Most teams should stay serverless. EKS only pays off at real scale. πŸ“ Blog Post Hey Reader πŸ‘‹πŸ½For years we told everyone the same thing: don't run Kubernetes! And we meant it. Running k8s yourself is a second full-time job. Cluster upgrades, etcd backups, some networking plugin that falls over on a Tuesday and nobody can say why.We're serverless people through and through. Lambda first, a queue behind it, scale to zero, go...

3 months agoΒ β€’Β 3 min read