profile

AWS for the Real World

The IAC Generator by AWS: A True Game-Changer or Just Hype?

Published about 2 months agoย โ€ขย 5 min read

โ€‹

Hi Reader ๐Ÿ‘‹๐Ÿฝ

This week, we're diving headfirst into the deep end of AWS's pool of innovations with the IAC Generator โ€” a tool that's been causing quite the buzz.

The promise AWS gives us is quite bold:

Generate AWS CloudFormation templates and AWS CDK apps for existing AWS resources in minutes.

But does it really work? Let's find out together.

This will be a hands-on ๐Ÿ‘ท๐Ÿฝโ€โ™€๏ธ issue.

For the impatient, TL;DR, in a nutshell, ๐Ÿฅœ: Good for a start, but far away from usable in production.

๐Ÿ›  The Experiment

We took the Terraform-powered web app we built for the CloudWatch Book ๐Ÿ“• for a spin with the IAC Generator, aiming to seamlessly migrate it into a CDK app.

We want to offer the code in CDK & Terraform, and CDK needs yet to be developed. That sounds like the perfect approach! ๐Ÿ—๏ธ

The architecture of that project looks like this:

Not overly complex, but also not simple.

Let's see how that goes. ๐Ÿ‘€

๐Ÿช„ The IAC Generator

The IAC Generator is responsible for importing AWS resources into CloudFormation.

The process of generating CloudFormation based on your account looks like this:

  1. Scan all resources ๐Ÿ‘€
  2. Create a template ๐Ÿ—๏ธ
  3. Add resources to the template โœš
  4. Import the resources managed by CloudFormation or move to CDK. ๐Ÿ”€

Step 1: Generate CloudFormation ๐Ÿง™๐Ÿฝ

Go over to the CloudFormation console in your desired region and open the IAC Generator Tab.

Scan your resources

Hit the button Start a new scan. Your account will now be scanned for resources and their relationships. This scan can take a while.

The scan in our account ran for about 5 minutes and found 1345 resources.

Create a template

Next, we create a new template called cw-book-example-app.

Selecting Resources

We can see all resources in the account now.

One problem is that the CloudFormation importer uses the Resource Identifier which is an ID. This is not always obvious. Look at API Gateway for example. The Resource Identifier doesn't give you a clue which API it is exactly.

Whereas, the name does it. Serverless Hero Yan already pointed that out in his blog post about the IAC Generator.

We don't want to copy the whole AWS account into one template, as we have development and production resources in the same account (shame on us).

That is why we need a way to choose resources that should be added to the template.

This is where Tags come in. As you will learn on your AWS journey, tags play a crucial role in multiple services on AWS. We can filter the resources by tags. Let's do that.

All our resources have the following tags:

That should be sufficient enough to distinguish.

We can't search by multiple tags, but since this is a dedicated account only for this one application it is okay to add the filter for:

  • Tag key: Environment
  • Tag value: dev

We add all 44 selected resources to our CloudFormation template, including all related resources.

Template is generated

The template is generated ๐ŸŽ‰

Step 2: Migrating to CDK ๐Ÿš€

With our CloudFormation template as our blueprint, we go into CDK territory. We download the template and execute the command:

cdk migrate --stack-name cw-book-example-app --from-path ./cw-book-example-app-1707551700519.yaml --language typescript

The migration is quite fast. But there is a warning about missing resources. After checking the README.md file you can see a few errors on resource options.

From that point, we are not 100% sure what is going wrong but it looks quite right, so we continue.

The CDK Project

Let's check the CDK project. This is the file structure of the new project:

It created a CDK App in bin/cw-book-example-app.ts

It also created one file which includes all stacks in lib/cw-book-example-app-stack.ts.

The process of the migration is done. We haven't deployed it because we knew we wouldn't continue using it for our actual production system.

The Good, The Bad, and The Techy โœจ

Pros ๐ŸŸข

Good to get started

If you have never worked with Infrastructure as Code and built your AWS infrastructure in a Click-Ops way it can be quite intimidating to get started.

With this approach, you get an idea of your resources and how to provision them with CloudFormation and CDK.

Finding resources in your account

With code, it can be easier to understand which resources you have. You can also use the resource manager for that (which is much easier IMO) but this could be a benefit.

Cons ๐Ÿ”ด

There are some parts we don't like and think can be improved.

Naming

Naming is hard; we get that. But after migrating we have several constructs with these names:

  • lambdaFunction00cwhotfdev...
  • apiGatewayRestApi00sq9n...
  • codeStarConnections...

It is not possible to work with that.

One Stack

The migration command created one stack. Designing stacks while developing a new application is not easy. If you want to separate stacks you should have thought about that while importing the resources into CloudFormation.

For customers using this feature, this will not be feasible.

One File

The migrate command created one file with 2393 lines of code.

That's too much and very unclear.

L1 Constructs

The beauty in CDK comes with L2 constructs. Not caring about IAM, and having additional APIs like grantRead makes working with CDK so much easier.

The migrate command migrates on the lowest level possible and gives us L1 constructs which are CloudFormation constructs. Just look at my Step Function definition. ๐Ÿฅฒ

This is probably not something that is easily maintained.

Lambda Code

We rely a lot on Lambda functions. We were already curious about how the IAC generator will get the Lambda execution code. But it simply doesn't. Which makes sense, but also defeats the purpose a bit.

Once you want to deploy you need to define the buckets where the code resides.

For each Lambda function, you need to declare the bucket.

Summary

In summary, migrating your resources from your AWS account to CloudFormation and then to CDK is possible.

What's not possible: using the generated code without making adaptions. At least not for a serious, long-term project.

We think it can be a great starting point. But it can also be quite complex to iterate from that point on.

The AWS Post mentions VPCs a lot. We do understand that understanding all resources that were manually created within that VPC is hard.

It is a great starting point. However, we won't use it for creating the CDK code for the CloudWatch Book. We will create it manually with all best practices in mind.

Since you are still dependent on CloudFormation, designing stacks is not possible with the automated approach.

We do understand that building such a kind of feature can be hard to get right for everybody. We don't see a lot of use cases here right at this moment.

Sprinkling in some LLMs together with natural language generation could improve a lot of things here! Maybe that is for the future. ๐Ÿ”ฎ

Have you used it?

What's your take on the IAC Generator? Have you encountered similar challenges, or did you find a workaround that made the process smoother?

Share your experiences, tips, or questions with us.

Your insights not only enrich our community but also pave the way for more informed, practical explorations in the future.

Until our next adventure, keep experimenting, learning, and sharing within the AWS cosmos. See you in two weeks for another dose of cloud wizardry! ๐ŸŒˆ

Best,
Sandro & Tobi

P.S. Curious about another AWS feature? Let us know, and it might be the star of our next hands-on guide! โœŒ๏ธ

Tobi & Sandro

our goal is to simplify AWS & Cloud Learning for everybody. You don't need expensive certifications to build on AWS!

Dr.-Otto-BรถรŸner-Weg 7a, Ottobrunn, Bavaria 85521 ยท Unsubscribe ยท Preferencesโ€‹

AWS for the Real World

by Tobi & Sandro

Join our community of over 8,800 readers delving into AWS. We highlight real-world best practices through easy-to-understand visualizations and one-pagers. Expect a fresh newsletter edition every two weeks.

Read more from AWS for the Real World

โŒ› Reading time: 6.3 minutes ๐ŸŽ“ Main Learning: Observability Aggregation with OAM ๐Ÿ‘จ๐Ÿฝ๐Ÿ’ป GitHub Code ๐Ÿ“ Blog Post Hey Reader ๐Ÿ‘‹๐Ÿฝ Ever tried setting up an AWS Landing Zone? If you have, you know it's not easy. AWS recommends using a separate account just for monitoring all your log data. We're here to introduce the AWS Observability Access Manager (OAM), designed to make this task easier. Previously, we couldn't use OAM effectively due to a major limitation, but that's changed. Interested in diving...

11 days agoย โ€ขย 3 min read

Hey Reader Imagine diving into AWS without a solid foundation on observability. It will lead to inefficiencies, wasted resources and money, and countless hours of troubleshooting. ๐Ÿ› The CloudWatch Book is here to guide you, ensuring you understand not just the how, but most importantly the why behind using AWS effectively so you'll work with ease, turning even complex infrastructure into a well-oiled machine. ๐Ÿค– In this issue, we want to show you the first sample chapter about Logs and...

22 days agoย โ€ขย 1 min read

Hey Reader! Hope you're having an awesome week! This week's edition is about implementing Step Function Workflows. One major use-case that Step Functions enable is the Wait for Callback pattern. Code Included ๐Ÿ‘จ๐Ÿฝ๐Ÿ’ป This edition comes with a special bonus - for the first time, we're including a GitHub repository just for you! Clone the repo and deploy it in your AWS Account. โšก Step Functions Orchestrate Workflows What are even Step Functions? Step Functions orchestrate workflows with different...

about 1 month agoย โ€ขย 5 min read
Share this post