profile

AWS for the Real World

πŸ’œ Terraform - The World's Most Used Iac Tool

Published 4 months agoΒ β€’Β 8 min read

πŸ‘€ This is not properly displayed? Read all of our issues online! πŸ’‘

Hi Reader πŸ‘‹πŸ½

In this issue, we will explore the basics of Terraform, an open-source Infrastructure-as-Code (IaC) tool, and how it can simplify and automate the provisioning and management of your cloud infrastructure.

Join us as we uncover the key concepts and best practices to help you get started with Terraform and unlock its full potential.

🚨 P.S.: If you make it through this issue, we'll farewell you with some fascinating facts about Terraform that could definitely blow your mind even more and two sweet discounts 🍬 you'll hopefully enjoy! πŸ’›

Next Project: The CloudWatch Book πŸ“•

First of all, some news from us. We decided to tackle the next project which will be πŸ₯ the CloudWatch Book. If you're interested in the project check out the link.

Without further ado, let's get going! 🌟

​

Infographic

Before jumping in, have a look at the most important things about Terraform in our infographic below.

​

A Little Reminder: What's IaC and why do we need it?

Before the cloud, changing the hardware your application runs on was difficult. But now, you can easily scale your application up or out, without any upfront costs.

You can provision infrastructure globally and delete it quickly when you no longer need it.

This has created a need for tools that can manage infrastructure resources programmatically.

With Infrastructure-as-Code tools, you can define all your infrastructure resources as code, alongside your application code.

Terraform stands out as one of the leading and highly adaptable IaC tools available today.

P.S.: If you want to delve deeper into the foundational aspects of IaC on AWS, we recommend checking out our previous blog posts about the history of IaC.

​

Declarative vs. Imperative Approaches

There are two types of IaC tools: declarative and imperative.

  • Declarative tools define the desired state of the infrastructure without specifying the steps. Examples include Terraform and CloudFormation, which use YAML or JSON.
  • Imperative tools list the commands needed to achieve the desired state. They often use Object-Oriented languages like TypeScript.

Declarative tools focus on defining the desired state of the infrastructure, while imperative tools explicitly list the commands needed to achieve that state. Each approach has its own benefits and considerations to take into account when choosing an IaC tool.

​

Key Concepts of Terraform

Terraform stands out as one of the most flexible options available. To fully grasp its capabilities and utilize it effectively, it's important to familiarize ourselves with its key concepts.

Multi-Platform Support via Providers

Terraform offers support for a wide range of cloud providers and services through its provider plugins. In addition to AWS, it also supports GCP, Azure, and other services like NewRelic.

This flexibility allows you to build a multi-cloud or multi-platform environment without the need for multiple tools.

Remote Tracking of Infrastructure State

One of the key features of Terraform is its ability to keep track of the state of your infrastructure. Whenever you make changes to your infrastructure by creating, updating, or deleting resources, Terraform saves the current state.

This means that Terraform remembers which resources are currently created and their configurations. This enables Terraform to determine the necessary changes required to reach the desired state, ensuring consistency and accuracy.

The infrastructure state can be stored remotely, such as in an S3 bucket. This allows you to run Terraform from different locations while maintaining the same results.

Abstractions and Blueprints via Modules

Terraform modules provide a way to create blueprints for your infrastructure and encapsulate configurations.

This allows you to create complex infrastructure setups hidden behind an abstraction layer. Modules also enable you to reuse these setups in different environments.

Previewing Changes

Terraform offers two important commands: plan and apply.

The plan command previews the changes that Terraform would execute. It helps prevent unintended changes and allows for safer and more controlled updates. The previewed plan can be saved locally and executed as-is.

The apply command executes all the changes indicated by Terraform to achieve the desired state of the infrastructure.

​

More Features

Terraform comes with a lot of features. Let's explore some of the most important ones!

Modules

Terraform modules provide a way to encapsulate and reuse infrastructure configurations, simplifying the management of complex environments and promoting the adoption of best practices.

In the example graphic below, we have two separate modules that define blueprints for different components:

  • Module 1: Frontend Distribution - Utilizes CloudFront, IAM, and S3 to build a frontend distribution.
  • Module 2: ECS-based Service - Uses Fargate, IAM, and an Elastic Load Balancer to create an ECS-based service.

By modularizing these configurations, we can easily reuse them, promote code sharing, and ensure adherence to best practices.

Resource Outputs

Each cloud platform has its own unique way of identifying resources, which is important for coupling components together.

For instance, AWS uses Amazon Resource Names (ARNs) to identify resources. If you want to grant access to a DynamoDB table while following the principle of least privilege (granting only necessary permissions for a user or service to perform its tasks), you need to include the corresponding ARN in the IAM policy.

To address this, Terraform provides outputs. You can reference the outputs of resources that are only available after the resource is created before Terraform executes.

Terraform determines resource dependencies by creating a Dependency Graph. This graph helps Terraform understand which resources rely on others and in what order they need to be created.

Let’s take a look at an example:

  1. The IAM policy references the ARN of the DynamoDB table.
  2. The IAM role relies on the IAM policy.
  3. The Lambda function needs to have the IAM role attached.

In this case, our dependency graph starts with the Lambda function and ends with the DynamoDB table. The table needs to be created first, and the Lambda function should be created last.

Terraform automatically detects and handles these dependencies, ensuring the correct order of resource creation.

HCL and Expression Functions

Terraform files utilize HashiCorp’s configuration language HCL (HashiCorp Configuration Language) for defining their structure. This language incorporates expression functions that provide a range of powerful capabilities for creating resources and values dynamically, manipulating data structures, and performing simple or complex calculations within your Terraform code.

Some examples of included expression functions are:

  1. regex(string, pattern) : This function returns true if the given string matches the specified regular expression pattern.
  2. join(separator, list) : This function returns a string that is the concatenation of the elements in the given list, separated by the specified separator.

The Terraform documentation contains a comprehensive list of all available expression functions.

By using expression functions, you can enhance the dynamism and flexibility of your Terraform code, thereby reducing the need for repetitive or manual tasks.

Meta Arguments

In Terraform, a resource block typically corresponds to a single infrastructure resource. However, there are cases where we need to create multiple objects that share similar characteristics without duplicating the code.

To address this properly, we can utilize meta arguments such as for_each .

Dynamic Blocks

The for_each function allows us to dynamically generate multiple resources. This functionality can also be applied to blocks within our resources by combining them with dynamic blocks.

For instance, we can attach multiple EBS volumes to an EC2 instance based on a variable:

In this example, we provide a list of complex objects (a map in Terraform) to configure our EC2 instance. We iterate over each value using the for_each function and assign the corresponding field values to the EBS configuration.

We also utilize the lookup expression, which retrieves the value for a given key (in this case, the IOPS value via "iops") or returns a fallback value of null.

If an empty list is passed, the dynamic block will be empty, resulting in no EBS volumes being added.

Managing Multiple Environments

Terraform provides a convenient way to efficiently handle multiple environments, such as development, staging, and production, by utilizing its module-based approach.

To create multiple environments, you have two options: utilizing different workspaces or employing separate state files. This approach not only simplifies the deployment process but also allows for the seamless configuration of various AWS regions. By duplicating the main region, each region can receive an exact replica of the original setup.

​

Valuable Resources

Useful References for Terraform Terraform is a complex and extensive ecosystem that thrives on community collaboration. Here are some valuable resources to explore:

  1. πŸ“š Module Registries: A wide range of module blueprints in both private and public registries. These registries provide a wealth of pre-built infrastructure components that can be easily integrated into your projects.
  2. πŸ› οΈ Enhanced Tools: Additional tools that complement and enhance the functionality of Terraform. These tools offer advanced features and capabilities to streamline your infrastructure provisioning process. One great example is infracost, which helps to know the costs of cloud resources even before launching them. πŸ’Έ
  3. πŸ‘₯ Community Resources: Tap into the vast knowledge and experience of the Terraform community via resources, including forums, documentation, tutorials, and case studies.

For a comprehensive list of curated resources, visit the awesome-terraform πŸ”— repository on Github. This repository serves as a centralized hub for discovering high-quality tools, modules, and community contributions.

​

Other Great IaC Tools

Besides Terraform, there are many other IaC tools that are worth exploring:

  1. AWS CloudFormation: An AWS-native IaC tool developed by Amazon. It uses YAML files to define resources and manages them in a CloudFormation Stack. It tracks the state of resources and can perform rollbacks during updates.
  2. AWS Cloud Development Kit: An abstraction layer on top of AWS CloudFormation. It supports multiple programming languages and offers high-level abstractions for infrastructure components. CDK deploys infrastructure using AWS CloudFormation.
  3. Pulumi: An open-source IaC platform that supports multiple cloud providers, including AWS, Azure, and Google Cloud Platform. It allows developers to use their preferred programming language and offers features like infrastructure testing and rollbacks.
  4. Serverless Framework: A declarative IaC framework that focuses on serverless architectures, particularly AWS Lambda. SF provides a high level of abstraction and simplifies the configuration of serverless components like AWS API Gateway. It uses CloudFormation to manage the infrastructure.

​

More Mindblowing Facts

As promised, there's even more to know.

  • 🌟 Terraform is platform-agnostic: While many people know that Terraform can manage a variety of service providers, it's less known that it's completely platform-agnostic. This means you can use it to manage a diverse set of infrastructures like public clouds, and private clouds, or directly with software APIs on your local machine.
    ​
  • πŸ”Œ Terraform uses a plugin-based architecture: This means that it's not just limited to the providers that HashiCorp (the company behind Terraform) supports. If you have a service that Terraform doesn't support, you can write your own provider plugin.
    ​
  • πŸ› οΈ Terraform isn't just for infrastructure: While Terraform is often used for spinning up servers or containers, it can also manage higher-level components such as DNS entries, email provider settings, or page rules on Cloudflare. This makes it a versatile tool for a wide range of use cases.

After working with Terraform for five consecutive years, we love it even more than in the beginning!

If you haven't started to use an Infrastructure-as-Code tool yet: today is the day and Terraform will be an awesome choice! πŸš€

​

Beyond AWS Fundamentals

This week, we're introducing Yan Cui, a well-known AWS Serverless Hero and the brilliant mind behind the Production-Ready Serverless* class as well as the books Testing Serverless Architectures* & The AppSync Masterclass*.

Yan also has a podcast realworldserverless where our one and only Sandro Volpicella just recently talked about AWS Fundamentals and our upcoming project, the CloudWatch Book. Watch for an upcoming episode.

With his expertise, clients have achieved remarkable results by accelerating their projects and achieving more with less effort. If you are eager to enhance your AWS skills and gain insights into building exceptional serverless applications, make sure to join his newsletter! πŸ’›

You'll receive valuable tips, step-by-step tutorials, and practical advice, all conveniently delivered straight to your inbox every week.

πŸŽ‰ Bonus #1: if you use the code HALLOWEEN23 in the checkout, you'll get a 30% discount - applicable for every product by Yan! πŸŽƒ If you're reading this newsletter issue well after Halloween, you can also use AWSFUNDAMENTALS to get a 15% discount!

🎁 Bonus #2: with the code NEWSLETTER15 you'll get a 15% discount for all of our AWS Fundamentals products πŸ™

* if you purchase one of Yan's books or courses through our links, we'll receive a small portion of the revenue which helps to keep this newsletter and other content free πŸ’›

​

That's all for today!

We wish you all the best and a great start to the week!

Tobi & Sandro πŸ‘‹

​

P.S.: If you've liked this issue and you're interested in learning more about Terraform, be sure to check out our starter guide which also includes hands-on examples! πŸ§‘β€πŸ’» πŸ‘©β€πŸ’»


Still hungry for AWS content? Have a look at our blog! πŸ“š ↓

AWS for the Real World

by Tobi & Sandro

Join our community of over 8,800 readers delving into AWS. We highlight real-world best practices through easy-to-understand visualizations and one-pagers. Expect a fresh newsletter edition every two weeks.

Read more from AWS for the Real World

Hi Reader πŸ‘‹πŸ½ This week, we're diving headfirst into the deep end of AWS's pool of innovations with the IAC Generator β€” a tool that's been causing quite the buzz. The promise AWS gives us is quite bold: Generate AWS CloudFormation templates and AWS CDK apps for existing AWS resources in minutes. But does it really work? Let's find out together. This will be a hands-on πŸ‘·πŸ½β€β™€οΈ issue. For the impatient, TL;DR, in a nutshell, πŸ₯œ: Good for a start, but far away from usable in production. The...

6 days agoΒ β€’Β 5 min read

This is not properly displayed? Read all of our issues online! πŸ’‘ Hi Reader πŸ‘‹πŸ½ We've said it more than once and we still stand with it: AWS IAM is the most complete but still underlooked service out there. Overwhelmingly, engineers look at the way of least resistance via "Which permissions do I have to grant to make this work? πŸ€”" or directly using wildcards for actions and resources on all policies, instead of diving into all the aspects of the service and its possibilities. Nevertheless, it's...

21 days agoΒ β€’Β 4 min read

Hey Reader, amazing that you're interested in creating the CloudWatch Book together with us. This is the third CloudWatch Update. Coincidentally, Sandro (I'm writing this update) turned 30 last week πŸ₯³ In this update, we'll introduce you to our newly created web application: The GitHub Repository Tracker. While it is not the fanciest name it, the project will help you a lot in learning & understanding CloudWatch for the real world. We thought it would be much easier to show it in a video so...

29 days agoΒ β€’Β 6 min read
Share this post