What is Terragrunt?
Terragrunt is a thin wrapper for Terraform that provides extra tools for keeping your Terraform configurations DRY (Don't Repeat Yourself). With Terragrunt, you can easily manage remote states and multiple environments. It also helps you keep your codebase clean and organized.
Why use Terragrunt?
There are several reasons to use Terragrunt over just using pure Terraform code. Below is a list of these and we will elaborate more under the Terragrunt Features section.
- DRY code and configurations
- Versioning and environment management
- Dependency management
- Hooks for custom actions
- Keep your remote state configuration DRY
- Keep your CLI flags DRY
- Execute Terraform commands on multiple modules at once
- Work with multiple AWS accounts
Getting Started with Terragrunt
Video Walk-through
βRequirements: A GitHub account (all the hands-on sections will utilize GitHubβs Codespaces so you wonβt need to install anything on your machine)
TL;DR: You can find the repo here.β
Installing Terragrunt
To install Terragrunt, download the binary for your operating system from the Releases Page and add it to your PATH. Alternatively, you could use a package manager as shown here. If you're following along with us with codespaces, you will have all the binaries already installed for you.
Basic Terragrunt Commands
Instead of running Terraform commands directly, you run the same commands with Terragrunt:
- terragrunt init -> equivalent to terraform init
- terragrunt plan -> equivalent to terraform plan
- terragrunt apply -> equivalent to terraform apply
- terragrunt output -> equivalent to terraform output
- terragrunt destroy -> equivalent to terraform destroy
These commands will call the corresponding Terraform commands, with Terragrunt performing additional logic before and after the Terraform calls.
Using Terragrunt for Infrastructure Management
Terragrunt can be used to manage your infrastructure configurations, plans, and Terraform backend. You can define your Terragrunt configuration in a terragrunt.hcl file, which allows you to reference specific versions of your Terraform modules and fill in variables specific to each environment. We will see an example later.
Terragrunt Features
Let's now elaborate more on the key features that Terragrunt offers:
- DRY code:
Terragrunt helps you avoid duplicating code by allowing you to define common input variables and environment-specific variables. - Infrastructure management:
Terragrunt simplifies the management of multiple environments by providing a clear separation between them. - Versioning:
Terragrunt allows you to reference specific versions of your Terraform modules for each environment, making it easier to manage and roll back changes. - Hooks:
Terragrunt supports hooks that can be used to perform actions before or after Terraform commands. - Keep your remote state configuration DRY:
βTerragrunt allows you to define your backend block once in a root terragrunt.hcl file and inherit it in all your Terraform environments. Terragrunt can also automatically create the remote state and locking resources (such as S3 buckets and DynamoDB tables) for you. - Keep your CLI flags DRY:
You can configure Terragrunt to pass specific CLI arguments for specific commands using an
β block in your terragrunt.hcl file. - Execute Terraform commands on multiple modules at once:
Instead of manually running [.code]terraform apply[.code] in each of the subfolders corresponding to different environments and waiting for them to complete, you can use Terragrunt with the [.code]run-all[.code] command to deploy multiple Terraform modules at once. - Work with multiple AWS accounts:
Terragrunt allows you to work with multiple AWS accounts by letting you specify the IAM role to assume for each account. You can use the [.code]--terragrunt-iam-role[.code] command line argument or the TERRAGRUNT_IAM_ROLE environment variable to tell Terragrunt which role to use. Terragrunt will then call the sts assume-role API on your behalf and expose the credentials it gets back as environment variables when running Terraform. This way, you can manage your infrastructure in different accounts without having to store your AWS credentials in plaintext on your hard drive, without having to manually call assume-role every time, and without having to modify your Terraform code or backend configuration
Terragrunt Workflow and Best Practices
To make the most of Terragrunt, follow these best practices:
- Structuring Configurations: Organize your Terraform code into modules and use Terragrunt to reference these modules with specific versions.
- Managing Dependencies: Use Terragrunt to manage dependencies between your infrastructure components.
- Using Terragrunt Hooks: Utilize hooks to perform actions before or after Terraform commands.
- Automation with CI/CD: Integrate Terragrunt with your CI/CD pipeline for automated infrastructure deployment.
Terragrunt Benefits and Drawbacks
While Terragrunt offers many benefits, it also has some drawbacks, such as:
- It adds an additional layer of complexity to your infrastructure management and may require more initial setup.
- It is also another tool to manage
- Doesn't work with Terraform Cloud
However, if you are using env0, you can make full use of Terragrunt's benefits because env0 is one of the few tools that support Terragrunt.
Yevgeniy Brikman, who is the co-founder of Gruntworks (the company that brought us Terragrunt), makes a great comparison between using Terraform workspaces, Git branches, and Terragrunt. He summed up his comparison with the table below:
Terragrunt Use Cases and Examples
Terragrunt can be used in various scenarios, such as managing infrastructure for different environments like development, staging, and production. In our example, we will see how to use Terragrunt to DRY out our Terraform configuration. We will first run everything with pure Terraform only then we will see how to improve our configuration using Terragrunt.
Furthermore, to keep things simple, we will only consider two environments: dev and prod.
Our WordPress Module
We will use an example WordPress application to showcase the difference between using pure Terraform only and using Terragrunt. This example was taken from this repo, but modified slightly to fit our needs. The WordPress Terraform module creates the following resources:
- A VPC
- 1 Public Subnet for the EC2 instance
- 2 Private Subnets for the RDS
- An Internet Gateway
- A route table
- Security groups for EC2 and RDS
- EC2 instance for the WordPress application
- RDS instance for the MySQL database for WordPress
You can check the Terraform code in our repo.
Terraform Only Scenario
Below is the folder structure when using Terraform only.
Under each environment folder, you can see that we're duplicating code in both the main.tf and the outputs.tf files. Even though we are using a terraform module structure, we still duplicate the code unnecessarily. Recall that using terraform modules is a great way for code reuse.
Below is the content of the main.tf file for the dev environment.
Below is the content of the main.tf file for the prod environment.
There are two main factors to notice between the two main.tf files:
- The backend configuration is different and it is quite error-prone to copy and paste the key in this configuration. You may accidentally use the prod state file in dev and vice versa. You may also override an existing state file. Variables are not allowed to be used in the backend configuration block.
- Some of the input variables to the aws_wordpress module are duplicated. In this example, it might not be a big deal, however, when you have multiple terraform modules you will start to see the difference.
Now let's take a look at the output variables in the outputs.tf file for the dev environment:
Notice that they are exactly the same. So again this is violating the DRY principle and it becomes worse when you have many other environments and applications.
Deploy with Terraform only
Follow the instructions below to deploy the WordPress application using pure Terraform code only.
In the Terraform_Only/environments/dev folder create a private/public key pair with an empty passphrase:
In order to create a remote backend to store Terraform state files, you will need to create an S3 bucket and a Dynamo DB table in AWS. This is a good guide.
Run these Terraform commands:
To deploy the production Wordpress application, run the same steps above but in the Terraform_Only/environments/prod folder.
Here is the output of the [.code]terraform apply[.code] command:
And going to the 'http://54.167.129.51' address shows you the WordPress setup screen:
Terragrunt Scenario
Now let's take a look at how Terragrunt can improve our Terraform code structure.
Below is the folder and file structure in our local file system, when using Terragrunt. Notice that we have a root terragrunt.hcl configuration file and a terragrunt.hcl configuration file per environment folder.
The Root Terragrunt Configuration File
Now let's take a look at the main or root Terragrunt configuration file called terragrunt.hcl just under the environments folder.
Notice how the [.code]key[.code] in the remote_state block is parameterized. Each environment folder will have its own key without us worrying about copying and pasting. So the dev environment's key will be:
βWhereas the prod environment's key will be:
The second thing to notice is that we define the input variables that are common to all our environments here. Once again this shows how we are following the DRY principle.
Now let's take a look at the dev Terragrunt configuration files under both the environments/dev and the environments/prod folders.
Below is the terragrunt.hcl file under the environments/dev folder.
and below is the terragrunt.hcl under the environments/prod folder.
In both the dev and prod Terragrunt configuration files you can see that we're including the inputs from the root Terragrunt config file using the [.code]find_in_parent_folders()[.code] function.
We're also including the input variables that are specific to each environment. This is as DRY as it gets.
The second thing to notice is the source attribute inside the terraform block. Here we are referencing the Terraform module Wordpress that exists two levels above inside the modules folder. It's also possible to reference a terraform module that lives in a git repository, which is actually a more realistic pattern. In this case, another benefit of using Terragrunt is that you can source different versions of all the Terraform modules for different environments. For example, the dev environment may be running on version v0.0.2 of our Wordpress module whereas the prod environment is still on version v0.0.1. Once the dev environment has been properly tested you can then upgrade the prod environment to version v0.0.2. With pure Terraform, you can't parameterize the source block rendering all environments running with the same version of the module unless you hard-code the version values.
Deploy with Terragrunt
Follow the instructions below to deploy the WordPress application using pure Terraform code only.
In the Terragrunt/environments/dev folder create a private/public key pair with an empty passphrase:
Then run the following Terragrunt commands:
Terragrunt sets the AWS remote backend for you. It will create an S3 bucket and a Dynamo DB table.
To deploy the production WordPress application, run the same steps above but in the Terragrunt/environments/prod folder.
Here is the output of the [.code]terragrunt apply[.code] command:
Notice the Terragrunt output is exactly the same as the previous Terraform only scenario except that we didn't need to define an outputs.tf file to output the variables from the module. Terragrunt did it for us for free.
And going to the URL shows you the same WordPress setup screen that we saw earlier.
Terragrunt Cache
The Terragrunt cache is a folder that Terragrunt creates in the current working directory to store the downloaded Terraform configurations, modules, providers, and backend settings. Terragrunt uses this cache to avoid downloading the same code multiple times and to speed up the execution of Terraform commands. You can safely delete this folder at any time and Terragrunt will recreate it as necessary. You can also change the location of this folder by setting the [.code]TERRAGRUNT_DOWNLOAD[.code] environment variable. Here is what it looks like
Cleanup and the run-all command
Terragrunt has a neat feature that allows you to run commands across multiple folders. In our case we will run the following command to destroy the dev and prod environments in parallel from within the Terragrunt/environments folder:
Below is the output showing both the dev and prod environments destroyed successfully.
Terragrunt with env0
As we've seen, Terragrunt offers several benefits over Terraform and helps to address some of the challenges inherent to Terraform implementations. Since Terragrunt is built on top of Terraform, it also benefits from env0 management.
When you use env0 in conjunction with Terragrunt, you get the following benefits:
- Automate your Terragrunt deployments in CI pipelines for Infrastructure as Code.
- Manage your Terragrunt variables centrally and securely.
- Integrate your Terragrunt-based environments with other environments managed by different IaC tools.
- Combine multiple Terragrunt deployments to create more sophisticated environments.
- Use [.code]Terragrunt run-all[.code] to execute commands on multiple modules efficiently and reliably.
- Manage your underlying Terraform state safely and easily.
- Increase the reusability and extensibility of your Terragrunt deployments.
Key Takeaways
Terragrunt is a powerful tool that enhances the Terraform experience by providing DRY code, versioning, dependency management, and hooks, among other features. By following best practices and leveraging Terragrunt's features, you can create a more efficient and maintainable infrastructure management process in large-scale deployments. Combine env0 with Terragrunt and you can unleash the full potential of your Infrastructure as Code strategy.
References
- https://terragrunt.gruntwork.io/docs/getting-started/quick-start/Β
- https://blog.gruntwork.io/terragrunt-how-to-keep-your-terraform-code-dry-and-maintainable-f61ae06959d8Β
- https://jhooq.com/terragrunt-guide/Β
- https://github.com/orgs/gruntwork-io/discussions/92Β
- https://blog.devops.dev/a-complete-overview-of-terragrunt-fbebb53fbd42Β
- https://itnext.io/structuring-terraform-project-using-terragrunt-part-i-4c6e936c4858Β
- https://github.com/gruntwork-io/terragrunt-infrastructure-live-exampleΒ
β
What is Terragrunt?
Terragrunt is a thin wrapper for Terraform that provides extra tools for keeping your Terraform configurations DRY (Don't Repeat Yourself). With Terragrunt, you can easily manage remote states and multiple environments. It also helps you keep your codebase clean and organized.
Why use Terragrunt?
There are several reasons to use Terragrunt over just using pure Terraform code. Below is a list of these and we will elaborate more under the Terragrunt Features section.
- DRY code and configurations
- Versioning and environment management
- Dependency management
- Hooks for custom actions
- Keep your remote state configuration DRY
- Keep your CLI flags DRY
- Execute Terraform commands on multiple modules at once
- Work with multiple AWS accounts
Getting Started with Terragrunt
Video Walk-through
βRequirements: A GitHub account (all the hands-on sections will utilize GitHubβs Codespaces so you wonβt need to install anything on your machine)
TL;DR: You can find the repo here.β
Installing Terragrunt
To install Terragrunt, download the binary for your operating system from the Releases Page and add it to your PATH. Alternatively, you could use a package manager as shown here. If you're following along with us with codespaces, you will have all the binaries already installed for you.
Basic Terragrunt Commands
Instead of running Terraform commands directly, you run the same commands with Terragrunt:
- terragrunt init -> equivalent to terraform init
- terragrunt plan -> equivalent to terraform plan
- terragrunt apply -> equivalent to terraform apply
- terragrunt output -> equivalent to terraform output
- terragrunt destroy -> equivalent to terraform destroy
These commands will call the corresponding Terraform commands, with Terragrunt performing additional logic before and after the Terraform calls.
Using Terragrunt for Infrastructure Management
Terragrunt can be used to manage your infrastructure configurations, plans, and Terraform backend. You can define your Terragrunt configuration in a terragrunt.hcl file, which allows you to reference specific versions of your Terraform modules and fill in variables specific to each environment. We will see an example later.
Terragrunt Features
Let's now elaborate more on the key features that Terragrunt offers:
- DRY code:
Terragrunt helps you avoid duplicating code by allowing you to define common input variables and environment-specific variables. - Infrastructure management:
Terragrunt simplifies the management of multiple environments by providing a clear separation between them. - Versioning:
Terragrunt allows you to reference specific versions of your Terraform modules for each environment, making it easier to manage and roll back changes. - Hooks:
Terragrunt supports hooks that can be used to perform actions before or after Terraform commands. - Keep your remote state configuration DRY:
βTerragrunt allows you to define your backend block once in a root terragrunt.hcl file and inherit it in all your Terraform environments. Terragrunt can also automatically create the remote state and locking resources (such as S3 buckets and DynamoDB tables) for you. - Keep your CLI flags DRY:
You can configure Terragrunt to pass specific CLI arguments for specific commands using an
β block in your terragrunt.hcl file. - Execute Terraform commands on multiple modules at once:
Instead of manually running [.code]terraform apply[.code] in each of the subfolders corresponding to different environments and waiting for them to complete, you can use Terragrunt with the [.code]run-all[.code] command to deploy multiple Terraform modules at once. - Work with multiple AWS accounts:
Terragrunt allows you to work with multiple AWS accounts by letting you specify the IAM role to assume for each account. You can use the [.code]--terragrunt-iam-role[.code] command line argument or the TERRAGRUNT_IAM_ROLE environment variable to tell Terragrunt which role to use. Terragrunt will then call the sts assume-role API on your behalf and expose the credentials it gets back as environment variables when running Terraform. This way, you can manage your infrastructure in different accounts without having to store your AWS credentials in plaintext on your hard drive, without having to manually call assume-role every time, and without having to modify your Terraform code or backend configuration
Terragrunt Workflow and Best Practices
To make the most of Terragrunt, follow these best practices:
- Structuring Configurations: Organize your Terraform code into modules and use Terragrunt to reference these modules with specific versions.
- Managing Dependencies: Use Terragrunt to manage dependencies between your infrastructure components.
- Using Terragrunt Hooks: Utilize hooks to perform actions before or after Terraform commands.
- Automation with CI/CD: Integrate Terragrunt with your CI/CD pipeline for automated infrastructure deployment.
Terragrunt Benefits and Drawbacks
While Terragrunt offers many benefits, it also has some drawbacks, such as:
- It adds an additional layer of complexity to your infrastructure management and may require more initial setup.
- It is also another tool to manage
- Doesn't work with Terraform Cloud
However, if you are using env0, you can make full use of Terragrunt's benefits because env0 is one of the few tools that support Terragrunt.
Yevgeniy Brikman, who is the co-founder of Gruntworks (the company that brought us Terragrunt), makes a great comparison between using Terraform workspaces, Git branches, and Terragrunt. He summed up his comparison with the table below:
Terragrunt Use Cases and Examples
Terragrunt can be used in various scenarios, such as managing infrastructure for different environments like development, staging, and production. In our example, we will see how to use Terragrunt to DRY out our Terraform configuration. We will first run everything with pure Terraform only then we will see how to improve our configuration using Terragrunt.
Furthermore, to keep things simple, we will only consider two environments: dev and prod.
Our WordPress Module
We will use an example WordPress application to showcase the difference between using pure Terraform only and using Terragrunt. This example was taken from this repo, but modified slightly to fit our needs. The WordPress Terraform module creates the following resources:
- A VPC
- 1 Public Subnet for the EC2 instance
- 2 Private Subnets for the RDS
- An Internet Gateway
- A route table
- Security groups for EC2 and RDS
- EC2 instance for the WordPress application
- RDS instance for the MySQL database for WordPress
You can check the Terraform code in our repo.
Terraform Only Scenario
Below is the folder structure when using Terraform only.
Under each environment folder, you can see that we're duplicating code in both the main.tf and the outputs.tf files. Even though we are using a terraform module structure, we still duplicate the code unnecessarily. Recall that using terraform modules is a great way for code reuse.
Below is the content of the main.tf file for the dev environment.
Below is the content of the main.tf file for the prod environment.
There are two main factors to notice between the two main.tf files:
- The backend configuration is different and it is quite error-prone to copy and paste the key in this configuration. You may accidentally use the prod state file in dev and vice versa. You may also override an existing state file. Variables are not allowed to be used in the backend configuration block.
- Some of the input variables to the aws_wordpress module are duplicated. In this example, it might not be a big deal, however, when you have multiple terraform modules you will start to see the difference.
Now let's take a look at the output variables in the outputs.tf file for the dev environment:
Notice that they are exactly the same. So again this is violating the DRY principle and it becomes worse when you have many other environments and applications.
Deploy with Terraform only
Follow the instructions below to deploy the WordPress application using pure Terraform code only.
In the Terraform_Only/environments/dev folder create a private/public key pair with an empty passphrase:
In order to create a remote backend to store Terraform state files, you will need to create an S3 bucket and a Dynamo DB table in AWS. This is a good guide.
Run these Terraform commands:
To deploy the production Wordpress application, run the same steps above but in the Terraform_Only/environments/prod folder.
Here is the output of the [.code]terraform apply[.code] command:
And going to the 'http://54.167.129.51' address shows you the WordPress setup screen:
Terragrunt Scenario
Now let's take a look at how Terragrunt can improve our Terraform code structure.
Below is the folder and file structure in our local file system, when using Terragrunt. Notice that we have a root terragrunt.hcl configuration file and a terragrunt.hcl configuration file per environment folder.
The Root Terragrunt Configuration File
Now let's take a look at the main or root Terragrunt configuration file called terragrunt.hcl just under the environments folder.
Notice how the [.code]key[.code] in the remote_state block is parameterized. Each environment folder will have its own key without us worrying about copying and pasting. So the dev environment's key will be:
βWhereas the prod environment's key will be:
The second thing to notice is that we define the input variables that are common to all our environments here. Once again this shows how we are following the DRY principle.
Now let's take a look at the dev Terragrunt configuration files under both the environments/dev and the environments/prod folders.
Below is the terragrunt.hcl file under the environments/dev folder.
and below is the terragrunt.hcl under the environments/prod folder.
In both the dev and prod Terragrunt configuration files you can see that we're including the inputs from the root Terragrunt config file using the [.code]find_in_parent_folders()[.code] function.
We're also including the input variables that are specific to each environment. This is as DRY as it gets.
The second thing to notice is the source attribute inside the terraform block. Here we are referencing the Terraform module Wordpress that exists two levels above inside the modules folder. It's also possible to reference a terraform module that lives in a git repository, which is actually a more realistic pattern. In this case, another benefit of using Terragrunt is that you can source different versions of all the Terraform modules for different environments. For example, the dev environment may be running on version v0.0.2 of our Wordpress module whereas the prod environment is still on version v0.0.1. Once the dev environment has been properly tested you can then upgrade the prod environment to version v0.0.2. With pure Terraform, you can't parameterize the source block rendering all environments running with the same version of the module unless you hard-code the version values.
Deploy with Terragrunt
Follow the instructions below to deploy the WordPress application using pure Terraform code only.
In the Terragrunt/environments/dev folder create a private/public key pair with an empty passphrase:
Then run the following Terragrunt commands:
Terragrunt sets the AWS remote backend for you. It will create an S3 bucket and a Dynamo DB table.
To deploy the production WordPress application, run the same steps above but in the Terragrunt/environments/prod folder.
Here is the output of the [.code]terragrunt apply[.code] command:
Notice the Terragrunt output is exactly the same as the previous Terraform only scenario except that we didn't need to define an outputs.tf file to output the variables from the module. Terragrunt did it for us for free.
And going to the URL shows you the same WordPress setup screen that we saw earlier.
Terragrunt Cache
The Terragrunt cache is a folder that Terragrunt creates in the current working directory to store the downloaded Terraform configurations, modules, providers, and backend settings. Terragrunt uses this cache to avoid downloading the same code multiple times and to speed up the execution of Terraform commands. You can safely delete this folder at any time and Terragrunt will recreate it as necessary. You can also change the location of this folder by setting the [.code]TERRAGRUNT_DOWNLOAD[.code] environment variable. Here is what it looks like
Cleanup and the run-all command
Terragrunt has a neat feature that allows you to run commands across multiple folders. In our case we will run the following command to destroy the dev and prod environments in parallel from within the Terragrunt/environments folder:
Below is the output showing both the dev and prod environments destroyed successfully.
Terragrunt with env0
As we've seen, Terragrunt offers several benefits over Terraform and helps to address some of the challenges inherent to Terraform implementations. Since Terragrunt is built on top of Terraform, it also benefits from env0 management.
When you use env0 in conjunction with Terragrunt, you get the following benefits:
- Automate your Terragrunt deployments in CI pipelines for Infrastructure as Code.
- Manage your Terragrunt variables centrally and securely.
- Integrate your Terragrunt-based environments with other environments managed by different IaC tools.
- Combine multiple Terragrunt deployments to create more sophisticated environments.
- Use [.code]Terragrunt run-all[.code] to execute commands on multiple modules efficiently and reliably.
- Manage your underlying Terraform state safely and easily.
- Increase the reusability and extensibility of your Terragrunt deployments.
Key Takeaways
Terragrunt is a powerful tool that enhances the Terraform experience by providing DRY code, versioning, dependency management, and hooks, among other features. By following best practices and leveraging Terragrunt's features, you can create a more efficient and maintainable infrastructure management process in large-scale deployments. Combine env0 with Terragrunt and you can unleash the full potential of your Infrastructure as Code strategy.
References
- https://terragrunt.gruntwork.io/docs/getting-started/quick-start/Β
- https://blog.gruntwork.io/terragrunt-how-to-keep-your-terraform-code-dry-and-maintainable-f61ae06959d8Β
- https://jhooq.com/terragrunt-guide/Β
- https://github.com/orgs/gruntwork-io/discussions/92Β
- https://blog.devops.dev/a-complete-overview-of-terragrunt-fbebb53fbd42Β
- https://itnext.io/structuring-terraform-project-using-terragrunt-part-i-4c6e936c4858Β
- https://github.com/gruntwork-io/terragrunt-infrastructure-live-exampleΒ
β