Terraform manages the infrastructure resources and deployment using the state file. By running the [.code]refresh[.code] command, you can update the state file with the actual infrastructure configuration.

As of Terraform version v0.15.4, the [.code]terraform refresh[.code] command was deprecated because its default behavior could be deemed unsafe if you have misconfigured credentials for any of your providers. 

In this blog, we will explore the [.code]terraform refresh[.code] command and how it works, and also discuss its limitations and alternatives through the use of practical hands-on examples.    

Disclaimer : All use cases of the Terraform [.code]refresh[.code] discussed here work similarly in OpenTofu, the open-source Terraform alternative. However, to keep it simple and familiar for DevOps engineers, we will use “Terraform [.code]refresh[.code]” as a catch-all term throughout this blog post.

What is Terraform Refresh

The [.code]terraform refresh[.code] command ensures that your state file reflects the current state of your infrastructure resources. 

Let’s say that the resources managed by the Terraform code are sometimes modified using a console, CLI, third-party software APIs, or scripts. This means that the current infrastructure configuration will not match your Terraform code, creating drift because configuration changes were made outside of the regular code-to-cloud CI/CD pipeline.

One way to resolve these drifts is by running [.code]refresh[.code], which updates the state file with the actual infrastructure configuration. However, this approach comes with several downsides and is generally not considered a best practice due to some of the reasons we’ll describe below. 

Before getting to that, here are some scenarios in which you might need to run the [.code]terraform refresh[.code] command.

• Syncing the state: As mentioned, when the state file is out of sync with the current infrastructure, you can run the [.code]refresh[.code] command to reconcile the differences between the state file and its actual state (drifts).
• Before running Terraform plan:  Running [.code]refresh[.code] before [.code]terraform plan[.code] ensures that the plan is based on the most recent resource configuration changes made in your cloud environment.

The syntax for the [.code]terraform refresh[.code] command is:

terraform refresh [options]

And the options could be any of the following:

• [.code]-state=<path>[.code]: Specifies the path to the state file. Unless specified, it is the default path to the terraform.tfstate file. 
• [.code]-state-out=<path>[.code]: Using this, you can define where you want to store your refreshed state file. If the value is not passed, it overwrites the existing state file.
• [.code]-lock=<true|false>[.code]: When the state is refreshed, it helps define whether the state lock should be acquired. The default value is ‘true’.
• [.code]-lock-timeout=<duration>[.code]: Determines the wait time for a state lock to be acquired. The default is ‘0s’ (no timeout).
• [.code]-backup=<path>[.code]: Used to pass the location path for a backup of the state file before it's overwritten. The default value is ‘.tfstate’backup'.

How does Terraform Refresh State Work

In Terraform CLI, when the actual configuration of your resources on cloud providers (such as AWS, GCP, or Azure) no longer matches the configuration defined in your Terraform configuration, it causes a drift.

In such a scenario, when you run [.code]terraform refresh[.code] command, it reconciles this difference between the desired infrastructure (your state file) and the current infrastructure (actual cloud configuration) to match the difference between them (a.k.a  drifts), by doing the following: 

• First, Terraform inspects the existing state file (terraform.tfstate), which contains the desired state of your infrastructure. While reading the state file, Terraform identifies the resources that need to be refreshed or updated as defined in your .tf files.
• Next, Terraform makes API calls to the providers, to retrieve the current state of the infrastructure resources, as they were defined in the Terraform code. 
• Once the configuration is fetched, Terraform compares the configuration in the terraform.tfstate file with the current state of your infrastructure. If there are any changes in the resource arguments, the file is refreshed and updated with the current state for all those resources. 

Two important things to note:

1. After running [.code]refresh[.code] it’s always a good idea to verify that the state file is updated using the [.code]terraform show[.code] command.
2. It’s important to keep in mind that the [.code]refresh[.code] command doesn’t actually make any changes to the current infrastructure, and only updates the state file.

Example Scenario

To better demonstrate how [.code]terraform refresh[.code] works, let’s go into a quick hands-on example of how it can be used to update your state file.

First, let’s define an AWS S3 bucket using the Terraform main.tf file.

provider "aws" {
  region     = "us-east-1"
}
resource "aws_s3_bucket" "terraform_state" {
  bucket = "env0-terraform-state-bucket"
  lifecycle {
    prevent_destroy = false
  }
  tags = {
    Name        = "Terraform State Bucket"
  }
}

Next, let’s run the [.code]terraform init[.code] and [.code]terraform apply[.code] commands to deploy the S3 bucket on AWS.

➜  env0 git:(main) ✗ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Installing hashicorp/aws v5.77.0...
- Installed hashicorp/aws v5.77.0 (signed by HashiCorp)
…
Terraform has been successfully initialized!
…
➜  env0 git:(main) ✗ terraform apply --auto-approve
  + create
Terraform will perform the following actions:
  # aws_s3_bucket.terraform_state will be created
  + resource "aws_s3_bucket" "terraform_state" {
      + bucket                      = "env0-terraform-state-bucket"
      + force_destroy               = false
      + tags                        = {
          + "Name" = "Terraform State Bucket"
        }
      + tags_all                    = {
          + "Name" = "Terraform State Bucket"
        }
…
    }
Plan: 1 to add, 0 to change, 0 to destroy.
aws_s3_bucket.terraform_state: Creating...
aws_s3_bucket.terraform_state: Creation complete after 5s [id=env0-terraform-state-bucket]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

To close the loop, you can verify that the bucket was created by visiting the AWS console. Navigate to ‘S3’ and search with the bucket name. Here, you’ll see an ‘env0-terraform-state-bucket’. 

With the bucket created, run the [.code]terraform show[.code] commands to view the current local state file with the resources managed using Terraform.

➜  env0 git:(main) ✗ terraform show      
# aws_s3_bucket.terraform_state:
resource "aws_s3_bucket" "terraform_state" {
    arn                         = "arn:aws:s3:::env0-terraform-state-bucket"
    bucket                      = "env0-terraform-state-bucket"
    force_destroy               = false
    tags                        = {
        "Name" = "Terraform State Bucket"
    }
    tags_all                    = {
        "Name" = "Terraform State Bucket"
    }
…

Now, in the AWS console, edit the bucket tags under ‘Properties.’ You can add an owner tag, which helps you identify the owner of this bucket. Here, it is ‘Saksham’.

This manual change creates a drift in the infrastructure, which means that the current Terraform state file is out of sync with the current AWS infrastructure. 

Now it’s time to run the [.code]terraform plan[.code] command to detect the drift changes in your infrastructure.

➜  env git:(main) ✗ expoterraform plan                                 
aws_s3_bucket.terraform_state: Refreshing state... [id=env0-terraform-state-bucket]
  ~ update in-place
  # aws_s3_bucket.terraform_state will be updated in-place
  ~ resource "aws_s3_bucket" "terraform_state" {
      ~ tags                        = {
            "Name"  = "Terraform State Bucket"
          - "Owner" = "Saksham" -> null
        }
      ~ tags_all                    = {
          - "Owner" = "Saksham" -> null
…
Plan: 0 to add, 1 to change, 0 to destroy.

The above [.code]plan[.code] output displays that the new tag – ‘Owner’ – was added to the S3 bucket outside of your Terraform configuration. If you [.code]apply[.code] your Terraform code, the tag will be removed from the bucket. 

This means that there is a drift and the Terraform configuration in your state file does not contain any ‘Owner’ tag.

You can update your state file by running the [.code]terraform refresh[.code] command.

➜  env0 git:(main) ✗ terraform refresh
aws_s3_bucket.terraform_state: Refreshing state... [id=env0-terraform-state-bucket]

Once the Terraform state file is refreshed, you can review the updated state file output using [.code]terraform show[.code].

➜  env0 git:(main) ✗ expoterraform show                                 
# aws_s3_bucket.terraform_state:
resource "aws_s3_bucket" "terraform_state" {
    arn                         = "arn:aws:s3:::env0-terraform-state-bucket"
    bucket                      = "env0-terraform-state-bucket"
    id                          = "env0-terraform-state-bucket"
    tags                        = {
        "Name"  = "Terraform State Bucket"
        "Owner" = "Saksham"
    }
    tags_all                    = {
        "Name"  = "Terraform State Bucket"
        "Owner" = "Saksham"
    }
…

In the above output, you can see that the S3 bucket configuration has been updated with the latest tags from the AWS in the state file. 

If you want to overwrite this change with your Terraform code configuration, you need to run the [.code]apply[.code] command. Otherwise, if you want to accept this change, add the tags in your Terraform code and run [.code]apply[.code] to deploy the changes to the cloud. 

Concerns with Terraform Refresh

As mentioned above, the [.code]terrafrom refresh[.code] command was deprecated because of unsafe behavior in the case of misconfigured providers. 

For instance, if you have misconfigured the provider credentials for one AWS account (A)  with another AWS account (B), the command could trick Terraform into updating the state file with changes from account (B) rather than account (A), which could lead to various issue such as the deletion of all the resources in the state file without any confirmation.

In addition to the above concerns, the [.code]refresh[.code] command has other inherent limitations, which include:

• Running [.code]refresh[.code] does not modify the configuration in the .tf files. In case there was a configuration change, you will need to manually update your Terraform management code. 
• The [.code]refresh[.code] command helps you detect drift by comparing the current and desired states. However, you still need to fix this drift to avoid resource misconfigurations or security issues. And fixing the drift doesn’t always mean reverting to the configuration in the state file, as some drift could be a result of an intended change (e.g., manual error fix or the result of other software behavior).
• When multiple team members work on a large infrastructure, continuously running [.code]terraform refresh[.code] just to fetch any manual changes made to the infrastructure is not really feasible, running the risk of creating merge conflicts and plenty of overheard.

Apply refresh-only: A (Slightly) Better Alternative:

Terraform version 0.15.4 introduced the [.code]-refresh-only[.code] flag to provide more control over the functionality of the [.code]refresh[.code] command. When you run this with the [.code]plan[.code] and [.code]apply[.code] commands, you will be greeted with an interactive prompt where you can review and confirm the detected changes.

For example, run the following command: 

terraform apply -refresh-only

This will allow you to review the changes in your current infrastructure before updating your state file with them. Once you have reviewed them, simply approve or reject these changes. 

Unlike the [.code]refresh[.code] command, using [.code]apply -refresh-only[.code] offers a better approach to handling drift. However, it is still far from being a comprehensive solution as it doesn’t solve the challenges of drift detection, nor does it help provide context for the drift. 

As a result, some drifts go unnoticed, and their reconciliation could end up causing more infrastructure chaos than remediation. Also, rolling back some of the drifts could cause a domino effect of issues, effectively removing necessary – albeit manual or third-party – changes.

Drift Detection with env0

With env0’s drift detection and cause analysis features, you do not need to worry about scheduling runs for [.code]plan[.code] or [.code]refresh[.code] to continuously monitor your infrastructure or identify potential drifts. Moreover, you will also have additional context to ensure that the drifts are reconciled without causing any unwanted cascading issues across your cloud infrastructure.

Let’s look at how you can identify and investigate drifts using env0:

Automated Drift Detection

Using env0, you can automate the process of drift detection in your infrastructure code by enabling the ‘Drift Detection’ in ‘Settings’ with a cron job. 

For example, schedule a drift detection run once every hour in your environment. This will run the scheduled [.code]plan[.code] command and analyze its output for any changes done outside your Terraform configuration or drifts. When any drift is detected, you can see the result as ‘drifted’ in the [.code]plan[.code] stage of your deployment

 This capability saves plenty of time and prevents any incidents due to neglected drift.

Using some of the platform's more advanced capabilities, you can even be preemptively aware of future drifts that might occur after applying the new [.code]plan[.code]. 

These, for example, could be a result of using dynamic variables, and they will appear as  <computed> line items in the env0 dashboard, enabling you to further investigate by going back to reviewing the plan.

Drift Cause 

In addition to the above, env0 platform also offers a unique Drift Cause feature which connects the dots between the state of the codified infrastructure and out-of-code audit logs to offer additional context about drifts, and enable team to:

• Identify who made the change, when, and how
• Understand the specific event or action responsible for the drift (e.g., automated or scripted procedure via CLI or API, or a human being via cloud provider interface)

Check out the video below to see it in action.

‍

Drift Monitoring and Alerting

env0 also provides a centralized dashboard that displays the ‘Drift Status’ for each infrastructure environment deployed using Terraform. You can also see more information on what percentage of environments are drifted or if these environments have active, inactivate, or failed deployments, like so:

Using this dashboard, teams can easily monitor multiple environments for drift simultaneously under one unified dashboard, improving team efficiency. Additionally, team members can quickly identify which environments are drifting from their desired Terraform configuration. 

Approval Policies

Related to the topic of drifts, env0 allows you to enable approval policies (e.g., using Open Policy Agent). These can be tailored to preemptively address common causes of drifts, which could be identified through the use of a dashboard and insights provided by Drift Cause. 

Leveraging these insights on what could typically go wrong,  platform and infrastructure teams can enhance scheduled deployments with relevant approval policies together, to create a framework for a smart auto-remediation.

Final Thoughts

By now, you should understand what Terraform [.code]refresh[.code] does and when you can use it for your use case. We did a deep dive into what happens when you run the [.code]refresh[.code] command and shared some practical examples. 

Even though [.code]terraform refresh[.code] is really helpful for larger teams, we learned about its limitations and how you can overcome them with env0’s drift detection and remediation capabilities.

Frequently Asked Questions

Q1. Does Terraform plan refresh the state?

Yes, Terraform refreshes the state and updates the state file with the most recent resource configuration of the current infrastructure. It ensures that the plan reflects the current state rather than the desired state before any changes are made.

Q2. How do I apply Terraform without refresh state?

You can apply the Terraform [.code]refresh[.code] by running the [.code]terraform apply -refresh=false[.code] command. It skips the refresh and applies the plan based on the existing state.

Q3. How does Terraform refresh state work?

Terraform refreshes its state by querying the infrastructure to detect any changes made outside of Terraform (manual changes, updates by other tools, etc.). This process ensures that the Terraform state file reflects the current state of the infrastructure.

Q4. What is the difference between Terraform refresh and Terraform import?

Terraform’s [.code]refresh[.code] command updates the state file by checking the actual infrastructure to reflect any changes made outside Terraform (e.g., manually via the AWS Console). It doesn’t create or modify resources, only syncs the state with the real world.

Terraform’s [.code]import[.code] command is used to bring existing resources into Terraform’s management. It associates an existing infrastructure resource with a Terraform resource block, allowing Terraform to manage it going forward. It updates the state file but doesn’t modify the resource itself.

‍