If you’ve worked with Terraform long enough, you’ve inevitably hit the point where you need to mess with the state file. Maybe you’re migrating off a third-party module’s backend to your own S3 bucket. Maybe there’s a stale resource haunting your plan output. Whatever the reason, state operations are one of those things that feel high-stakes — because they are. One wrong move and Terraform loses track of your infrastructure.

I recently had to migrate a backend from a CloudPosse module to my own S3 bucket with a DynamoDB lock table, and I want to walk through exactly what I did, what went wrong, and how I fixed it.

What Even Is the State File?

Before we get into the migration, let’s make sure we’re on the same page. The Terraform state file (terraform.tfstate) is a JSON file that acts as Terraform’s source of truth. It maps your .tf configuration to real-world resources. When you run terraform plan, Terraform compares your code against what’s in the state file — not against what actually exists in AWS (or whatever provider you’re using).

You can inspect your current state with:

# Show all tracked resources in a human-readable format
terraform show

# List every resource Terraform is tracking
terraform state list

This distinction matters. If a resource exists in your state but not in your config, Terraform wants to destroy it. If it exists in your config but not in your state, Terraform wants to create it. Understanding this relationship is key to not accidentally blowing things up during a migration.

Why I Needed to Migrate

I was using a CloudPosse module that managed its own S3 backend. It worked fine, but I wanted more control — my own bucket, my own DynamoDB table for state locking, my own naming conventions. The kind of thing where you start with someone else’s opinionated setup and eventually outgrow it.

The challenge: I had existing infrastructure tracked in the old state file, and I needed to move that state to a new backend without Terraform thinking it needed to recreate everything.

Step 1: Back Up Everything

I cannot stress this enough. Before you touch anything, make a local copy of your state file.

# Pull the current state to a local file
terraform state pull > terraform.tfstate.backup

Keep this file somewhere safe. If anything goes sideways during the migration, this backup is your lifeline. I’ve seen people skip this step and then spend hours trying to reconstruct state by hand. Don’t be that person.

Step 2: Set Up the New Backend

First, I created my new S3 bucket and DynamoDB table manually (or via a separate Terraform config — the classic chicken-and-egg problem of state backends). Then I configured the new backend.

I used a .tfvars file to keep the backend config clean and reusable across environments:

# backend.tfvars
bucket         = "my-terraform-state-bucket"
key            = "infrastructure/terraform.tfstate"
region         = "us-east-1"
encrypt        = true
dynamodb_table = "terraform-state-locks"

Then initialized with the new backend:

terraform init -backend-config=./backend.tfvars

Terraform will detect that the backend configuration changed. If you’re lucky, it’ll offer to migrate the state for you automatically. Sometimes it does, sometimes it doesn’t — especially when you’re switching between fundamentally different backend types or module-managed backends.

In my case, the automatic migration didn’t work cleanly, so I went the manual route.

Step 3: Set Up Workspaces

If you use workspaces to separate environments (dev, staging, production), you need to create them in the new backend before pushing state:

terraform workspace new production || terraform workspace select production

The || is a nice trick — if the workspace already exists, it just selects it instead of failing. I do this for every environment I need to migrate.

Step 4: Push the Old State

Remember that backup from Step 1? This is where it comes in. With the correct workspace selected, push the old state into the new backend:

terraform state push ../path/to/terraform.tfstate.backup

This tells the new backend “hey, these are the resources we’re already managing.” After this, running terraform plan should ideally show no changes — meaning the state matches your config and your real infrastructure.

Warning: terraform state push will refuse to push if it detects a higher serial number in the remote state. This is a safety mechanism. If you need to force it (and you’re absolutely sure), you can use -force, but think twice before doing that.

Step 5: Clean Up Stale Resources

Here’s where things got interesting for me. After pushing the state and running terraform plan, I got errors referencing an old module that no longer existed in my configuration. The module had been removed from the code, but it was still hanging around in the state file like a ghost.

First, I found exactly which resources were causing problems:

terraform state list | grep old-module-name

This gave me a list of all state entries related to that module. Then I removed them one by one:

terraform state rm 'module.old-module-name'

Important: terraform state rm does NOT destroy the actual infrastructure. It just tells Terraform to stop tracking it. The real resources still exist in your cloud provider. This is exactly what you want when a module has been removed from your config but the underlying resources should stay (or will be managed differently going forward).

If you have a lot of resources to remove, you can script it:

terraform state list | grep old-module-name | while read resource; do
  terraform state rm "$resource"
done

Step 6: Verify and Apply

After cleaning up the state, run the plan again:

terraform plan

If everything went well, you should see a clean plan — or at least only the changes you actually intend to make. At that point, you can apply:

terraform apply

I’d avoid using --auto-approve here unless you’ve very carefully reviewed the plan. After a migration is exactly the wrong time to skip the review step.

Other State Commands Worth Knowing

Beyond migrations, there are a few state operations I reach for regularly:

Replacing a Resource

Sometimes a resource is in a bad state and you just need to recreate it:

terraform plan -replace="aws_instance.example"

This marks the resource for replacement on the next apply. It’s cleaner than the old taint workflow and gives you a chance to review the plan first.

Moving Resources Between State Entries

If you refactor your Terraform code (renaming a module, reorganizing resources), you can move state entries to match:

terraform state mv 'module.old_name' 'module.new_name'

This prevents Terraform from destroying and recreating resources just because you renamed something.

Importing Existing Resources

If you have infrastructure that was created manually (or by another tool) and you want Terraform to manage it:

terraform import aws_s3_bucket.my_bucket my-existing-bucket-name

This adds the resource to your state without modifying the actual infrastructure.

Things That Can Go Wrong

Since we’re being practical here, let me list the things I’ve seen bite people:

  1. Forgetting to back up state before migrating. If the migration goes wrong and you don’t have a backup, you’re reconstructing state by hand with terraform import for every single resource. It’s painful.

  2. State lock conflicts. If someone else is running Terraform while you’re migrating, you’ll get lock errors. Coordinate with your team. If you get stuck with a stale lock, you can force-unlock it:

    terraform force-unlock LOCK_ID

  3. Serial number mismatches on push. The state file has a serial number that increments with every change. If your local backup has a lower serial than what’s in the remote backend, state push will refuse. Usually means someone applied changes after you took your backup.

  4. Sensitive data in state. Remember that the state file contains sensitive values in plain text — database passwords, API keys, whatever you’ve put in your config. Treat state files like secrets. Encrypt your S3 bucket, restrict access, and never commit state files to git.

Wrapping Up

State management isn’t glamorous, but it’s one of those skills that separates someone who uses Terraform from someone who really understands it. The key takeaways:

  • Always back up before migrating — terraform state pull is your friend
  • Use -backend-config for clean, reusable backend configurations
  • state rm removes tracking, not infrastructure — know the difference
  • state mv saves you from unnecessary destroy/recreate cycles
  • Treat state files as sensitive — they contain your secrets in plain text

If you’re coming from my previous post on Terraform best practices, think of this as the companion piece. That one covers how to structure your projects and write good Terraform code. This one covers what to do when the state file needs surgery.