AWS with Terraform Tutorial: Terraform Backends (19)

· 10 min read · Terraform & OpenTofu Tutorials

How to configure a Terraform backend in AWS S3 #

Using the Terraform and OpenTofu s3 backend to keep the state in a versioned, encrypted bucket that a whole team (and a CI/CD pipeline) can share safely.

Welcome to our tutorial series about Terraform or OpenTofu on AWS. Terraform remembers what it created in a state file, terraform.tfstate, which by default is saved next to the code. That is fine for learning, but it becomes a problem as soon as the infrastructure is important:

Terraform is an Infrastructure as Code (IaC) tool used to provision and manage infrastructure. It helps define and deploy resources across various cloud providers using code, making it easier to maintain and scale infrastructure.Terraform is an Infrastructure as Code (IaC) tool used to provision and manage infrastructure. It helps define and deploy resources across various cloud providers using code, making it easier to maintain and scale infrastructure.
Terraform
Basics
Terraform...
AWS is the world’s leading cloud platform, it is used by a wide range of organizations, from startups to large enterprises, to power their online businesses. AWS offers a wide range of services, including computing, storage, database, networking, analytics, machine learning, and artificial intelligence.AWS is the world’s leading cloud platform, it is used by a wide range of organizations, from startups to large enterprises, to power their online businesses. AWS offers a wide range of services, including computing, storage, database, networking, analytics, machine learning, and artificial intelligence.
AWS
Basics
AWS...
The Terraform official AWS provider acts as an abstraction layer that lets Terraform configurations written in HCL define AWS services and infrastructure using code (IaC). Internally Terraform and the AWS provider handle authentication, and make the necessary AWS API calls to query, create, modify, and destroy the resources.The Terraform official AWS provider acts as an abstraction layer that lets Terraform configurations written in HCL define AWS services and infrastructure using code (IaC). Internally Terraform and the AWS provider handle authentication, and make the necessary AWS API calls to query, create, modify, and destroy the resources.
Terraform
AWS Provider
Terraform...
How to Create, and Manage AWS VPCs with TerraformHow to Create, and Manage AWS VPCs with Terraform
AWS VPC
AWS VPC
How to configure and use the Terraform aws_subnet resource block to create and manage AWS Subnets inside a VPC.How to configure and use the Terraform aws_subnet resource block to create and manage AWS Subnets inside a VPC.
AWS Subnets
AWS Subnets
How to configure and use the Terraform aws_internet_gateway resource block to create and manage AWS Internet Gateway inside a VPC to enable Internet access to and from instances. How to configure and use the Terraform aws_internet_gateway resource block to create and manage AWS Internet Gateway inside a VPC to enable Internet access to and from instances.
AWS Internet
Gateway
AWS Internet...
How to configure and use the Terraform aws_nat_gateway and aws_eip resource blocks to create and manage AWS NAT Gateway and its corresponding Public IPs inside each availability zone to enable Internet access from instances in private subnets.How to configure and use the Terraform aws_nat_gateway and aws_eip resource blocks to create and manage AWS NAT Gateway and its corresponding Public IPs inside each availability zone to enable Internet access from instances in private subnets.
AWS NAT
Gateway
AWS NAT...
How to configure and use the Terraform aws_route_table, aws_route, and aws_main_route_table_association resource blocks to create and manage AWS Routing Tables.How to configure and use the Terraform aws_route_table, aws_route, and aws_main_route_table_association resource blocks to create and manage AWS Routing Tables.
AWS Routing
Tables
AWS Routing...
How to configure and use the Terraform aws_security_group and aws_security_group_rule resource blocks to create and manage AWS Security Groups and secure the infrastructure.How to configure and use the Terraform aws_security_group and aws_security_group_rule resource blocks to create and manage AWS Security Groups and secure the infrastructure.
AWS Security
Groups
AWS Security...
How to configure and use the Terraform aws_key_pair resource block to create and manage AWS Key Pairs for performing SSH Public Key Authentication into EC2 instances.How to configure and use the Terraform aws_key_pair resource block to create and manage AWS Key Pairs for performing SSH Public Key Authentication into EC2 instances.
AWS Key
Pairs
AWS Key...
How to configure and use the Terraform aws_ami data source block to find and use AWS AMIs as templates (root volume snapshot with operating system and applications) for EC2 instances.How to configure and use the Terraform aws_ami data source block to find and use AWS AMIs as templates (root volume snapshot with operating system and applications) for EC2 instances.
AWS AMIs
AWS AMIs
Using the Terraform aws_instance resource block to configure, launch, and secure EC2 instances.Using the Terraform aws_instance resource block to configure, launch, and secure EC2 instances.
AWS EC2
Instances
AWS EC2...
AWS RDS
AWS RDS
Using the Terraform aws_route53_delegation_set, aws_route53_zone, and aws_route53_record resource blocks to configure DNS in AWS. Using the Terraform aws_route53_delegation_set, aws_route53_zone, and aws_route53_record resource blocks to configure DNS in AWS.
AWS Route 53
(DNS)
AWS Route 53...
Amazon EC2 Auto Scaling groups keep the right number of instances running, replace failed ones and scale with the load.Amazon EC2 Auto Scaling groups keep the right number of instances running, replace failed ones and scale with the load.
AWS Auto
Scaling
AWS Auto...
Elastic Load Balancing distributes the traffic between healthy instances. The tutorial creates an Application Load Balancer with HTTPS.Elastic Load Balancing distributes the traffic between healthy instances. The tutorial creates an Application Load Balancer with HTTPS.
AWS Load
Balancers
AWS Load...
This tutorial shows how to create infrastructure in AWS using Terraform and configure the operating system and applications using Ansible.This tutorial shows how to create infrastructure in AWS using Terraform and configure the operating system and applications using Ansible.
Terraform,
AWS & Ansible
Terraform,...
Modules package resources behind variables and outputs so they can be reused. The tutorial refactors the network into a module.Modules package resources behind variables and outputs so they can be reused. The tutorial refactors the network into a module.
Terraform
Modules
Terraform...
A backend stores the Terraform state. The tutorial uses an encrypted, versioned S3 bucket with state locking.A backend stores the Terraform state. The tutorial uses an encrypted, versioned S3 bucket with state locking.
Terraform
Backends
Terraform...
fmt, validate, TFLint, Trivy, Checkov, terraform-docs, Infracost and pre-commit: check the code before it reaches AWS.fmt, validate, TFLint, Trivy, Checkov, terraform-docs, Infracost and pre-commit: check the code before it reaches AWS.
Terraform
Tools
Terraform...
Run checks, plans and approved applies automatically with GitHub Actions and OIDC, without access keys.Run checks, plans and approved applies automatically with GitHub Actions and OIDC, without access keys.
Terraform
CI/CD
Terraform...
Select a tutorial section 
Select a tutorial sectio...
  • if the file is lost, Terraform no longer knows its resources,
  • two people (or a person and a pipeline) running tofu apply at the same time can corrupt it,
  • the state contains sensitive data (database passwords, keys) in plain text and must not be committed to Git.

A backend defines where the state is stored. This section moves it to Amazon S3.

Prerequisites #

Read the previous sections of the tutorial, listed in the series index at the end of this page. You need an AWS profile (ditwl_infradmin, see Terraform AWS Provider) with permissions to create an S3 bucket.

Terraform backends #

The backend is configured in the terraform block. The most used ones are:

Backend State stored in Locking
local (default) A file in the project directory Local file lock
s3 An Amazon S3 bucket S3 lock file (or DynamoDB)
gcs / azurerm Google Cloud Storage / Azure Blob Storage Yes
http, pg, kubernetes A REST endpoint (for example GitLab), PostgreSQL, a Kubernetes secret Depends on the backend
cloud HCP Terraform (Terraform Cloud) Yes

The S3 backend is the most common choice on AWS: it is cheap, durable (99.999999999%), supports versioning and encryption, and uses the same IAM permissions as the rest of the infrastructure.

Step 1: create the bucket #

There is a chicken-and-egg problem: the bucket that stores the state must exist before Terraform can use it, so it is created by a separate, small project (backend-bootstrap/) that keeps its own state locally. It is run once.

backend-bootstrap/main.tf
terraform {
  required_version = "> 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  profile = "ditwl_infradmin"
}

data "aws_caller_identity" "current" {}

locals {
  # Bucket names are global: the account number makes it unique
  bucket = "ditwl-tfstate-${data.aws_caller_identity.current.account_id}"
}

resource "aws_s3_bucket" "tfstate" {
  bucket = local.bucket

  # Deleting this bucket would delete the state of all the infrastructure
  lifecycle {
    prevent_destroy = true
  }

  tags = {
    Name = local.bucket
  }
}

# Keep every version of the state: it allows going back after a mistake
resource "aws_s3_bucket_versioning" "tfstate" {
  bucket = aws_s3_bucket.tfstate.id
  versioning_configuration {
    status = "Enabled"
  }
}

# Encrypt the state at rest
resource "aws_s3_bucket_server_side_encryption_configuration" "tfstate" {
  bucket = aws_s3_bucket.tfstate.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

# The state is never public
resource "aws_s3_bucket_public_access_block" "tfstate" {
  bucket                  = aws_s3_bucket.tfstate.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# Delete old versions after 90 days
resource "aws_s3_bucket_lifecycle_configuration" "tfstate" {
  bucket = aws_s3_bucket.tfstate.id

  rule {
    id     = "expire-old-versions"
    status = "Enabled"
    filter {}

    noncurrent_version_expiration {
      noncurrent_days = 90
    }
  }
}

# Only encrypted connections (HTTPS)
data "aws_iam_policy_document" "tfstate-tls" {
  statement {
    sid       = "DenyInsecureTransport"
    effect    = "Deny"
    actions   = ["s3:*"]
    resources = [aws_s3_bucket.tfstate.arn, "${aws_s3_bucket.tfstate.arn}/*"]

    principals {
      type        = "*"
      identifiers = ["*"]
    }

    condition {
      test     = "Bool"
      variable = "aws:SecureTransport"
      values   = ["false"]
    }
  }
}

resource "aws_s3_bucket_policy" "tfstate" {
  bucket = aws_s3_bucket.tfstate.id
  policy = data.aws_iam_policy_document.tfstate-tls.json

  depends_on = [aws_s3_bucket_public_access_block.tfstate]
}

output "bucket" {
  value = aws_s3_bucket.tfstate.bucket
}
$ cd backend-bootstrap
$ tofu init
$ tofu apply
...
Outputs:

bucket = "ditwl-tfstate-123456789012"

Step 2: configure the backend #

In the main project add the backend block (inside the same terraform block as required_providers):

providers.tf
terraform {
  required_version = "> 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket       = "ditwl-tfstate-123456789012"
    key          = "aws-tutorial/pro/terraform.tfstate"
    region       = "us-east-1"
    profile      = "ditwl_infradmin"
    encrypt      = true
    use_lockfile = true
  }
}
  • key is the path of the state file inside the bucket. Use one key per project and environment (aws-tutorial/pro/..., aws-tutorial/dev/...) to keep the states small and independent.
  • encrypt = true asks S3 to encrypt the object.
  • use_lockfile = true enables native state locking: the backend creates a terraform.tfstate.tflock object next to the state while a command that changes it runs, and S3 conditional writes guarantee that only one process gets it. No other service is needed.

Step 3: migrate the state #

$ tofu init -migrate-state
Initializing the backend...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "s3" backend. ...

  Enter a value: yes

Successfully configured the backend "s3"!

The state is now in S3. Verify it and keep the local copy until you are sure that everything works, then delete it (and make sure that *.tfstate* is in .gitignore):

$ aws s3 ls s3://ditwl-tfstate-123456789012/aws-tutorial/pro/ --profile ditwl_infradmin
$ tofu plan
$ tofu state list

Test the state locking #

Run tofu apply in two terminals at the same time. The second one waits and fails with a message like this one, which shows who holds the lock:

Error: Error acquiring the state lock

Lock Info:
  ID:        0d1b2c3d-...
  Operation: OperationTypeApply
  Who:       jruiz@laptop
  Created:   2026-10-05 10:35:12 UTC

If a process dies and leaves a stale lock, release it with tofu force-unlock <ID>, only after being sure that nobody is running.

Who can access the state #

The state may contain secrets, so access is controlled with IAM. This policy lets a user or pipeline use only the state of this project:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::ditwl-tfstate-123456789012",
      "Condition": { "StringLike": { "s3:prefix": ["aws-tutorial/pro/*"] } }
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::ditwl-tfstate-123456789012/aws-tutorial/pro/*"
    }
  ]
}

The lock file is stored under the same key prefix, so the policy covers it. Read-only users (for example, a pipeline that only runs tofu plan -lock=false) need just s3:GetObject and s3:ListBucket. To also encrypt the contents of the state before it is sent to S3, read How to Encrypt Terraform State with OpenTofu.

Locking with DynamoDB #

For versions without native locking, create a table with a LockID key and point the backend to it:

backend-bootstrap/main.tf
resource "aws_dynamodb_table" "tflock" {
  name         = "ditwl-tflock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}
providers.tf
  backend "s3" {
    # ... the same arguments ...
    dynamodb_table = "ditwl-tflock"
  }

The IAM policy of the users also needs dynamodb:GetItem, dynamodb:PutItem and dynamodb:DeleteItem on the table.

Partial configuration #

The backend block cannot use Terraform variables (OpenTofu allows variables and locals in recent versions). To avoid repeating values between environments, or to keep them out of the repository, leave them out of the block and pass them when initializing:

backend.hcl
bucket = "ditwl-tfstate-123456789012"
key    = "aws-tutorial/dev/terraform.tfstate"
region = "us-east-1"
$ tofu init -backend-config=backend.hcl

AWS S3 Cost #

A state file is a few kilobytes: storage, versions and requests cost a few cents a month at most. It is the cheapest insurance for your infrastructure.

Common Questions About Terraform Backends #

Can I store the state in Git? #

No. It contains secrets in plain text, has no locking and merging two versions of a state file is not possible.

What if I change the key or the bucket? #

Terraform sees an empty state and would try to create everything again. Change the configuration and run tofu init -migrate-state so that the existing state is copied to the new location.

Should I use workspaces for the environments? #

Workspaces keep several states for the same code and backend. Many teams prefer one directory (or one key) per environment because it is more explicit and avoids applying to the wrong one.

How can another project read the outputs of this one? #

With the terraform_remote_state data source, which only needs read access to the state, or better with a data source of the real resource (for example aws_vpc), which does not depend on the other project's state.

Next Steps #

The state is shared and protected. Continue with Terraform Tools to validate, lint and document the code.

#AWS #AWS S3 #Terraform #OpenTofu #Security #AWS Terraform Tutorial