AWS with Terraform Tutorial: Terraform Modules (18)
How to create Terraform modules for AWS infrastructure #
Using Terraform and OpenTofu modules to turn the VPC, subnets, gateways and routing tables of the tutorial into a reusable building block.
Welcome to our tutorial series about Terraform or OpenTofu on AWS. Every section so far added resources to a single file, terraform-aws-tutorial.tf. That is the best way to learn, but it does not scale: the file grows, the same patterns are copied between projects and one mistake can affect everything. A module groups resources behind a small interface of inputs (variables) and outputs, so they can be reused, tested and versioned.
Prerequisites #
Read the previous sections of the tutorial, listed in the series index at the end of this page. This section refactors the network created in AWS VPC, AWS Subnets, AWS Internet Gateway, AWS NAT Gateway and AWS Routing Tables.
Terraform modules #
A module is a directory with Terraform files. Every Terraform project is already a module, the root module, and it can call other modules (child modules) with a module block:
- Inputs:
variableblocks, set as arguments of themoduleblock. - Outputs:
outputblocks, read asmodule.<name>.<output>. - Sources: a local path (
./modules/network), the public registry (terraform-aws-modules/vpc/aws), a Git repository (git::https://github.com/org/repo.git//modules/network?ref=v1.2.0) or an S3 bucket.
Good modules do one thing, expose few and well-documented variables, hide implementation details and do not configure providers (the provider is inherited from the root module).
The new structure of the project #
.
├── providers.tf # terraform {} and provider "aws" {}
├── main.tf # calls the module and the rest of the resources
├── moved.tf # tells Terraform that the resources changed address
└── modules
└── network
├── main.tf
├── variables.tf
└── outputs.tf
The goal is the same infrastructure: after the refactor, tofu plan must not create, change or destroy anything.
The network module #
Input variables #
variable "name_prefix" {
type = string
description = "Prefix used in the names of the resources, for example ditwl"
}
variable "vpc_name" {
type = string
description = "Name tag of the VPC"
}
variable "vpc_cidr_block" {
type = string
description = "IPv4 CIDR block of the VPC, for example 172.21.0.0/19"
validation {
condition = can(cidrnetmask(var.vpc_cidr_block))
error_message = "vpc_cidr_block must be a valid IPv4 CIDR block, for example 172.21.0.0/19."
}
}
variable "subnets" {
type = map(object({
cidr_block = string
availability_zone = string
public = bool
}))
description = "Subnets by name. Private subnets need a public subnet in the same Availability Zone for the NAT Gateway."
validation {
condition = alltrue([
for s in var.subnets : s.public || anytrue([for p in var.subnets : p.public && p.availability_zone == s.availability_zone])
])
error_message = "Every Availability Zone with a private subnet needs a public subnet."
}
}The subnets variable is a map: each key is the name of a subnet. A for_each over it creates all the subnets, so adding a subnet means adding one entry, not copying a resource block.
Resources #
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 5.0"
}
}
}
locals {
public_subnets = { for name, s in var.subnets : name => s if s.public }
private_subnets = { for name, s in var.subnets : name => s if !s.public }
# One NAT Gateway per Availability Zone, in the public subnet of that zone
nat_subnet_by_az = { for name, s in local.public_subnets : s.availability_zone => name }
private_azs = toset([for s in local.private_subnets : s.availability_zone])
# us-east-1a -> za
zone = { for az in keys(local.nat_subnet_by_az) : az => "z${substr(az, -1, 1)}" }
}
resource "aws_vpc" "this" {
cidr_block = var.vpc_cidr_block
tags = {
Name = var.vpc_name
tool = "Terraform"
}
}
resource "aws_subnet" "this" {
for_each = var.subnets
vpc_id = aws_vpc.this.id
cidr_block = each.value.cidr_block
availability_zone = each.value.availability_zone
map_public_ip_on_launch = each.value.public
tags = {
Name = each.key
}
}
resource "aws_internet_gateway" "this" {
vpc_id = aws_vpc.this.id
tags = {
Name = "${var.name_prefix}-ig"
}
}
resource "aws_eip" "nat" {
for_each = local.nat_subnet_by_az
domain = "vpc"
tags = {
Name = "${var.name_prefix}-eip-ngw-${local.zone[each.key]}"
}
}
resource "aws_nat_gateway" "this" {
for_each = local.nat_subnet_by_az
subnet_id = aws_subnet.this[each.value].id
allocation_id = aws_eip.nat[each.key].id
tags = {
Name = "${var.name_prefix}-ngw-${local.zone[each.key]}-pub"
}
depends_on = [aws_internet_gateway.this]
}
# Public routing table, the main one of the VPC: access to the Internet through the Internet Gateway
resource "aws_route_table" "public" {
vpc_id = aws_vpc.this.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.this.id
}
tags = {
Name = "${var.name_prefix}-rt-pub-main"
}
}
resource "aws_main_route_table_association" "public" {
vpc_id = aws_vpc.this.id
route_table_id = aws_route_table.public.id
}
# One private routing table per Availability Zone: access to the Internet through the NAT Gateway of the zone
resource "aws_route_table" "private" {
for_each = local.private_azs
vpc_id = aws_vpc.this.id
tags = {
Name = "${var.name_prefix}-rt-priv-${local.zone[each.key]}"
}
}
resource "aws_route" "private_nat" {
for_each = local.private_azs
route_table_id = aws_route_table.private[each.key].id
destination_cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.this[each.key].id
}
resource "aws_route_table_association" "private" {
for_each = local.private_subnets
subnet_id = aws_subnet.this[each.key].id
route_table_id = aws_route_table.private[each.value.availability_zone].id
}Outputs #
output "vpc_id" {
description = "ID of the VPC"
value = aws_vpc.this.id
}
output "subnet_ids" {
description = "IDs of all the subnets by name"
value = { for name, s in aws_subnet.this : name => s.id }
}
output "public_subnet_ids" {
description = "IDs of the public subnets"
value = [for name, s in local.public_subnets : aws_subnet.this[name].id]
}
output "private_subnet_ids" {
description = "IDs of the private subnets"
value = [for name, s in local.private_subnets : aws_subnet.this[name].id]
}Using the module #
Replace the VPC, subnet, gateway and routing table resources in terraform-aws-tutorial.tf with one module block:
module "network" {
source = "./modules/network"
name_prefix = "ditwl"
vpc_name = "ditlw-vpc"
vpc_cidr_block = "172.21.0.0/19"
subnets = {
"ditwl-sn-za-pro-pub-00" = { cidr_block = "172.21.0.0/23", availability_zone = "us-east-1a", public = true }
"ditwl-sn-za-pro-pri-02" = { cidr_block = "172.21.2.0/23", availability_zone = "us-east-1a", public = false }
"ditwl-sn-zb-pro-pub-04" = { cidr_block = "172.21.4.0/23", availability_zone = "us-east-1b", public = true }
"ditwl-sn-zb-pro-pri-06" = { cidr_block = "172.21.6.0/23", availability_zone = "us-east-1b", public = false }
}
}The rest of the infrastructure used to reference the resources directly. Now it uses the outputs of the module:
| Before | After |
|---|---|
aws_vpc.ditlw-vpc.id |
module.network.vpc_id |
aws_subnet.ditwl-sn-za-pro-pub-00.id |
module.network.subnet_ids["ditwl-sn-za-pro-pub-00"] |
aws_subnet.ditwl-sn-zb-pro-pri-06.id |
module.network.subnet_ids["ditwl-sn-zb-pro-pri-06"] |
For example, an EC2 instance now uses subnet_id = module.network.subnet_ids["ditwl-sn-za-pro-pub-00"] and a security group vpc_id = module.network.vpc_id.
Moving the existing resources: the moved block #
If you run tofu plan now, Terraform sees that aws_vpc.ditlw-vpc disappeared from the code and that module.network.aws_vpc.this is new: it would destroy the whole network and create it again. A moved block tells Terraform that the resource did not change, only its address did, and the state is updated without touching the real infrastructure.
moved {
from = aws_vpc.ditlw-vpc
to = module.network.aws_vpc.this
}
moved {
from = aws_subnet.ditwl-sn-za-pro-pub-00
to = module.network.aws_subnet.this["ditwl-sn-za-pro-pub-00"]
}
moved {
from = aws_subnet.ditwl-sn-za-pro-pri-02
to = module.network.aws_subnet.this["ditwl-sn-za-pro-pri-02"]
}
moved {
from = aws_subnet.ditwl-sn-zb-pro-pub-04
to = module.network.aws_subnet.this["ditwl-sn-zb-pro-pub-04"]
}
moved {
from = aws_subnet.ditwl-sn-zb-pro-pri-06
to = module.network.aws_subnet.this["ditwl-sn-zb-pro-pri-06"]
}
moved {
from = aws_internet_gateway.ditwl-ig
to = module.network.aws_internet_gateway.this
}
moved {
from = aws_eip.ditwl-eip-ngw-za
to = module.network.aws_eip.nat["us-east-1a"]
}
moved {
from = aws_eip.ditwl-eip-ngw-zb
to = module.network.aws_eip.nat["us-east-1b"]
}
moved {
from = aws_nat_gateway.ditwl-ngw-za-pub
to = module.network.aws_nat_gateway.this["us-east-1a"]
}
moved {
from = aws_nat_gateway.ditwl-ngw-zb-pub
to = module.network.aws_nat_gateway.this["us-east-1b"]
}
moved {
from = aws_route_table.ditwl-rt-pub-main
to = module.network.aws_route_table.public
}
moved {
from = aws_main_route_table_association.ditwl-rta-default
to = module.network.aws_main_route_table_association.public
}
moved {
from = aws_route_table.ditwl-rt-priv-za
to = module.network.aws_route_table.private["us-east-1a"]
}
moved {
from = aws_route_table.ditwl-rt-priv-zb
to = module.network.aws_route_table.private["us-east-1b"]
}
moved {
from = aws_route.ditwl-r-rt-priv-za-ngw-za
to = module.network.aws_route.private_nat["us-east-1a"]
}
moved {
from = aws_route.ditwl-r-rt-priv-zb-ngw-zb
to = module.network.aws_route.private_nat["us-east-1b"]
}
moved {
from = aws_route_table_association.ditwl-rta-za-pro-pri-02
to = module.network.aws_route_table_association.private["ditwl-sn-za-pro-pri-02"]
}
moved {
from = aws_route_table_association.ditwl-rta-zb-pro-pri-06
to = module.network.aws_route_table_association.private["ditwl-sn-zb-pro-pri-06"]
}Run the Terraform Plan #
A new module has to be installed before it can be used:
$ tofu init
Initializing modules...
- network in modules/network
Then check the plan. Every moved resource is listed, and nothing is created or destroyed:
$ tofu plan
...
# aws_vpc.ditlw-vpc has moved to module.network.aws_vpc.this
resource "aws_vpc" "this" {
id = "vpc-0361cf67e9e74acf6"
# (14 unchanged attributes hidden)
}
...
Plan: 0 to add, 0 to change, 0 to destroy.
Apply the plan to save the new addresses in the state:
$ tofu apply
Keep moved.tf until everyone who shares the state has applied the change, then delete it.
Public registry modules #
You do not always have to write the module. The Terraform Registry and the OpenTofu Registry have thousands of them. The most popular one for this job, terraform-aws-modules/vpc/aws, builds the same network:
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 5.0" # always pin the version
name = "ditlw-vpc"
cidr = "172.21.0.0/19"
azs = ["us-east-1a", "us-east-1b"]
public_subnets = ["172.21.0.0/23", "172.21.4.0/23"]
private_subnets = ["172.21.2.0/23", "172.21.6.0/23"]
enable_nat_gateway = true
one_nat_gateway_per_az = true
}
Read the code of a registry module before using it and pin its version, as it will create resources in your AWS account.
Module best practices #
- One purpose per module, with a short list of variables. If a module needs 40 variables, it is probably several modules.
- Describe and validate every variable (
description,type,validation) and every output. - No provider blocks inside modules, only
required_providers. The root module configures the provider. - Use
for_eachover a map instead ofcount: adding or removing an element does not renumber the others. - Version the modules that are shared (Git tags or a registry) and update them in a controlled way.
- Document them with a README. The Terraform Tools section shows how to generate it automatically.
Common Questions About Terraform Modules #
What is the difference between a module and a workspace? #
A module is a way to reuse code. A workspace is a way to keep several states for the same code (for example, one per environment). They solve different problems and can be combined.
Can a module use resources created outside of it? #
Yes, but pass them as variables (for example vpc_id) instead of reading them inside the module. The module stays independent and easier to test.
How do I use the same module for several environments? #
Call it several times with different inputs, from different root modules (one directory and one state per environment) or in the same one with different names.
What happens if I rename a resource inside a module? #
Terraform considers it a different resource and recreates it. Add a moved block inside the module with the old and the new address.
Next Steps #
The infrastructure is now organized in modules. Before sharing it with a team, the state must stop living on your computer: continue with Terraform Backends.