How to Share Infrastructure in Multiple Terraform Projects?

There are many ways to share and manage AWS infrastructure

Sharing infrastructure across multiple Terraform projects can help maintain consistency, reduce redundancy, and promote collaboration.

Terraform Data Sources

This is the most common and recommended approach. Data Sources allow you to import information about existing resources from another project and use them in your configuration. This helps avoid redundancy and promotes modularity. See a Terraform example using Terraform Data Sources.

Terraform Remote State

This approach allows you to store the state of one Terraform project in a remote location and then access it from other projects. This can be useful for sharing common infrastructure configurations across multiple environments or teams.

Terraform Modules

Modules provide a way to encapsulate reusable infrastructure code. You can create a module for your shared infrastructure and then import it into other projects. This helps promote code reuse and consistency.

Shared Backend Configuration

You can store common configuration values, such as VPC IDs, in a shared backend like Terraform Cloud or AWS SSM Parameter Store. This allows all projects to access the same configuration values without needing to hard-code them.

Infrastructure as Code (IaC) Tool Integration

Tools like Terraform Cloud, Packer, and Ansible can be integrated to create a complete IaC pipeline. This can help automate the process of building and deploying shared infrastructure across multiple projects. See How to Use Ansible and Terraform Together.

Terraform Workspaces

Workspaces provide a way to manage multiple environments or configurations within a single Terraform project. This can be useful if you have different sets of infrastructure for development, staging, and production.

Terraform Providers

Some cloud providers offer Terraform providers that allow you to manage shared resources across multiple projects. For example, the AWS provider has a data "aws_resourcegroup" resource that allows you to manage AWS Resource Groups, which can be used to group resources across multiple projects.

Using Terraform Data Sources

With this method, multiple projects can share resources (VPC, Subnets, Route Tables, Security Groups, EBS volumes, Customer Gateways... ) using a Terraform Data Source to find the resource ID in a different Terraform project.

Terraform Data Sources allow you to fetch and use external information within your configuration. This includes information from:

  • Existing resources or systems
  • Cloud providers like AWS
  • Databases
  • Other external services
  • Data Sources are defined using the data block and can import data into your Terraform project without creating any actual resources.

By using Data Sources, separate projects can reference the IDs of these components created in another project and leverage them in their configurations.

Updates:

  • 10/Dec/2023: Updated to Terraform > 1.5 and AWS provider v5.30.0

The following example shows a Terraform Base project that creates the basic AWS infrastructure and a Terraform Web Project that creates an EC2 instance on the subnet created by the Base project.

Base Project

The base project creates:

  • A VPC named ditwl-vpc
  • A Subnet named ditwl-sn-za-pro-pub-00.

Once created, other projects can use Terraform data sources to query AWS for the IDs of these elements and use them in other Terraform projects.

base/aws-base.tf
# Copyright (C) 2018 - 2023 IT Wonder Lab (https://www.itwonderlab.com)
#
# This software may be modified and distributed under the terms
# of the MIT license.  See the LICENSE file for details.
# -------------------------------- WARNING --------------------------------
# IT Wonder Lab's best practices for infrastructure include modularizing 
# Terraform/OpenTofu configuration. 
# In this example, we define everything in a single file. 
# See other tutorials for best practices at itwonderlab.com
# -------------------------------- WARNING --------------------------------

#Define Terrraform Providers and Backend
terraform {
  required_version = "> 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"      
    }
  }
}

#-----------------------------------------
# Default provider: AWS
#-----------------------------------------
provider "aws" {
  shared_credentials_files = ["~/.aws/credentials"]
  profile                  = "ditwl_infradmin"
  region                   = "us-east-1" //See BUG https://github.com/hashicorp/terraform-provider-aws/issues/30488
}

# VPC with name ditwl-vpc
resource "aws_vpc" "ditwl-vpc" {
  cidr_block = "172.21.0.0/19" #172.21.0.0 - 172.21.31.254
  tags = {
    Name = "ditwl-vpc"
  }
}

# Subnet with name ditwl-sn-za-pro-pub-00
resource "aws_subnet" "ditwl-sn-za-pro-pub-00" {
  vpc_id     = aws_vpc.ditwl-vpc.id
  cidr_block = "172.21.0.0/23" #172.21.0.0 - 172.21.1.255
  tags = {
    Name = "ditwl-sn-za-pro-pub-00"
  }
}

Terraform Web Project

The sub-project creates only the needed infrastructure for a simple website. Since it needs a VPC and a Subnet, it uses Terraform data sources to find the IDs of the VPC and the Subnet created by the Base project.

Using Terraform Data Sources find:

  • A VPC named ditwl-vpc.
  • A Subnet named ditwl-sn-za-pro-pub-00 and located in the VPC named ditwl-vpc.

Use the value of the subnet resource to create an EC2 instance on it.

web/aws-web.tf
# Copyright (C) 2018 - 2023 IT Wonder Lab (https://www.itwonderlab.com)
#
# This software may be modified and distributed under the terms
# of the MIT license.  See the LICENSE file for details.
# -------------------------------- WARNING --------------------------------
# IT Wonder Lab's best practices for infrastructure include modularizing 
# Terraform/OpenTofu configuration. 
# In this example, we define everything in a single file. 
# See other tutorials for best practices at itwonderlab.com
# -------------------------------- WARNING --------------------------------

#Define Terrraform Providers and Backend
terraform {
  required_version = "> 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"          
    }
  }
}

#-----------------------------------------
# Default provider: AWS
#-----------------------------------------
provider "aws" {
  shared_credentials_files = ["~/.aws/credentials"]
  profile                  = "ditwl_infradmin"
  region                   = "us-east-1" //See BUG https://github.com/hashicorp/terraform-provider-aws/issues/30488
}

#Find AMI focal-20.04-amd64
data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}


#Find a VPC named "ditwl-vpc"
data "aws_vpc" "ditwl-vpc" {  
  filter {
    name = "tag:Name"
    values = ["ditwl-vpc"]
  }  
}
#Find a Subnet located at the VPC named "ditwl-vpc" with tag Name="ditwl-sn-za-pro-pub-00"
data "aws_subnet" "ditwl-sn-za-pro-pub-00" {
  vpc_id = data.aws_vpc.ditwl-vpc.id
  tags = {
    Name = "ditwl-sn-za-pro-pub-00"
  }
}
# Create an AWS instance in the Subnet "ditwl-sn-za-pro-pub-00"
resource "aws_instance" "ditwl-web-01" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
  subnet_id     = data.aws_subnet.ditwl-sn-za-pro-pub-00.id
  tags = {
    Name = "HelloWorld"
  }
}

Next Steps

3 comments on “How to Share Infrastructure in Multiple Terraform Projects?”

Leave a Reply

Your email address will not be published. Required fields are marked *


Related Cloud Tutorials

Control traffic to AWS resources using security groups
How to configure and use the Terraform aws_security_group and aws_security_group_rule resource blocks to create and manage AWS Security Groups and secure the infrastructure.
AWS Routing Tables with Terraform
How to configure and use the Terraform aws_route_table, aws_route, and aws_main_route_table_association resource blocks to create and manage AWS Routing Tables.
AWS NAT Gateway
How to configure and use the Terraform aws_nat_gateway and aws_eip resource blocks to create and manage AWS NAT Gateway and its corresponding Public IPs inside each availability zone to enable Internet access from instances in private subnets.
An AWS NAT Gateway is a managed service that allows instances in a private subnet to connect to the Internet while keeping them secure. It provides network address translation (NAT) for outbound traffic, allowing resources in a private subnet to access the internet while maintaining a private IP address.
AWS Internet Gateway
How to configure and use the Terraform aws_internet_gateway resource block to create and manage AWS Internet Gateway inside a VPC to enable instances access to and from the Internet.
Javier Ruiz Cloud and SaaS Expert

Javier Ruiz

IT Wonder Lab tutorials are based on the diverse experience of Javier Ruiz, who founded and bootstrapped a SaaS company in the energy sector. His company, later acquired by a NASDAQ traded company, managed over €2 billion per year of electricity for prominent energy producers across Europe and America. Javier has over 25 years of experience in building and managing IT companies, developing cloud infrastructure, leading cross-functional teams, and transitioning his own company from on-premises, consulting, and custom software development to a successful SaaS model that scaled globally.

Are you looking for cloud automation best practices tailored to your company?

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram