Sharing infrastructure across multiple Terraform projects can help maintain consistency, reduce redundancy, and promote collaboration.
This is the most common and recommended approach. Data Sources allow you to import information about existing resources from another project and use them in your configuration. This helps avoid redundancy and promotes modularity. See a Terraform example using Terraform Data Sources.
This approach allows you to store the state of one Terraform project in a remote location and then access it from other projects. This can be useful for sharing common infrastructure configurations across multiple environments or teams.
Modules provide a way to encapsulate reusable infrastructure code. You can create a module for your shared infrastructure and then import it into other projects. This helps promote code reuse and consistency.
You can store common configuration values, such as VPC IDs, in a shared backend like Terraform Cloud or AWS SSM Parameter Store. This allows all projects to access the same configuration values without needing to hard-code them.
Tools like Terraform Cloud, Packer, and Ansible can be integrated to create a complete IaC pipeline. This can help automate the process of building and deploying shared infrastructure across multiple projects. See How to Use Ansible and Terraform Together.
Workspaces provide a way to manage multiple environments or configurations within a single Terraform project. This can be useful if you have different sets of infrastructure for development, staging, and production.
Some cloud providers offer Terraform providers that allow you to manage shared resources across multiple projects. For example, the AWS provider has a data "aws_resourcegroup" resource that allows you to manage AWS Resource Groups, which can be used to group resources across multiple projects.
With this method, multiple projects can share resources (VPC, Subnets, Route Tables, Security Groups, EBS volumes, Customer Gateways... ) using a Terraform Data Source to find the resource ID in a different Terraform project.
Terraform Data Sources allow you to fetch and use external information within your configuration. This includes information from:
By using Data Sources, separate projects can reference the IDs of these components created in another project and leverage them in their configurations.
Updates:
The following example shows a Terraform Base project that creates the basic AWS infrastructure and a Terraform Web Project that creates an EC2 instance on the subnet created by the Base project.
The base project creates:
ditwl-vpc
ditwl-sn-za-pro-pub-00
.Once created, other projects can use Terraform data sources to query AWS for the IDs of these elements and use them in other Terraform projects.
# Copyright (C) 2018 - 2023 IT Wonder Lab (https://www.itwonderlab.com) # # This software may be modified and distributed under the terms # of the MIT license. See the LICENSE file for details. # -------------------------------- WARNING -------------------------------- # IT Wonder Lab's best practices for infrastructure include modularizing # Terraform/OpenTofu configuration. # In this example, we define everything in a single file. # See other tutorials for best practices at itwonderlab.com # -------------------------------- WARNING -------------------------------- #Define Terrraform Providers and Backend terraform { required_version = "> 1.5" required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } } #----------------------------------------- # Default provider: AWS #----------------------------------------- provider "aws" { shared_credentials_files = ["~/.aws/credentials"] profile = "ditwl_infradmin" region = "us-east-1" //See BUG https://github.com/hashicorp/terraform-provider-aws/issues/30488 } # VPC with name ditwl-vpc resource "aws_vpc" "ditwl-vpc" { cidr_block = "172.21.0.0/19" #172.21.0.0 - 172.21.31.254 tags = { Name = "ditwl-vpc" } } # Subnet with name ditwl-sn-za-pro-pub-00 resource "aws_subnet" "ditwl-sn-za-pro-pub-00" { vpc_id = aws_vpc.ditwl-vpc.id cidr_block = "172.21.0.0/23" #172.21.0.0 - 172.21.1.255 tags = { Name = "ditwl-sn-za-pro-pub-00" } }
The sub-project creates only the needed infrastructure for a simple website. Since it needs a VPC and a Subnet, it uses Terraform data sources to find the IDs of the VPC and the Subnet created by the Base project.
Using Terraform Data Sources find:
ditwl-vpc
.ditwl-sn-za-pro-pub-00
and located in the VPC named ditwl-vpc
.Use the value of the subnet resource to create an EC2 instance on it.
# Copyright (C) 2018 - 2023 IT Wonder Lab (https://www.itwonderlab.com) # # This software may be modified and distributed under the terms # of the MIT license. See the LICENSE file for details. # -------------------------------- WARNING -------------------------------- # IT Wonder Lab's best practices for infrastructure include modularizing # Terraform/OpenTofu configuration. # In this example, we define everything in a single file. # See other tutorials for best practices at itwonderlab.com # -------------------------------- WARNING -------------------------------- #Define Terrraform Providers and Backend terraform { required_version = "> 1.5" required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } } #----------------------------------------- # Default provider: AWS #----------------------------------------- provider "aws" { shared_credentials_files = ["~/.aws/credentials"] profile = "ditwl_infradmin" region = "us-east-1" //See BUG https://github.com/hashicorp/terraform-provider-aws/issues/30488 } #Find AMI focal-20.04-amd64 data "aws_ami" "ubuntu" { most_recent = true filter { name = "name" values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"] } filter { name = "virtualization-type" values = ["hvm"] } owners = ["099720109477"] # Canonical } #Find a VPC named "ditwl-vpc" data "aws_vpc" "ditwl-vpc" { filter { name = "tag:Name" values = ["ditwl-vpc"] } } #Find a Subnet located at the VPC named "ditwl-vpc" with tag Name="ditwl-sn-za-pro-pub-00" data "aws_subnet" "ditwl-sn-za-pro-pub-00" { vpc_id = data.aws_vpc.ditwl-vpc.id tags = { Name = "ditwl-sn-za-pro-pub-00" } } # Create an AWS instance in the Subnet "ditwl-sn-za-pro-pub-00" resource "aws_instance" "ditwl-web-01" { ami = data.aws_ami.ubuntu.id instance_type = "t3.micro" subnet_id = data.aws_subnet.ditwl-sn-za-pro-pub-00.id tags = { Name = "HelloWorld" } }
IT Wonder Lab tutorials are based on the diverse experience of Javier Ruiz, who founded and bootstrapped a SaaS company in the energy sector. His company, later acquired by a NASDAQ traded company, managed over €2 billion per year of electricity for prominent energy producers across Europe and America. Javier has over 25 years of experience in building and managing IT companies, developing cloud infrastructure, leading cross-functional teams, and transitioning his own company from on-premises, consulting, and custom software development to a successful SaaS model that scaled globally.
Are you looking for cloud automation best practices tailored to your company?
Hello Javi,
I wanted to thank you for the exceptional tutorials.
Thanks, Rodrigo, I appreciate your feedback.