How to Share Infrastructure in Multiple Terraform Projects?

How to share AWS Infrastructure between multiple teams and projects?

There are many ways to share and manage AWS infrastructure created by Terraform with different teams inside a company.

In this tutorial, we share some methods to divide the responsibility, reduce the risk and give each team access to the infrastructure resources.

Using Terraform Data Sources

With this method, multiple projects can share resources (i.e. a VPC, Subnets, Route Tables, Security Groups, EBS volumes, Customer Gateways… ) by using a Terraform Data Source to find the resource ID in a different Terraform project.

The following example shows a Terraform Base project that creates the basic AWS infrastructure and a Terraform Web Project that creates an EC2 instance on the subnet created by the Base project.

Base Project

The base project creates a VPC and a Subnet. Once created, other projects can ask AWS for the IDs of this elements and use them in other Terraform configuration files.

terraform {
  required_version = "~> 0.12" 
}

provider "aws" {
  shared_credentials_file = pathexpand("~/keys/ditwl_kp_infradmin.pem")
  profile                 = "ditwl_infradmin"
  region                  = "us-east-1"
  version                 = "~> 2.0"
}

resource "aws_vpc" "ditwl-vpc" {
  cidr_block           = "172.17.32.0/19"
  enable_dns_support   = true
  enable_dns_hostnames = true
  tags = {
    Name = "ditwl-vpc"
  }
}

resource "aws_subnet" "ditwl-sn-za-pro-pub-32" {
  vpc_id                  = aws_vpc.ditwl-vpc.id
  cidr_block              = "172.17.32.0/23"
  availability_zone       = "us-east-1a"
  map_public_ip_on_launch = "true"
  tags = {
    Name = "ditwl-sn-za-pro-pub-32"
  }
}

Terraform Web Project

The sub project creates only the needed infrastructure for a simple website. Since it needs a VPC and a Subnet, it uses Terraform Data Sources to find out the IDs of the VPC and the Subnet created by the Base project.

terraform {
  required_version = "~> 0.12" 
}

provider "aws" {
  shared_credentials_file = pathexpand("~/keys/ditwl_kp_infradmin.pem")
  profile                 = "ditwl_infradmin"
  region                  = "us-east-1"
  version                 = "~> 2.0"
}

data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}

#Find a VPC named "ditwl-vpc"
data "aws_vpc" "ditwl-vpc" {  
  filter {
    name = "tag:Name"
    values = ["ditwl-vpc"]
  }  
}

#Find a Subnet located at the VPC named "ditwl-vpc" with tag Name="ditwl-sn-za-pro-pub-32"
data "aws_subnet" "ditwl-sn-za-pro-pub-32" {
  vpc_id = data.aws_vpc.ditwl-vpc.id
  tags = {
    Name = "ditwl-sn-za-pro-pub-32"
  }
}

# Create an AWS instance in the Subnet "ditwl-sn-za-pro-pub-32"
resource "aws_instance" "ditwl-web-01" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
  subnet_id     = data.aws_subnet.ditwl-sn-za-pro-pub-32.id
  tags = {
    Name = "HelloWorld"
  }
}

Accessing a Remote Terraform State-file From Other Project

WIP