Terraform Setup and VPC Subnet Creation

In this Terraform and Ansible demo for AWS you can find all the code needed to create a VPC (Virtual Private Cloud) in AWS (Amazon Web Services) with an EC2 (Elastic Compute) instance connected to MariaDB data base running in RDS (Relational Database Services) using a single Terraform plan and installing and configuring an Ubuntu server  with nginx, php, and Let’s Encrypt to run WordPress with Ansible.

For an introduction to AWS, see AWS VPC Basic Elements.

Continue the demo, see:

Download de the code from IT Wonder Lab public GitHub repository.

Terraform logoThis VPC can be used to install a simple web application like WordPress using Ansible, that will be shown in another post.

The demo follows some IT Wonder Lab best practices using only* resources included in the AWS free tier plan.

A real infrastructure should make use of some resources that are not included in AWS free tier plan like NAT instances and IPSec VPNs. It should also have different VPCs or at least sub nets for each environment, reuse Terraform modules, store the tfstate in a shared location, …

* The DNS zone is not included in AWS Free tier.

General Diagram

The Terraform plan will create:

  • 1 VPC
  • 4 subnets
  • 4 security groups
  • 3 route tables
  • 1 EC2 instance
  • 1 Internet Gateway
  • 1 Route 53 Hosted zone
  • 1 DB Subnet Group
  • 1 RDS MariaDB

The diagram shows the main elements:

terraform-aws-ec2-rds-basic-free - ITWL_AWS_Terraform_VCP_WP_RDS.png

Apply

The following asciicast shows how Terraform creates all the resources in AWS. Applying the whole plan takes around 10 minutes *.

Click on the play button to see the execution of Terraform plan.

* The asciicast has been edited to show everything in less than a minute.

Prerequisites

A workstation running an operating system compatible with the required software.  We use Ubuntu.

  1. Create a new AWS account for the demo.
  2. Create an AWS IAM User for Demos.
  3. Make sure you understand the AWS VPC Basic Elements.
  4. Install the required software to run the demo:
    1. Terraform
    2. GIT
    3. Atom or your favorite text editor
  5. Read Use your public Internet IP address in Terraform.
  6. Read Avoiding AWS instance destroy with Terraform.
  7. Download the source code for the demo from IT Wonder Lab public GitHub repository
  8. Make whatismyip.sh executable using:

File Layout

Terraform configuration files are used to describe infrastructure and to set variables, most of the examples found on the Internet use a single file or at most some few files to configure all the infrastructure.

On this example to create an AWS VPC with an EC2 Instance and a MariaDB RDS Data Base you will find a Terraform layout that follows our best practice recommendation for  defining multiple environments on the same VPC.

The example has the following files:

NameDescription
modulesDirectory containing Terraform modules
aws_ds_aws_ami.tfData source to get the ID of the latests AMI for selected OS
aws_ec2_pro_wp.tfWord Press Server, associated security groups, DNS registration
aws_ec2_pro_wp_vars.tfWord Press Server variables
aws_internet_gateway.tfInternet Gateway
aws_internet_gateway_vars.tfInternet Gateway variables
aws_rds_pro_mariadb_01.tfRDS MariaDB and associated security groups
aws_rds_pro_mariadb_01_vars.tfRDS MariaDB variables
aws_rds_sn_pro.tfRDS subnets
aws_rds_sn_pro_vars.tfRDS subnets variables
aws_route53.tfRoute 53 (DNS)
aws_route53_vars.tfRoute 53 (DNS) variables
aws_sec_group_ec2_default.tfDefault security group to assign to all EC2 instances
aws_sec_group_ec2_default_vars.tfDefault EC2 security group variables
aws_sec_group_rds_mariadb_default.tfDefault security group to assign to all RDS MariaDB instances
aws_sec_group_rds_mariadb_default_vars.tfDefault RDS security group variables
aws_vpc_routing.tfRouting tables for subnets
aws_vpc_routing_vars.tfRouting tables for subnets variables
aws_vpc_subnets.tfVPC subnets
aws_vpc_subnets_vars.tfVPC subnets variables
aws_vpc.tfVPC
aws_vpc_vars.tfVPC variables
external_whatismyip.tfObtains current Public Internet IP for usage in firewall rules
provider_aws.tfDefines AWS provider
provider_aws_vars.tfDefines AWS provider variables
terraform.tfDefines terraform
terraform.tfvarsValues for all the variables in the demo
terraform_vars.tfDefines terraform variables
whatismyip.shScript to obtain current Public Internet IP
.gitignoreDefines the file and patterns that should not be pushed to git repository
.terraformWorking directory created by Terraform

Configure Terraform AWS Provider

Terraform needs to have the Account Id and the credentials for the AWS account that will be used to interact with AWS API.

AWS credentials will be stored outside Terraform in ~/.aws/credentials, using the Shared Credentials file option for Terraform.

Terraform AWS provider has other options for credential configuration, but this one is an IT Wonder Lab best practice as it stores the credentials outside the Terraform configuration, therefore reducing the risk of credential leaking and it also allows each user that checkouts the code to run the Terraform plans with its own credentials.

Create or open the credentials file and add the following content:

The profile name is surrounded by square brackets [ditwl_infradmin] and will be the name used in Terraform configuration to identify the credentials to use.

aws_access_key_id and aws_secret_access_key are the ones obtained when creating the AWS IAM User for Demos.

You will also need to set other configuration related to AWS credentials in the file terraform.tfvars  located in the root directory of the checked example:

  • provider_default_aws_profile: Corresponds to the profile name used in the credentials file.
  • provider_default_aws_region: us-east-1 corresponds to US East (N. Virginia) and it is the cheapest region. See  choosing your AWS Region wisely.
  • provider_default_aws_account_id: it is the AWS account id, see Create a new AWS account for the demo.
  • provider_default_aws_shared_credentials_file: the full path to the credentials file. Default path “~/.aws/credentials” is used.
  • provider_default_aws_key_file: this is the full path to a private key file used for Terraform instance provisioning. Not used in this example.

The values from terraform.tfvars are used in provider_aws.tf to configure the AWS Provider.

Setting the allowed_account_ids prevents Terraform from applying changes to a different AWS account.

IT Wonder Lab best practice is to specify the provider_default_aws_account_id in the terraform.tfvars and use it to fill the allowed_account_ids in the AWS provider.

The aws_access_key_id of an IAM user is used for AWS to identify the Account ID that the user belongs to, and all operations are applied to that Account, so there is technically no need to specify the Account ID again.

This is a intended redundancy, as many others that you will find in IT Wonder Lab, its a double check on the account to prevent catastrophic errors.

VPC Subnet Creation

AWS creates a default VPC for each AWS Region when an account is created.

Deleting the default VPC is irreversible and that you will need to contact AWS or recreate the account if you want to have a default VPC again.

Terraform provides a data source that lets you “adopt”  the default VPC but that will not be used.

An IT Wonder Lab best practice is to delete the default VPC and recreate everything using Terraform.  The reason is that if you are going to be responsible for the Cloud Infrastructure, better know everything about it. I like to control every piece of the infrastructure (like the chosen IP Range) and I like to be aware of all the details.

The VPC will be created using the 172.17.32.0/19 private IPv4 address range (8192 IPs) divided into many smaller sub nets of 512 hosts (sub net mask /23). See ipv4 Subnet Calculator.

terraform.tfvars

For the demo we will only provision 4 sub nets.

IT Wonder Lab best practice is to distribute sub nets in more than one Availability Zone and have private and public zones.

terraform-aws-ec2-rds-basic-free - ITWL_AWS_Terraform_VCP_SN2.png

Instance placement has to take into account that data transfer between different Availability Zones has a cost, therefore achieving high availability by distributing infrastructure between Availability Zones has an impact in the total budget.

CIDRHost Address RangeTerraform nameDescription
172.17.32.0/23172.17.32.1 - 172.17.33.254aws_sn_za_pro_pub_32Zone: A
Env: PRO
Type: PUBLIC
Code: 32
172.17.34.0/23172.17.34.1 - 172.17.35.254aws_sn_za_pro_pri_34Zone: A
Env: PRO
Type: PRIVATE
Code: 34
172.17.36.0/23172.17.36.1 - 172.17.37.254aws_sn_zb_pro_pub_36Zone: B
Env: PRO
Type: PUBLIC
Code: 36
172.17.38.0/23172.17.38.1 - 172.17.39.254aws_sn_zb_pro_pri_38Zone: B
Env: PRO
Type: PRIVATE
Code: 38
172.17.40.0/23172.17.40.1 - 172.17.41.254aws_sn_z?_pr?_p??_40Not in use
172.17.42.0/23172.17.42.1 - 172.17.43.254aws_sn_z?_pr?_p??_42Not in use
172.17.44.0/23172.17.44.1 - 172.17.45.254aws_sn_z?_pr?_p??_44Not in use
172.17.46.0/23172.17.46.1 - 172.17.47.254aws_sn_z?_pr?_p??_46Not in use
172.17.48.0/23172.17.48.1 - 172.17.49.254aws_sn_z?_pr?_p??_48Not in use
172.17.50.0/23172.17.50.1 - 172.17.51.254aws_sn_z?_pr?_p??_50Not in use
172.17.52.0/23172.17.52.1 - 172.17.53.254aws_sn_z?_pr?_p??_52Not in use
172.17.54.0/23172.17.54.1 - 172.17.55.254aws_sn_z?_pr?_p??_54Not in use
172.17.56.0/23172.17.56.1 - 172.17.57.254aws_sn_z?_pr?_p??_56Not in use
172.17.58.0/23172.17.58.1 - 172.17.59.254aws_sn_z?_pr?_p??_58Not in use
172.17.60.0/23172.17.60.1 - 172.17.61.254aws_sn_z?_pr?_p??_60Not in use
172.17.62.0/23172.17.62.1 - 172.17.63.254aws_sn_z?_pr?_p??_62Not in use

IT Wonder Lab best practice is to name each Terraform variable following a pattern of elements.

For subnets variables the pattern used is, lowercase with the following elements separated by underscores:

  • Provider: a prefix specifying the name of the provider, in this case aws
  • Resource: a short name identifying the resource, in this case sn stands for subnet
  • Environment: for resources that are not to be shared between environments, a 3 letter acronym for the environment:
    • pro: production
    • pre: preproduction
    • dev: development
  • Visibility: for resources that can be either public or private, a 3 letter acronym for the visibility:
    • pub: for public resources
    • pri: for private resources
  • Unique ID: a unique number related to resource. In this case the third octet in the IP address (172.17.NN.0) is used because it is different in all the subnets. The idea behind having a unique ID on the names is to be able to see errors from copy/paste easily by detecting anti patterns.

Example:

aws_sn_za_pro_pub_32 = { …}

The name of the resource follows almost the same pattern but replaces the provider with a prefix and uses a dash as separator:

  • Cloud: a prefix specifying the unique name of this cloud across all available clouds and providers. In this case the prefix will be: ditwl that stands for Demo IT Wonder Lab in lowercase.

It is important to have this unique prefix as external elements (like Monitoring System) will have to be able to differentiate between many clouds.

Example:

name =”ditwl-sn-za-pro-pub-32″

Using a consistent pattern allows the variable name to have all needed information to understand its purpose making the description almost unnecessary.

terraform.tfvars Subnet definition

Plan

The following asciicast shows the elements that Terraform will create in AWS.

IT Wonder Lab best practice is to run the Terraform plan command often as it is the best way to check that everything is correct. Some errors are difficult to identify, my recommendation is to make changes to the Terraform configuration in small doses and test the plan.

Click on the play button to see the execution of Terraform plan.

Continue the Terraform and Ansible demo, see:


Leave a Reply

avatar

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
Notify of