AWS VPC, Route 53, RDS MariaDB, EC2 using Ansible and Terraform (1/5)

Terraform Setup and VPC Subnet Creation

In this Terraform and Ansible demo for AWS you can find all the code needed to create a VPC (Virtual Private Cloud) in AWS (Amazon Web Services) with an EC2 (Elastic Compute) instance connected to MariaDB database running in RDS (Relational Database Services) using a single Terraform 0.12 plan and installing and configuring an Ubuntu server with Nginx, PHP, and Let’s Encrypt to run WordPress with Ansible.

This VPC can be used to install a simple web application like WordPress using Ansible, which will be shown in another post.

The demo follows some IT Wonder Lab best practices using only¹ resources included in the AWS free tier plan.

A real infrastructure should make use of some resources that are not included in the AWS free tier plan like NAT instances and IPSec VPNs. It should also have different VPCs or at least subnets for each environment, reuse Terraform modules, and store the tfstate in a shared location (AWS S3), …

Updates:

• 23 Jun 2020: Updated to Terraform 0.12

General Diagram

The diagram shows the main elements that will be created by Terraform:

• 1 VPC
• 4 subnets
• 4 security groups
• 3 route tables
• 1 EC2 instance
• 1 Internet Gateway
• 1 Route 53 Hosted zone
• 1 DB Subnet Group
• 1 RDS MariaDB

Apply

The following asciicast shows how Terraform creates all the resources in AWS. Applying the whole plan takes around 10 minutes * (AWS RDS MariaDb creation takes more than 7 minutes).

Click on the play button to see the execution of the Terraform plan.

* The asciicast has been edited to show everything in 2 minutes.

Prerequisites

A workstation running an operating system compatible with the required software.  We will use Ubuntu for the demo.

1. Create a new AWS account for the demo.
2. Create an AWS IAM User for Demos.
3. Make sure you understand the AWS VPC Basic Elements.
4. Install the required software to run the demo:
   1. Terraform 0.12
   2. GIT
   3. Atom, Visual Studio Code or your favorite text editor
5. Read Use your public Internet IP address in Terraform.
6. Read Avoiding AWS instance destroy with Terraform.
7. Download the source code for the demo from IT Wonder Lab public GitHub repository. 
8. Make whatismyip.sh executable using:

chmod 764 whatismyip.sh

File Layout

Terraform configuration files are used to describe infrastructure and to set variables, most of the examples are found on the Internet using a single file or at most a few files to configure all the infrastructure.

Best Practice: Terraform File Naming and Layout
On this example to create an AWS VPC with an EC2 Instance and a MariaDB RDS Data Base you will find a Terraform layout that follows our best practice recommendation for defining multiple environments on the same VPC using Terraform modules

The example has the following files:

Configure Terraform AWS Provider

Terraform needs to have the Account ID and the credentials for the AWS account that will be used to interact with AWS API.

AWS credentials will be stored outside Terraform in ~/.aws/credentials, using the Shared Credentials file option for Terraform.

Best Practice: Terraform and AWS credentials: Terraform AWS provider has other options for credential configuration, but this one is an IT Wonder Lab best practice as it stores the credentials outside the Terraform configuration, therefore reducing the risk of credential leaking and it also allows each user that checkouts the code to run the Terraform plans with its own credentials.

Create or open the credentials file and add the following content:

[ditwl_infradmin]
aws_access_key_id=A1B2C3D4E5F6G7H8I9J0
aws_secret_access_key=QwertYuiopASDFGHJKL123456789sadfghjkvcbn

The profile name is surrounded by square brackets [ditwl_infradmin] and will be the name used in the Terraform configuration to identify the credentials to use.

aws_access_key_id and aws_secret_access_key are the ones obtained when creating the AWS IAM User for Demos.

You will also need to set other configurations related to AWS credentials in the file terraform.tfvars  located in the root directory of the checked example:

#------------------------
# PROVIDERS
#------------------------

# DEFAULT AWS
provider_default_aws_profile = "ditwl_infradmin"
provider_default_aws_region = "us-east-1"
provider_default_aws_account_id = ["134567891011"] 
provider_default_aws_shared_credentials_file = "~/.aws/credentials"
provider_default_aws_key_file = "~/keys/ditwl_kp_infradmin.pem"

• provider_default_aws_profile: Corresponds to the profile name used in the credentials file.
• provider_default_aws_region: us-east-1 corresponds to US East (N. Virginia) and it is the cheapest region. See choosing your AWS Region wisely.
• provider_default_aws_account_id: It is the AWS account id, see Create a new AWS account for the demo.
• provider_default_aws_shared_credentials_file: the full path to the credentials file. Default path “~/.aws/credentials” is used.
• provider_default_aws_key_file: This is the full path to a private key file used for Terraform instance provisioning. It will be used by Ansible to connect to the instance to install the needed software.

The values from terraform.tfvars are used in provider_aws.tf to configure the AWS Provider.

Setting the allowed_account_ids prevents Terraform from applying changes to a different AWS account.

Best Practice: Redundancy in Configuration Files: IT Wonder Lab’s best practice is to specify the provider_default_aws_account_id in the terraform.tfvars and use it to fill the allowed_account_ids in the AWS provider.

The aws_access_key_id of an IAM user is used for AWS to identify the Account ID that the user belongs to, and all operations are applied to that Account, so there is technically no need to specify the Account ID again.

This is an intended redundancy, as many others that you will find in IT Wonder Lab, it is a double check on the account to prevent catastrophic errors.

provider "aws" {
  shared_credentials_file = pathexpand(var.provider_default_aws_shared_credentials_file)
  profile                 = var.provider_default_aws_profile
  region                  = var.provider_default_aws_region
  allowed_account_ids     = var.provider_default_aws_account_id
  version                 = "~> 2.0"
}

VPC Subnet Creation

AWS creates a default VPC for each AWS Region when an account is created.

Deleting the default VPC is irreversible and that you will need to contact AWS or recreate the account if you want to have a default VPC again.

Terraform provides a data source that lets you “adopt”  the default VPC but that will not be used.

Best Practice: Terraform managed AWS VPC: An IT Wonder Lab best practice is to delete the default VPC and recreate everything using Terraform. 
If you are going to be responsible for the Cloud Infrastructure, better know everything about it.
You have to be in control of every piece of the infrastructure (like the chosen IP Range) and I like to be aware of all the details

The VPC will be created using the 172.17.32.0/19 private IPv4 address range (8192 IPs) divided into many smaller subnets of 512 hosts (subnet mask/23). See ipv4 Subnet Calculator.

terraform.tfvars

#------------------------
# VPC
#------------------------

aws_vpc_tag_name = "ditwl-vpc"
aws_vpc_block = "172.17.32.0/19" #172.17.32.1 - 172.16.67.254

For the demo, we will only provision 4 subnets.

Best Practice: Multiple AWS Availability Zones: An IT Wonder Lab’s best practice is to distribute sub-nets in more than one Availability Zone and have private and public zones.

Instance placement has to take into account that data transfer between different Availability Zones has a cost, therefore achieving high availability by distributing infrastructure between Availability Zones has an impact on the total budget.

Terraform Subnet Definition

terraform.tfvars

#------------------------
# SUBNETS
#------------------------

  #------------------------
  # For EC2 instances
  #------------------------

    #Zone: A, Env: PRO, Type: PUBLIC, Code: 32
    aws_sn_za_pro_pub_32={
      cidr   ="172.17.32.0/23" #172.17.32.1 - 172.17.33.254
      name   ="ditwl-sn-za-pro-pub-32"
      az     ="us-east-1a"
      public = "true"
    }

    #Zone: A, Env: PRO, Type: PRIVATE, Code: 34
    aws_sn_za_pro_pri_34={
      cidr   = "172.17.34.0/23" #172.17.34.1 - 172.17.35.254
      name   = "ditwl-sn-za-pro-pri-34"
      az     = "us-east-1a"
      public = "false"
    }

    #Zone: B, Env: PRO, Type: PUBLIC, Code: 36
    aws_sn_zb_pro_pub_36={
      cidr   = "172.17.36.0/23" #172.17.36.1 - 172.17.37.254
      name   = "ditwl-sn-zb-pro-pub-36"
      az     = "us-east-1b"
      public = "false"
    }

    #Zone: B, Env: PRO, Type: PRIVATE, Code: 38
    aws_sn_zb_pro_pri_38={
      cidr   = "172.17.38.0/23" #172.17.38.1 - 172.17.39.254
      name   = "ditwl-sn-zb-pro-pri-38"
      az     = "us-east-1b"
      public = "false"
    }

Plan

Run the Terraform plan command often as it is the best way to check that everything is correct. Some errors are difficult to identify, our recommendation is to make changes to the Terraform configuration in small doses and test the plan.

The following asciicast shows the elements that Terraform will create in AWS.

Click on the play button to see the execution of Terraform plan.

Continue the Terraform and Ansible howto, see:

• Terraform AWS Tutorials
  • Terraform Setup and AWS VPC Subnet Creation (1/5)
  • VPC Subnet Routing and Internet Access with Terraform (2/5)
  • AWS Route 53, AMI Lookup and EC2 Creation with Terraform (3/5)
  • AWS RDS MariaDB Database and Resource Tags with Terraform (4/5)
  • EC2 Instances and Resource Security with Terraform (5/5)
• Ansible Tutorials
  • Using Ansible for Terraform Managed AWS Infrastructure

1. The DNS zone is not included in the AWS Free Tier. ↩︎