How to use Terraform, AWS, and Ansible Together

Linking Terraform AWS, and Ansible #

Terraform is an Infrastructure as Code (IaC) tool used to create infrastructure and should never be used to configure operating systems or applications, instead, Ansible is the de facto standard for the operating system and application configuration as code.

This tutorial shows how to create infrastructure in AWS using Terraform and configure the operating system and applications using Ansible.

Linking the infrastructure created in AWS with Terraform is done using infrastructure tags.

Prerequisites #

• Install Terraform or OpenTofu.
• Install Ansible.
• AWS account, see How to Create an AWS Account for demos.
• AWS CLI profile, see How to Install AWS CLI V2.
• A Private Key Pair, See Generating and using AWS Key Pairs with Terraform or OpenTofu.
• Download the Terraform and Ansible Tutorial source code from the IT Wonder Lab GitHub repository.

Define the Infrastructure Tags #

All the AWS resources created with Terraform will have tags that follow a company-wide standard. Tags are used for provisioning, monitoring, and cost control. Check AWS Resource Tagging best practices.

Create the Infrastructure in AWS using Terraform #

Please be aware of the following limitations and risks:

• Security Flaws: The infrastructure is intentionally designed with security flaws for learning purposes. Implementing these configurations in production would expose your systems to vulnerabilities.
• Design Flaws: The architecture may not be optimized for scalability, performance, or cost-efficiency. Using these configurations in production could lead to inefficient resource utilization and performance issues.
• Cost Considerations: Running cloud resources associated with the tutorial may incur costs. Be sure to monitor and terminate resources promptly to avoid unnecessary charges.
• Limited Scope: The tutorial is intended to provide a basic understanding of Terraform and Ansible. It does not cover advanced topics or production-ready configurations.

Before proceeding with this tutorial:

• Carefully review the provided code and configurations to understand the potential risks.
• Only run the tutorial in a non-production environment.
• Terminate all resources created during the tutorial when finished.
• Do not use the generated infrastructure in any production environment.

By proceeding with this tutorial, you acknowledge the risks and limitations outlined above and agree to take full responsibility for your actions.

The Terraform file terraform/aws-basic-vpc-ec2-tags.tf contains all the code necessary to create the AWS infrastructure. You can either download the code from the IT Wonder Lab GitHub repository or create the files manually.

The following sections show the Terraform code used to create each AWS element.

terraform/aws-basic-vpc-ec2-tags.tf

Providers and AWS Provider configuration #

Defines the required Terraform version and the AWS required provider.

# Copyright (C) 2018 - 2023 IT Wonder Lab (https://www.itwonderlab.com)
#
# This software may be modified and distributed under the terms
# of the MIT license.  See the LICENSE file for details.
# -------------------------------- WARNING --------------------------------
# IT Wonder Lab's best practices for infrastructure include modularizing 
# Terraform/OpenTofu configuration. 
# In this example, we define everything in a single file. 
# See other tutorials for best practices at itwonderlab.com
# -------------------------------- WARNING --------------------------------

#Define Terrraform Providers and Backend
terraform {
  required_version = "> 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

#-----------------------------------------
# Default provider: AWS
#-----------------------------------------
provider "aws" {
  shared_credentials_files = ["~/.aws/credentials"]
  profile                  = "ditwl_infradmin"
  region                   = "us-east-1"
}

Networking (VPC, Subnet and Route Table) #

Create a VPC named ditlw-vpc, a single subnet named ditwl-sn-za-pro-pub-00 configured as a public subnet to assign Public IPs to its EC2 instances.

An Internet Gateway, named ditwl-ig, is created to allow EC2 instances with public IPs to access the Internet.

A Routing table named ditwl-rt-pub-main, is created and assigned as the main or default routing table for the VPC, it includes a route to send all traffic directed to the Internet through the Internet Gateway.

# VPC
resource "aws_vpc" "ditlw-vpc" {
  cidr_block = "172.21.0.0/19" #172.21.0.0 - 172.21.31.254
  tags = {
    Name = "ditlw-vpc"
  }
}

# Subnet
resource "aws_subnet" "ditwl-sn-za-pro-pub-00" {
  vpc_id                  = aws_vpc.ditlw-vpc.id
  cidr_block              = "172.21.0.0/23" #172.21.0.0 - 172.21.1.255
  map_public_ip_on_launch = true
  tags = {
    Name = "ditwl-sn-za-pro-pub-00"
  }
}

# Internet Gateway
resource "aws_internet_gateway" "ditwl-ig" {
  vpc_id = aws_vpc.ditlw-vpc.id
  tags = {
    Name = "ditwl-ig"
  }
}

# Routing table for public subnet (access to Internet)
resource "aws_route_table" "ditwl-rt-pub-main" {
  vpc_id = aws_vpc.ditlw-vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.ditwl-ig.id
  }

  tags = {
    Name = "ditwl-rt-pub-main"
  }
}

# Set new main_route_table as main
resource "aws_main_route_table_association" "ditwl-rta-default" {
  vpc_id         = aws_vpc.ditlw-vpc.id
  route_table_id = aws_route_table.ditwl-rt-pub-main.id
}

Network and Instance Security #

AWS Security groups hold the security rules that allow ingress and egress traffic to and from the EC2 instances. In this example, a basic group will be created to be assigned to all instances and the other two security groups will hold the rules specific to each instance.

Base Security Group: ditwl-sg-base-ec2

Creates a Security Group named ditwl-sg-base-ec2 and the following rules:

• ditwl-sr-internet-to-ec2-ssh: Rule Allow SSH to EC2 instances (ingress).
• ditwl-sr-internet-to-ec2-icmp: Allow ICMP (Ping) from the Internet into the instances (ingress).
• ditwl-sr-all-outbund: Allow all protocols from the instances to the Internet (egress).

The Base Security Group will be assigned to all the EC2 instances.

# Create a "base" Security Group to be assigned to all EC2 instances
resource "aws_security_group" "ditwl-sg-base-ec2" {
  name        = "ditwl-sg-ssh-ec2"
  vpc_id      = aws_vpc.ditlw-vpc.id
}

# DANGEROUS!!
# Allow access from the Internet to port 22 (SSH) in the EC2 instances
resource "aws_security_group_rule" "ditwl-sr-internet-to-ec2-ssh" {
  security_group_id        = aws_security_group.ditwl-sg-base-ec2.id
  type                     = "ingress"
  from_port                = 22
  to_port                  = 22
  protocol                 = "tcp"
  cidr_blocks              = ["0.0.0.0/0"] # Internet
}

# Allow access from the Internet for ICMP protocol (e.g. ping) to the EC2 instances
resource "aws_security_group_rule" "ditwl-sr-internet-to-ec2-icmp" {
  security_group_id        = aws_security_group.ditwl-sg-base-ec2.id
  type                     = "ingress"
  from_port                = -1
  to_port                  = -1
  protocol                 = "icmp"
  cidr_blocks              = ["0.0.0.0/0"] # Internet
}

# Allow all outbound traffic to Internet
resource "aws_security_group_rule" "ditwl-sr-all-outbund" {
  security_group_id = aws_security_group.ditwl-sg-base-ec2.id
  type              = "egress"
  from_port         = "0"
  to_port           = "0"
  protocol          = "-1"
  cidr_blocks       = ["0.0.0.0/0"]
}

Front-end Server Security Group: ditwl-sg-front-end

Creates a Security Group named ditwl-sg-front-end with a single rule:

• ditwl-sr-internet-to-front-end-80: Rule to allow access from the Internet to TCP port 80.

This Security Group will be assigned to the front-end instance allowing access from the Internet to the instance port 80 (HTTP web server).

# Create a Security Group for the Front end Server
resource "aws_security_group" "ditwl-sg-front-end" {
  name        = "ditwl-sg-front-end"
  vpc_id      = aws_vpc.ditlw-vpc.id
}

# Allow access from the Internet to port 80 in the EC2 instances
resource "aws_security_group_rule" "ditwl-sr-internet-to-front-end-80" {
  security_group_id        = aws_security_group.ditwl-sg-front-end.id
  type                     = "ingress"
  from_port                = 80
  to_port                  = 80
  protocol                 = "tcp"
  cidr_blocks              = ["0.0.0.0/0"] # Internet
}

Back-end Server Security Group: ditwl-sg-back-end

Creates a Security Group named ditwl-sg-back-end with a single rule that uses another Security Group as a source of traffic.

• ditwl-sr-front-end-to-mariadb: Rule to allow access from the ditwl-sg-front-end security group to TCP port 3306.

This Security Group will be assigned to the back-end instance allowing access from the front-end instance (because that instance will have the ditwl-sg-front-end security group assigned) to the MariaDB service.

# Create a Security Group for the Back-end Server
resource "aws_security_group" "ditwl-sg-back-end" {
  name        = "ditwl-sg-back-end"
  vpc_id      = aws_vpc.ditlw-vpc.id
}

# Allow access from the front-end to port 3306 in the back-end (MariaDB)
resource "aws_security_group_rule" "ditwl-sr-front-end-to-mariadb" {
  security_group_id        = aws_security_group.ditwl-sg-back-end.id
  type                     = "ingress"
  from_port                = 3306
  to_port                  = 3306
  protocol                 = "tcp"
  source_security_group_id = aws_security_group.ditwl-sg-front-end.id
}

Private Key and AMIs #

Create a Private Key Pair for SSH Instance Authentication (see how to create the file in Generating and using AWS Key Pairs with Terraform or OpenTofu). The public key of the pair will be included by AWS in the instance authorized access keys while running the initial cloud-init/cloud-config script.

Ansible will use the same Private Key pair to access the instances for configuration from the management node.

Find the AMIs for Ubuntu 22.04 ARM64 Server and Ubuntu 23.04 ARM64 Minimal that will be later used to spin up two EC2 instances (Ubuntu 23.04 has reached its end of life; use a supported release as explained in AWS AMIs).

See Terraform EC2 creation below for private key and AMI assignment.

# Upload a Private Key Pair for SSH Instance Authentication
resource "aws_key_pair" "ditwl-kp-config-user" {
  key_name   = "ditwl-kp-config-user"
  public_key = file("~/keys/ditwl-kp-config-user-ecdsa.pub")
}

#Find AMI Ubuntu 22.04 ARM64 Server
data "aws_ami" "ubuntu-22-04-arm64-server" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-arm64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}

#Find AMI Ubuntu 23.04 ARM64 Minimal
data "aws_ami" "ubuntu-23-04-arm64-minimal" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu-minimal/images/hvm-ssd/ubuntu-lunar-23.04-arm64-minimal-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}

Create the EC2 Instances #

Two EC2 instances named ditwl-ec-front-end-001 and ditwl-ec-back-end-123 are created for different purposes with some common characteristics and some differences (e.g. instance types t4g.micro and t4g.small), tags and values are added to further classify and later on differentiate their purpose, instances have the same tags with some value differences.

Later, Ansible will get an inventory with all instances, and their technical characteristics, like the assigned IPs, the Tags, and the assigned values. The inventory values are used to filter and apply different configurations to each set of instances. (e.g. change CPU limits to all instances running Ubuntu as the operating system, remove swap in instances with less than 1G of memory).

In the Ansible inventory, all dashes and dots will be converted to underscore.

# Front end server running Ubuntu 23.04 ARM Minimal.
resource "aws_instance" "ditwl-ec-front-end-001" {
  ami           = data.aws_ami.ubuntu-23-04-arm64-minimal.id
  instance_type = "t4g.micro"
  subnet_id     = aws_subnet.ditwl-sn-za-pro-pub-00.id
  key_name      = "ditwl-kp-config-user"
  vpc_security_group_ids = [aws_security_group.ditwl-sg-base-ec2.id,aws_security_group.ditwl-sg-front-end.id]
  tags = {
    "Name"         = "ditwl-ec-front-end-001"
    "private_name" = "ditwl-ec-front-end-001"
    "public_name"  = "www"
    "app"          = "front-end"
    "app_ver"      = "2.3"
    "os"           = "ubuntu"
    "os_ver"       = "23.04"
    "os_arch"      = "arm64"
    "environment"  = "pro"
    "cost_center"  = "green-department"
  }
}

# Back end server running Ubuntu 22.04 ARM Server.
resource "aws_instance" "ditwl-ec-back-end-123" {
  ami           = data.aws_ami.ubuntu-22-04-arm64-server.id
  instance_type = "t4g.small"
  subnet_id     = aws_subnet.ditwl-sn-za-pro-pub-00.id
  key_name      = "ditwl-kp-config-user"
  vpc_security_group_ids = [aws_security_group.ditwl-sg-base-ec2.id, aws_security_group.ditwl-sg-back-end.id]
  tags = {
    "Name"         = "ditwl-ec-back-end-123"
    "private_name" = "ditwl-ec-back-end-123"
    "public_name"  = "server"
    "app"          = "back-end"
    "app_ver"      = "1.2"
    "os"           = "ubuntu"
    "os_ver"       = "22.04"
    "os_arch"      = "arm64"
    "environment"  = "pro"
    "cost_center"  = "blue-department"
  }
}

Init, Plan, and Apply the Terraform Plan #

Initialize, plan, and apply the Terraform plan:

$ tofu apply
data.aws_ami.ubuntu-23-04-arm64-minimal: Reading...
data.aws_ami.ubuntu-22-04-arm64-server: Reading...
data.aws_ami.ubuntu-23-04-arm64-minimal: Read complete after 2s [id=ami-03c0a7fe7530c7928]
data.aws_ami.ubuntu-22-04-arm64-server: Read complete after 2s [id=ami-05d47d29a4c2d19e1]

OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  + create

OpenTofu will perform the following actions:

  # aws_instance.ditwl-ec-back-end-123 will be created
  + resource "aws_instance" "ditwl-ec-back-end-123" {
      + ami                                  = "ami-05d47d29a4c2d19e1"
      + instance_type                        = "t4g.small"
      + key_name                             = "ditwl-kp-config-user"
      + tags                                 = {
          + "Name"         = "ditwl-ec-back-end-123"
          + "app"          = "back-end"
          + "app_ver"      = "1.2"
          + "cost_center"  = "blue-department"
          + "environment"  = "pro"
          + "os"           = "ubuntu"
          + "os_arch"      = "arm64"
          + "os_ver"       = "22.04"
          + "private_name" = "ditwl-ec-back-end-123"
          + "public_name"  = "server"
        }
     ...
    }

  # aws_instance.ditwl-ec-front-end-001 will be created
  + resource "aws_instance" "ditwl-ec-front-end-001" {
      + ami                                  = "ami-03c0a7fe7530c7928"
      + get_password_data                    = false
      + instance_type                        = "t4g.micro"
      + key_name                             = "ditwl-kp-config-user"
      + source_dest_check                    = true
      + tags                                 = {
          + "Name"         = "ditwl-ec-front-end-001"
          + "app"          = "front-end"
          + "app_ver"      = "2.3"
          + "cost_center"  = "green-department"
          + "environment"  = "pro"
          + "os"           = "ubuntu"
          + "os_arch"      = "arm64"
          + "os_ver"       = "23.04"
          + "private_name" = "ditwl-ec-front-end-001"
          + "public_name"  = "www"
        }
      ... 
    }

  ...

Plan: 16 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  OpenTofu will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes
..
aws_instance.ditwl-ec-front-end-001: Creating...
aws_instance.ditwl-ec-back-end-123: Creating...
aws_instance.ditwl-ec-front-end-001: Still creating... [10s elapsed]
aws_instance.ditwl-ec-back-end-123: Still creating... [10s elapsed]
aws_instance.ditwl-ec-back-end-123: Creation complete after 16s [id=i-08727eee6f8452337]
aws_instance.ditwl-ec-front-end-001: Creation complete after 16s [id=i-0e30e5c5969f14fea]

Apply complete! Resources: 16 added, 0 changed, 0 destroyed.

Configure Ansible Dynamic Inventory #

Ansible dynamic inventory is a feature in Ansible that allows you to generate inventory (host and group information) dynamically from the current cloud state rather than statically defining it in a static inventory file.

The inventory file contains a list of the AWS instances and all their metadata, properties, tags, and values.

Create an AWS Inventory Configuration File #

Create a file named ansible/inventory.aws_ec2.yml (the name can be changed as long as the file ends in .aws_ec2.yml):

The file defines:

• plugin: the name of the plugin being configured amazon.aws.aws_ec2
• cache: Store local information (set to false during development)
• use_ssm_inventory: use metadata from the AWS SSM service to complement the inventory.
• regions: configures the regions to query for available instances, we will only query the us-east-1 region.
• profile: the name of the AWS profile to use for authentication (see Configure AWS CLI Security), ditwl_infradmin.
• keyed_groups: defines the Ansible Inventory Groups to create, we will create groups based on tags, and instance types.
• hostname: the hostname is set to the value of the tag private_name for easy identification and tool traceability (e.g. ditwl-ec-back-end-123 and ditwl-ec-front-end-001). It can include fallback values.
• ansible_host: defines what IP address, hostname, or FQDN to use for SSH access to the instance. We chose to use the public_ip_address as all our tutorial hosts have a public IP. In a production environment hosts will not have a public IP and instead, a VPN is used to access their private IP.

ansible/inventory.aws_ec2.yml

# AWS Ansible Inventory Plugin Configuration File
# See options: https://docs.ansible.com/ansible/latest/collections/amazon/aws/aws_ec2_inventory.html
plugin: amazon.aws.aws_ec2
cache: false
use_ssm_inventory: true

# Limit query to regions
regions:
  - us-east-1

# A named AWS profile to use for authentication.
profile: ditwl_infradmin

# Create Ansible hosts groups using the following elements: 
keyed_groups:
  - prefix: tag
    key: tags
  - prefix: instance_type
    key: instance_type

hostnames:
- tag:private_name

compose:
  ansible_host: public_ip_address

Test the Ansible Dynamic AWS Inventory #

Use the ansible-inventory command to list in a short graph format the AWS hosts using the Ansible Dynamic Inventory plugin for AWS:

$ ansible-inventory -i inventory.aws_ec2.yml --graph
@all:
  |--@ungrouped:
  |--@aws_ec2:
  |  |--ditwl-ec-back-end-123
  |  |--ditwl-ec-front-end-001
  |--@tag_environment_pro:
  |  |--ditwl-ec-back-end-123
  |  |--ditwl-ec-front-end-001
  |--@tag_cost_center_blue_department:
  |  |--ditwl-ec-back-end-123
  |--@tag_os_ver_22_04:
  |  |--ditwl-ec-back-end-123
  |--@tag_private_name_ditwl_ec_back_end_123:
  |  |--ditwl-ec-back-end-123
  |--@tag_os_ubuntu:
  |  |--ditwl-ec-back-end-123
  |  |--ditwl-ec-front-end-001
  |--@tag_Name_ditwl_ec_back_end_123:
  |  |--ditwl-ec-back-end-123
  |--@tag_app_back_end:
  |  |--ditwl-ec-back-end-123
  |--@tag_app_ver_1_2:
  |  |--ditwl-ec-back-end-123
  |--@tag_os_arch_arm64:
  |  |--ditwl-ec-back-end-123
  |  |--ditwl-ec-front-end-001
  |--@tag_public_name_server:
  |  |--ditwl-ec-back-end-123
  |--@instance_type_t4g_small:
  |  |--ditwl-ec-back-end-123
  |--@tag_private_name_ditwl_ec_front_end_001:
  |  |--ditwl-ec-front-end-001
  |--@tag_public_name_www:
  |  |--ditwl-ec-front-end-001
  |--@tag_Name_ditwl_ec_front_end_001:
  |  |--ditwl-ec-front-end-001
  |--@tag_cost_center_green_department:
  |  |--ditwl-ec-front-end-001
  |--@tag_os_ver_23_04:
  |  |--ditwl-ec-front-end-001
  |--@tag_app_front_end:
  |  |--ditwl-ec-front-end-001
  |--@tag_app_ver_2_3:
  |  |--ditwl-ec-front-end-001
  |--@instance_type_t4g_micro:
  |  |--ditwl-ec-front-end-001

The above list shows all the groups that will be available for Ansible Playbooks as host selectors.

In the Ansible inventory, all dots and dashes in the AWS tags and their values are converted to underscore, except for the hostnames.

For example:

• The group tag_os_ubuntu includes all EC2 instances that run Ubuntu as the operating system (set by Terraform in the tag “os” with the value “ubuntu”).
• The group tag_os_arch_arm64 lists all hosts running ARM 64 architecture.
• The group boolean combination tag_os_ubuntu & tag_os_ver_22_04 includes all EC2 instances running Ubuntu version 22.04, in this example only the instance ditwl-ec-back-end-123

Create a Script to set the Ansible Environment #

Ansible commands require specific configurations that can be set in different locations:

• ANSIBLE_CONFIG (environment variable if set)
• ansible.cfg (in the current directory)
• ~/.ansible.cfg (in the home directory)
• /etc/ansible/ansible.cfg

Additionally, many configuration settings can be set using session environment variables.

For this tutorial, we use environment variables as this approach allows code portability (e.g. to be included in automated CI/CD DevOps flows).

ANSIBLE INVENTORY

Set the environment variable ANSIBLE_INVENTORY pointing to the AWS Inventory configuration file.

export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml

SSH PRIVATE KEY

Ansible requires SSH access to the instances and therefore the authentication is performed using a private key. We will use the private key pair created for the AWS instances (see Generating and using AWS Key Pairs with Terraform or OpenTofu).

export ANSIBLE_PRIVATE_KEY_FILE=${HOME}/keys/ditwl-kp-config-user-ecdsa

Previous Ansible versions accepted the environment variable ANSIBLE_PRIVATE_KEY, which is no longer supported: use ANSIBLE_PRIVATE_KEY_FILE.

Write the Ansible Configuration in a script file #

For convenience write the configuration instructions in a script file as Ansible needs to be configured every time a new shell session is started.

Review the configuration from the file ansible/ditwl_pro.sh:

# source this file before using ansible
# source ditwl_pro.sh

#Set Private KEY for SSH
export ANSIBLE_PRIVATE_KEY_FILE=${HOME}/keys/ditwl-kp-config-user-ecdsa

#Set Ansible Inventory configuration
export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml

echo ANSIBLE_PRIVATE_KEY_FILE=$ANSIBLE_PRIVATE_KEY_FILE
echo ANSIBLE_INVENTORY=$ANSIBLE_INVENTORY

Before running any ansible commands, source the configuration file ditwl_pro.sh to set the appropriate environment variables.

$ source ditwl_pro.sh
ANSIBLE_PRIVATE_KEY_FILE=/home/user/keys/ditwl-kp-config-user-ecdsa
ANSIBLE_INVENTORY=/home/user/git/.../ansible/inventory.aws_ec2.yml

Access Detailed EC2 Inventory Host Information #

Get detailed inventory host information by running ansible-inventory for a specific host or review the whole inventory.

All this host info can be used inside Ansible Playbooks for selecting target hosts and for configuration.

Single host Inventory #

$ ansible-inventory -i inventory.aws_ec2.yml --host ditwl-ec-front-end-001
{
    "ami_launch_index": 0,
    "ansible_host": "18.234.149.2",
    "architecture": "arm64",
    "block_device_mappings": [
        {
            "device_name": "/dev/sda1",
            "ebs": {
                "attach_time": "2023-12-13T09:02:55+00:00",
                "delete_on_termination": true,
                "status": "attached",
                "volume_id": "vol-0c1547809a44aa5cd"
            }
        }
    ],
    "boot_mode": "uefi",
    "capacity_reservation_specification": {
        "capacity_reservation_preference": "open"
    },
    "client_token": "terraform-20231213090254327700000001",
    "cpu_options": {
        "core_count": 2,
        "threads_per_core": 1
    },
    "ebs_optimized": false,
    "ena_support": true,
    "enclave_options": {
        "enabled": false
    },
    "hibernation_options": {
        "configured": false
    },
    "hypervisor": "xen",
    "image_id": "ami-03c0a7fe7530c7928",
    "instance_id": "i-0e30e5c5969f14fea",
    "instance_type": "t4g.micro",
    "key_name": "ditwl-kp-config-user",
    "launch_time": "2023-12-13T09:02:55+00:00",
    "metadata_options": {
        "http_endpoint": "enabled",
        "http_protocol_ipv6": "disabled",
        "http_put_response_hop_limit": 1,
        "http_tokens": "optional",
        "instance_metadata_tags": "disabled",
        "state": "applied"
    },
    "monitoring": {
        "state": "disabled"
    },
    "network_interfaces": [
        {
            "association": {
                "ip_owner_id": "amazon",
                "public_dns_name": "",
                "public_ip": "18.234.149.2"
            },
            "attachment": {
                "attach_time": "2023-12-13T09:02:55+00:00",
                "attachment_id": "eni-attach-09234e6992f5e5c39",
                "delete_on_termination": true,
                "device_index": 0,
                "network_card_index": 0,
                "status": "attached"
            },
            "description": "",
            "groups": [
                {
                    "group_id": "sg-095f8ebf532627f53",
                    "group_name": "ditwl-sg-front-end"
                },
                {
                    "group_id": "sg-06293ee469299179f",
                    "group_name": "ditwl-sg-ssh-ec2"
                }
            ],
            "interface_type": "interface",
            "ipv6_addresses": [],
            "mac_address": "0a:8a:84:0f:aa:f7",
            "network_interface_id": "eni-004e9402ef4e6e0ca",
            "owner_id": "431960152202",
            "private_ip_address": "172.21.1.133",
            "private_ip_addresses": [
                {
                    "association": {
                        "ip_owner_id": "amazon",
                        "public_dns_name": "",
                        "public_ip": "18.234.149.2"
                    },
                    "primary": true,
                    "private_ip_address": "172.21.1.133"
                }
            ],
            "source_dest_check": true,
            "status": "in-use",
            "subnet_id": "subnet-0e0726ade683c52af",
            "vpc_id": "vpc-03546f2eaca41aead"
        }
    ],
    "owner_id": "431960152202",
    "placement": {
        "availability_zone": "us-east-1c",
        "group_name": "",
        "region": "us-east-1",
        "tenancy": "default"
    },
    "platform_details": "Linux/UNIX",
    "private_dns_name": "ip-172-21-1-133.ec2.internal",
    "private_dns_name_options": {
        "enable_resource_name_dns_a_record": false,
        "enable_resource_name_dns_aaaa_record": false,
        "hostname_type": "ip-name"
    },
    "private_ip_address": "172.21.1.133",
    "product_codes": [],
    "public_dns_name": "",
    "public_ip_address": "18.234.149.2",
    "requester_id": "",
    "reservation_id": "r-0a681315893f5676c",
    "root_device_name": "/dev/sda1",
    "root_device_type": "ebs",
    "security_groups": [
        {
            "group_id": "sg-095f8ebf532627f53",
            "group_name": "ditwl-sg-front-end"
        },
        {
            "group_id": "sg-06293ee469299179f",
            "group_name": "ditwl-sg-ssh-ec2"
        }
    ],
    "source_dest_check": true,
    "state": {
        "code": 16,
        "name": "running"
    },
    "state_transition_reason": "",
    "subnet_id": "subnet-0e0726ade683c52af",
    "tags": {
        "Name": "ditwl-ec-front-end-001",
        "app": "front-end",
        "app_ver": "2.3",
        "cost_center": "green-department",
        "environment": "pro",
        "os": "ubuntu",
        "os_arch": "arm64",
        "os_ver": "23.04",
        "private_name": "ditwl-ec-front-end-001",
        "public_name": "www"
    },
    "usage_operation": "RunInstances",
    "usage_operation_update_time": "2023-12-13T09:02:55+00:00",
    "virtualization_type": "hvm",
    "vpc_id": "vpc-03546f2eaca41aead"
}

Ansible AWS Full Inventory #

Access all inventory data for the AWS infrastructure with the ansible-inventory --list command. As it can be a long list of properties, it is recommended to pipe the output to a file (inventory.json).

$ ansible-inventory --list > inventory.json

Inside the inventory and for each EC2 instance, a node is added to hostvars, the node contains properties and groups that will be available when running a playbook.

Example from a different tutorial showing two different groups holding PRE and PRO environment instances:

{
  "_meta": {
  ...
  "tag_environment_pre": [
    "ditwl_ec2_pre_pub_abc01_1",
    "ditwl_ec2_pre_pub_abc01_2",
    "ditwl_ec2_pre_pub_abc01_3"
  ],
  "tag_environment_pro": [
    "ditwl_ec2_pro_pub_wp01_1"
  ]
  ...
}

Create the Ansible Playbook #

The Ansible Playbook is a YAML file that defines a set of tasks to be executed on one or more hosts.

Hosts are selected using Inventory Groups and Metadata, pattern selection allows for complex host selection.

Testing Playbook #

The file ansible/testing.playbook.yaml is used for testing host group selection.

- name: All hosts
  hosts: all
  remote_user: ubuntu 
  tasks:
   - name: Ping all
     ansible.builtin.ping:

- name: One host, by name
  hosts: ditwl-ec-front-end-001
  remote_user: ubuntu 
  tasks:
   - name: Ping 
     ansible.builtin.ping:     

- name: Multiple Hosts, by name
  hosts: ditwl-ec-front-end-001:ditwl-ec-back-end-123
  remote_user: ubuntu
  tasks:
   - name: Ping 
     ansible.builtin.ping:

- name: One group, hosts in group Cost Center blue_department (AWS hosts with tag cost_center=blue-department)
  hosts: tag_cost_center_blue_department
  remote_user: ubuntu
  tasks:
   - name: Ping 
     ansible.builtin.ping:   

- name: Multiple groups, hosts in groups Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue-department or cost_center=green_department)
  hosts: tag_cost_center_blue_department:tag_cost_center_green_department
  remote_user: ubuntu
  tasks:
   - name: Ping 
     ansible.builtin.ping:        

- name: Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g_small)
  hosts: tag_environment_pro:!instance_type_t4g_small
  remote_user: ubuntu
  tasks:
   - name: Ping 
     ansible.builtin.ping:        

- name: Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04)
  hosts: tag_os_ubuntu:&tag_os_ver_23_04
  remote_user: ubuntu
  tasks:
   - name: Ping
     ansible.builtin.ping:

Ansible Hosts Pattern Testing #

Show the list of matching hosts for the patterns but not executing the playbook:

$ ansible-playbook testing.playbook.yaml --list-hosts

playbook: testing.playbook.yaml

  play #1 (all): All hosts      TAGS: []
    pattern: ['all']
    hosts (2):
      ditwl-ec-front-end-001
      ditwl-ec-back-end-123

  play #2 (ditwl-ec-front-end-001): One host, by name   TAGS: []
    pattern: ['ditwl-ec-front-end-001']
    hosts (1):
      ditwl-ec-front-end-001

  play #3 (ditwl-ec-front-end-001:ditwl-ec-back-end-123): Multiple Hosts, by name       TAGS: []
    pattern: ['ditwl-ec-front-end-001:ditwl-ec-back-end-123']
    hosts (2):
      ditwl-ec-front-end-001
      ditwl-ec-back-end-123

  play #4 (tag_cost_center_blue_department): One group, hosts in Cost Center blue_department (AWS hosts with tag cost_center=blue_department)   TAGS: []
    pattern: ['tag_cost_center_blue_department']
    hosts (1):
      ditwl-ec-back-end-123

  play #5 (tag_cost_center_blue_department:tag_cost_center_green_department): Multiple groups, hosts in Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue_department or cost_center=green_department)    TAGS: []
    pattern: ['tag_cost_center_blue_department:tag_cost_center_green_department']
    hosts (2):
      ditwl-ec-front-end-001
      ditwl-ec-back-end-123

  play #6 (tag_environment_pro:!instance_type_t4g_small): Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g_small)      TAGS: []
    pattern: ['tag_environment_pro:!instance_type_t4g_small']
    hosts (1):
      ditwl-ec-front-end-001

  play #7 (tag_os_ubuntu:&tag_os_ver_23_04): Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04) TAGS: []
    pattern: ['tag_os_ubuntu:&tag_os_ver_23_04']
    hosts (1):
      ditwl-ec-front-end-001

Run the Ansible Test Playbook #

Run the playbook to test SSH connectivity:

$ ansible-playbook testing.playbook.yaml

PLAY [All hosts] **********************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]
ok: [ditwl-ec-back-end-123]

TASK [Ping all] ***********************************************************************************************************************************************************************
ok: [ditwl-ec-back-end-123]
ok: [ditwl-ec-front-end-001]

PLAY [One host, by name] **************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]

TASK [Ping] ***************************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]

PLAY [Multiple Hosts, by name] ********************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]
ok: [ditwl-ec-back-end-123]

TASK [Ping] ***************************************************************************************************************************************************************************
ok: [ditwl-ec-back-end-123]
ok: [ditwl-ec-front-end-001]

PLAY [One group, hosts in Cost Center blue_department (AWS hosts with tag cost_center=blue-department)] *******************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ditwl-ec-back-end-123]

TASK [Ping] ***************************************************************************************************************************************************************************
ok: [ditwl-ec-back-end-123]

PLAY [Multiple groups, hosts in Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue-department or cost_center=green-department)] ******************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]
ok: [ditwl-ec-back-end-123]

TASK [Ping] ***************************************************************************************************************************************************************************
ok: [ditwl-ec-back-end-123]
ok: [ditwl-ec-front-end-001]

PLAY [Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g.small)] ****************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]

TASK [Ping] ***************************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]

PLAY [Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04)] *************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]

TASK [Ping] ***************************************************************************************************************************************************************************
ok: [ditwl-ec-front-end-001]

PLAY RECAP ****************************************************************************************************************************************************************************
ditwl-ec-back-end-123      : ok=8    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
ditwl-ec-front-end-001     : ok=12   changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Troubleshooting #

Empty Ansible Inventory #

Ansible shows an empty inventory or the hosts list is empty when it has not been properly configured to use a dynamic inventory.

$ ansible-inventory --list
{
    "_meta": {
        "hostvars": {}
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    }
}

$ ansible-playbook playbook.yaml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Check that the ANSIBLE_INVENTORY environment variable has been properly configured to point to the dynamic inventory configuration file and the file name ends in .aws_ec2.yml.

export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml

Make sure that the dynamic inventory is working, see the section on how to test the Ansible dynamic directory.

Unprotected Private Key File #

Ansible checks that the Private Key file permissions are properly configured.

WARNING: UNPROTECTED PRIVATE KEY FILE!
Permissions 0660 for '/home/user/keys/ditwl-kp-config-user-ecdsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key \"/home/user/keys/ditwl-kp-config-user-ecdsa\": bad permissions\
Permission denied (publickey)

See How to set the correct permissions for the Ansible Private Key file.

The authenticity of host ‘IP/NAME’ can’t be established #

When running an ansible-playbook for the first time, Ansible opens an SSH connection to all targeted hosts.

The message “The authenticity of host can’t be established” in SSH occurs when you try to connect to a server for the first time or if the server’s host key has changed. This message serves as a security measure to warn you about a potential man-in-the-middle attack.

When you connect to a server for the first time, your SSH client doesn’t have the server’s host key stored locally. This host key acts like a digital fingerprint, uniquely identifying the server. Without this stored key, your client cannot verify the server’s identity and prevent the possibility of connecting to an imposter server.

The authenticity of host '3.94.214.81 (3.94.214.81)' can't be established.
ED25519 key fingerprint is SHA256:mIBL6hfMMeEuaoxW+UL8JV3HaKShxFFLcgEYXaVD/3c.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

If you trust that the connection is being done to the correct server, answer yes to add the key fingerprint to your known_hosts file.

Instance type (name) is not supported in your requested Availability Zone #

When a minimal Terraform configuration doesn’t specify the availability zone for subnets, there is a possibility of randomly getting assigned an availability zone that doesn’t have all AWS Services or instance types.

│ Error: creating EC2 Instance: Unsupported: Your requested instance type (t4g.micro) is not supported in your requested Availability Zone (us-east-1e). Please retry your request by not specifying an Availability Zone or choosing us-east-1a, us-east-1b, us-east-1c, us-east-1d, us-east-1f.

Set the availability zone for the subnet and run again.

# Subnet
resource "aws_subnet" "ditwl-sn-za-pro-pub-00" {
  vpc_id                  = aws_vpc.ditlw-vpc.id
  cidr_block              = "172.21.0.0/23" #172.21.0.0 - 172.21.1.255
  map_public_ip_on_launch = true
  availability_zone = "us-east-1c"
  tags = {
    Name = "ditwl-sn-za-pro-pub-00"
  }
}