Terraform is an Infrastructure as Code (IaC) tool used to create infrastructure and should never be used to configure operating systems or applications, instead, Ansible is the de facto standard for the operating system and application configuration as code.
This tutorial shows how to create infrastructure in AWS using Terraform and configure the operating system and applications using Ansible.
Linking the infrastructure created in AWS with Terraform is done using infrastructure tags.
How to use Terraform, AWS, and Ansible Together
Install Terraform or Open Tofu and Ansible.
Define the Infrastructure Tags
All the resources created in AWS will have a set of company-wide tags to identify each resource across applications (cloud, IaC, accounting, departments...).
Create the Infrastructure in AWS using Terraform
The tutorial includes basic Terraform code to create an AWS VPC, one subnet, an Internet Gateway, a main routing table, security rules, a private key pair, and two EC2 instances with tags.
Configure Ansible AWS Dynamic Inventory
Configure the AWS dynamic inventory plugin.
Create a bash script file to set common environment variables for Ansible.
Access Detailed EC2 Inventory Host Information
Get detailed inventory host information by running an ansible-inventory for a specific host or reviewing the whole inventory.
The Ansible Playbook is a YAML file that defines a set of tasks to be executed on one or more hosts. Hosts are selected using patterns based on the Dynamic Inventory Groups.
Empty Ansible Inventory, Permissions too open, not supported in your requested Availability Zone.
All the AWS resources created with Terraform will have tags that follow a company-wide standard. Tags are used for provisioning, monitoring, and cost control. Check AWS Resource Tagging best practices.
| Tag Name | Description | Example |
|---|---|---|
| Name | Unique name for the instance across the organization and the world (in case of mergers) | ditwl-ec-front-end-001 |
| private_name | The name assigned in the private DNS, Has detailed information and is usually the Name. | ditwl-ec-front-end-001 |
| public_name | The name assigned in the public DNS (hides releases, can be empty if not published on external DNS) | www |
| app | Name of the application running on the instance | front-end |
| app_ver | The version of the application that should be deployed in the Instance | 1.2 |
| os | Operating System of the Instance | ubuntu |
| os_ver | Release of the Operating System | 22.04 |
| os_arch | Hardware architecture of the Instance (and Operating System) | arm64 |
| environment | Environment served by the application, usually dev, test, pre/qa, pro | pro |
| cost_center | Cost center assigned for the instance (who pays the instance cost) | blue-department |
WARNING: The infrastructure demonstrated in this Terraform and Ansible tutorial has been extremely simplified for educational purposes only and is not intended for production use. It contains numerous security and design flaws that would pose significant risks in a real-world environment. Running the tutorial may incur costs for cloud resources, and it is the user's responsibility to manage and terminate resources to avoid unnecessary charges.
Please be aware of the following limitations and risks:
Before proceeding with this tutorial:
By proceeding with this tutorial, you acknowledge the risks and limitations outlined above and agree to take full responsibility for your actions.
The Terraform file terraform/aws-basic-vpc-ec2-tags.tf contains all the code necessary to create the AWS infrastructure. You can either download the code from the IT Wonder Lab GitHub repository or create the files manually.
The following sections show the Terraform code used to create each AWS element.
terraform/aws-basic-vpc-ec2-tags.tf
Defines the required Terraform version and the AWS required provider.
# Copyright (C) 2018 - 2023 IT Wonder Lab (https://www.itwonderlab.com)
#
# This software may be modified and distributed under the terms
# of the MIT license. See the LICENSE file for details.
# -------------------------------- WARNING --------------------------------
# IT Wonder Lab's best practices for infrastructure include modularizing
# Terraform/OpenTofu configuration.
# In this example, we define everything in a single file.
# See other tutorials for best practices at itwonderlab.com
# -------------------------------- WARNING --------------------------------
#Define Terrraform Providers and Backend
terraform {
required_version = "> 1.5"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
#-----------------------------------------
# Default provider: AWS
#-----------------------------------------
provider "aws" {
shared_credentials_files = ["~/.aws/credentials"]
profile = "ditwl_infradmin"
region = "us-east-1"
}Create a VPC named ditlw-vpc, a single subnet name ditwl-sn-za-pro-pub-00 configured as a public subnet to assign Public IPs to its EC2 instances.
An Internet Gateway, named ditwl-ig, is created to allow EC2 instances with public IPs to access and to the Internet.
A Routing table named ditwl-rt-pub-main, is created and assigned as the main or default routing table for the VPC, it includes a route to send all traffic directed to the Internet through the Internet Gateway.
# VPC
resource "aws_vpc" "ditlw-vpc" {
cidr_block = "172.21.0.0/19" #172.21.0.0 - 172.21.31.254
tags = {
Name = "ditlw-vpc"
}
}
# Subnet
resource "aws_subnet" "ditwl-sn-za-pro-pub-00" {
vpc_id = aws_vpc.ditlw-vpc.id
cidr_block = "172.21.0.0/23" #172.21.0.0 - 172.21.1.255
map_public_ip_on_launch = true
tags = {
Name = "ditwl-sn-za-pro-pub-00"
}
}
# Internet Gateway
resource "aws_internet_gateway" "ditwl-ig" {
vpc_id = aws_vpc.ditlw-vpc.id
tags = {
Name = "ditwl-ig"
}
}
# Routing table for public subnet (access to Internet)
resource "aws_route_table" "ditwl-rt-pub-main" {
vpc_id = aws_vpc.ditlw-vpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.ditwl-ig.id
}
tags = {
Name = "ditwl-rt-pub-main"
}
}
# Set new main_route_table as main
resource "aws_main_route_table_association" "ditwl-rta-default" {
vpc_id = aws_vpc.ditlw-vpc.id
route_table_id = aws_route_table.ditwl-rt-pub-main.id
}AWS Security groups hold the security rules that allow ingress and egress traffic to and from the EC2 instances. In this example, a basic group will be created to be assigned to all instances and the other two security groups will hold the rules specific to each instance.
Creates a Security Group name ditwl-sg-base-ec2 and the following rules:
ditwl-sr-internet-to-ec2-ssh: Rule Allow SSH to EC2 instances (ingress).ditwl-sr-internet-to-ec2-icmp: Allow ICMP (Ping) from the Internet into the instances (ingress).ditwl-sr-all-outbund: Allow all protocols from the instances to the Internet (egress). Be aware that SSH access from the Internet is dangerous and is only used for demo simplicity.
The Base Security Group will be assigned to all the EC2 instances.
# Create a "base" Security Group to be assigned to all EC2 instances
resource "aws_security_group" "ditwl-sg-base-ec2" {
name = "ditwl-sg-ssh-ec2"
vpc_id = aws_vpc.ditlw-vpc.id
}
# DANGEROUS!!
# Allow access from the Internet to port 22 (SSH) in the EC2 instances
resource "aws_security_group_rule" "ditwl-sr-internet-to-ec2-ssh" {
security_group_id = aws_security_group.ditwl-sg-base-ec2.id
type = "ingress"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # Internet
}
# Allow access from the Internet for ICMP protocol (e.g. ping) to the EC2 instances
resource "aws_security_group_rule" "ditwl-sr-internet-to-ec2-icmp" {
security_group_id = aws_security_group.ditwl-sg-base-ec2.id
type = "ingress"
from_port = -1
to_port = -1
protocol = "icmp"
cidr_blocks = ["0.0.0.0/0"] # Internet
}
# Allow all outbound traffic to Internet
resource "aws_security_group_rule" "ditwl-sr-all-outbund" {
security_group_id = aws_security_group.ditwl-sg-base-ec2.id
type = "egress"
from_port = "0"
to_port = "0"
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}Creates a Security Group name ditwl-sg-front-end with a single rule:
ditwl-sr-internet-to-front-end-80: Rule to allow access from the Internet to TCP port 80.This Security Group will be assigned to the front-end instance allowing access from the Internet to the instance port 80 (HTTP web server).
# Create a Security Group for the Front end Server
resource "aws_security_group" "ditwl-sg-front-end" {
name = "ditwl-sg-front-end"
vpc_id = aws_vpc.ditlw-vpc.id
}
# Allow access from the Internet to port 80 in the EC2 instances
resource "aws_security_group_rule" "ditwl-sr-internet-to-front-end-80" {
security_group_id = aws_security_group.ditwl-sg-front-end.id
type = "ingress"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # Internet
}Creates a Security Group name ditwl-sg-back-end with a single rule that uses another Security Group as a source of traffic.
ditwl-sr-front-end-to-mariadb: Rule to allow access from the ditwl-sg-front-end security group to TCP port 3306.This Security Group will be assigned to the back-end instance allowing access from the front-end instance (because that instance will have the ditwl-sg-front-end security group assigned) to the MariaDB service.
# Create a Security Group for the Back-end Server
resource "aws_security_group" "ditwl-sg-back-end" {
name = "ditwl-sg-back-end"
vpc_id = aws_vpc.ditlw-vpc.id
}
# Allow access from the front-end to port 3306 in the back-end (MariaDB)
resource "aws_security_group_rule" "ditwl-sr-front-end-to-mariadb" {
security_group_id = aws_security_group.ditwl-sg-back-end.id
type = "ingress"
from_port = 3306
to_port = 3306
protocol = "tcp"
source_security_group_id = aws_security_group.ditwl-sg-front-end.id
}Create a Private Key Pair for SSH Instance Authentication (see how to create the file in Generating and using AWS Key Pairs with Terraform or OpenTofu). The public key of the pair will be included by AWS in the instance authorized access keys while running the initial cloud-init/cloud-config script.
Ansible will use the same Private Key pair to access the instances for configuration from the management node.
Find the AMIs for Ubuntu 22.04 ARM64 Server and Ubuntu 23.04 ARM64 Minimal that will be later used to spin up two EC2 instances.
See Terraform EC2 creation below for private key and AMI assignment.
# Upload a Private Key Pair for SSH Instance Authentication
resource "aws_key_pair" "ditwl-kp-config-user" {
key_name = "ditwl-kp-config-user"
public_key = file("~/keys/ditwl-kp-config-user-ecdsa.pub")
}
#Find AMI Ubuntu 22.04 ARM64 Server
data "aws_ami" "ubuntu-22-04-arm64-server" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-arm64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"] # Canonical
}
#Find AMI Ubuntu 23.04 ARM64 Minimal
data "aws_ami" "ubuntu-23-04-arm64-minimal" {
most_recent = true
filter {
name = "name"
values = ["ubuntu-minimal/images/hvm-ssd/ubuntu-lunar-23.04-arm64-minimal-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"] # Canonical
}
Two EC2 instances named ditwl-ec-front-end-001 and ditwl-ec-back-end-123 are created for different purposes with some common characteristics and some differences (e.g. instant type t4g.micro and t4g.small), tags and values are added to further classify and later on differentiate their purpose, instances have the same tags with some value differences.
Later, Ansible will get an inventory with all instances, and their technical characteristics, like the assigned IPs, the Tags, and the assigned values. The inventory values are used to filter and apply different configurations to each set of instances. (e.g. change CPU limits to all instances running Ubuntu as the operating system, remove swap in instances with less than 1G of memory).
In the Ansible inventory, all dashes and dots will be converted to underscore.
| Tag Name | ditwl-ec-front-end-001 Front-end server | ditwl-ec-back-end-123 Back-end server |
|---|---|---|
| Name | ditwl-ec-front-end-001 | ditwl-ec-back-end-123 |
| private_name | ditwl-ec-front-end-001 | ditwl-ec-back-end-123 |
| public_name | www | server |
| app | front-end | back-end |
| app_ver | 2.3 | 1.2 |
| os | ubuntu | ubuntu |
| os_ver | 23.04 | 22.04 |
| os_arch | arm64 | arm64 |
| environment | pro | pro |
| cost_center | green-department | blue-department |
# Front end server running Ubuntu 23.04 ARM Minimal.
resource "aws_instance" "ditwl-ec-front-end-001" {
ami = data.aws_ami.ubuntu-23-04-arm64-minimal.id
instance_type = "t4g.micro"
subnet_id = aws_subnet.ditwl-sn-za-pro-pub-00.id
key_name = "ditwl-kp-config-user"
vpc_security_group_ids = [aws_security_group.ditwl-sg-base-ec2.id,aws_security_group.ditwl-sg-front-end.id]
tags = {
"Name" = "ditwl-ec-front-end-001"
"private_name" = "ditwl-ec-front-end-001"
"public_name" = "www"
"app" = "front-end"
"app_ver" = "2.3"
"os" = "ubuntu"
"os_ver" = "23.04"
"os_arch" = "arm64"
"environment" = "pro"
"cost_center" = "green-department"
}
}
# Back end server running Ubuntu 22.04 ARM Server.
resource "aws_instance" "ditwl-ec-back-end-123" {
ami = data.aws_ami.ubuntu-22-04-arm64-server.id
instance_type = "t4g.small"
subnet_id = aws_subnet.ditwl-sn-za-pro-pub-00.id
key_name = "ditwl-kp-config-user"
vpc_security_group_ids = [aws_security_group.ditwl-sg-base-ec2.id, aws_security_group.ditwl-sg-back-end.id]
tags = {
"Name" = "ditwl-ec-back-end-123"
"private_name" = "ditwl-ec-back-end-123"
"public_name" = "server"
"app" = "back-end"
"app_ver" = "1.2"
"os" = "ubuntu"
"os_ver" = "22.04"
"os_arch" = "arm64"
"environment" = "pro"
"cost_center" = "blue-department"
}
}Initialize, plan, and apply the Terraform plan:
$ tofu apply
data.aws_ami.ubuntu-23-04-arm64-minimal: Reading...
data.aws_ami.ubuntu-22-04-arm64-server: Reading...
data.aws_ami.ubuntu-23-04-arm64-minimal: Read complete after 2s [id=ami-03c0a7fe7530c7928]
data.aws_ami.ubuntu-22-04-arm64-server: Read complete after 2s [id=ami-05d47d29a4c2d19e1]
OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
+ create
OpenTofu will perform the following actions:
# aws_instance.ditwl-ec-back-end-123 will be created
+ resource "aws_instance" "ditwl-ec-back-end-123" {
+ ami = "ami-05d47d29a4c2d19e1"
+ instance_type = "t4g.small"
+ key_name = "ditwl-kp-config-user"
+ tags = {
+ "Name" = "ditwl-ec-back-end-123"
+ "app" = "back-end"
+ "app_ver" = "1.2"
+ "cost_center" = "blue-department"
+ "environment" = "pro"
+ "os" = "ubuntu"
+ "os_arch" = "arm64"
+ "os_ver" = "22.04"
+ "private_name" = "ditwl-ec-back-end-123"
+ "public_name" = "server"
}
...
}
# aws_instance.ditwl-ec-front-end-001 will be created
+ resource "aws_instance" "ditwl-ec-front-end-001" {
+ ami = "ami-03c0a7fe7530c7928"
+ get_password_data = false
+ instance_type = "t4g.micro"
+ key_name = "ditwl-kp-config-user"
+ source_dest_check = true
+ tags = {
+ "Name" = "ditwl-ec-front-end-001"
+ "app" = "front-end"
+ "app_ver" = "2.3"
+ "cost_center" = "green-department"
+ "environment" = "pro"
+ "os" = "ubuntu"
+ "os_arch" = "arm64"
+ "os_ver" = "23.04"
+ "private_name" = "ditwl-ec-front-end-001"
+ "public_name" = "www"
}
...
}
...
Plan: 16 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
OpenTofu will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
..
aws_instance.ditwl-ec-front-end-001: Creating...
aws_instance.ditwl-ec-back-end-123: Creating...
aws_instance.ditwl-ec-front-end-001: Still creating... [10s elapsed]
aws_instance.ditwl-ec-back-end-123: Still creating... [10s elapsed]
aws_instance.ditwl-ec-back-end-123: Creation complete after 16s [id=i-08727eee6f8452337]
aws_instance.ditwl-ec-front-end-001: Creation complete after 16s [id=i-0e30e5c5969f14fea]
Apply complete! Resources: 16 added, 0 changed, 0 destroyed.Ansible dynamic inventory is a feature in Ansible that allows you to generate inventory (host and group information) dynamically from the current cloud state rather than statically defining it in a static inventory file.
The inventory file contains a list of the AWS instances and all their metadata, properties, tags, and values.
Create a file named ansible/inventory.aws_ec2.yml (the name can be changed as long as the file ends in .aws_ec2.yml):
The file defines:
amazon.aws.aws_ec2 us-east-1 region.ditwl_infradmin.private_name for easy identification and tool traceability (e.g. ditwl-ec-back-end-123 and ditwl-ec-front-end-001). It can include fallback values.public_ip_address as all our tutorial hosts have a public IP. In a production environment hosts will not have a public IP and instead, a VPN is used to access their private IP.ansible/inventory.aws_ec2.yml
# AWS Ansible Inventory Plugin Configuration File
# See options: https://docs.ansible.com/ansible/latest/collections/amazon/aws/aws_ec2_inventory.html
plugin: amazon.aws.aws_ec2
cache: false
use_ssm_inventory: true
# Limit query to regions
regions:
- us-east-1
# A named AWS profile to use for authentication.
profile: ditwl_infradmin
# Create Ansible hosts groups using the following elements:
keyed_groups:
- prefix: tag
key: tags
- prefix: instance_type
key: instance_type
hostnames:
- tag:private_name
compose:
ansible_host: public_ip_address
Use the ansible-inventory command to list in a short graph format the AWS hosts using tha Ansible Dynamic Inventory plugin for AWS:
$ ansible-inventory -i inventory.aws_ec2.yml --graph @all: |--@ungrouped: |--@aws_ec2: | |--ditwl-ec-back-end-123 | |--ditwl-ec-front-end-001 |--@tag_environment_pro: | |--ditwl-ec-back-end-123 | |--ditwl-ec-front-end-001 |--@tag_cost_center_blue_department: | |--ditwl-ec-back-end-123 |--@tag_os_ver_22_04: | |--ditwl-ec-back-end-123 |--@tag_private_name_ditwl_ec_back_end_123: | |--ditwl-ec-back-end-123 |--@tag_os_ubuntu: | |--ditwl-ec-back-end-123 | |--ditwl-ec-front-end-001 |--@tag_Name_ditwl_ec_back_end_123: | |--ditwl-ec-back-end-123 |--@tag_app_back_end: | |--ditwl-ec-back-end-123 |--@tag_app_ver_1_2: | |--ditwl-ec-back-end-123 |--@tag_os_arch_arm64: | |--ditwl-ec-back-end-123 | |--ditwl-ec-front-end-001 |--@tag_public_name_server: | |--ditwl-ec-back-end-123 |--@instance_type_t4g_small: | |--ditwl-ec-back-end-123 |--@tag_private_name_ditwl_ec_front_end_001: | |--ditwl-ec-front-end-001 |--@tag_public_name_www: | |--ditwl-ec-front-end-001 |--@tag_Name_ditwl_ec_front_end_001: | |--ditwl-ec-front-end-001 |--@tag_cost_center_green_department: | |--ditwl-ec-front-end-001 |--@tag_os_ver_23_04: | |--ditwl-ec-front-end-001 |--@tag_app_front_end: | |--ditwl-ec-front-end-001 |--@tag_app_ver_2_3: | |--ditwl-ec-front-end-001 |--@instance_type_t4g_micro: | |--ditwl-ec-front-end-001
The above list shows all the groups that will be available for Ansible Playbooks as host selectors.
In the Ansible inventory, all dots and dashes in the AWS tags and their values are converted to underscore, except for the hostnames.
For example:
tag_os_ubuntu includes all EC2 instances that run Ubuntu as the operating system (set by Terraform in the tag "os" with the value "ubuntu").tag_os_ubuntu & tag_os_ver_22_04 includes all EC2 instances running Ubuntu version 22.04, in this example only the instance ditwl-ec-back-end-123Ansible commands require specific configurations that can be set in different locations:
Additionally, many configuration settings can be set using session environment variables.
For this tutorial, we use environment variables as this approach allows code portability (e.g. to be included in automated CI/CD DevOps flows).
ANSIBLE INVENTORY
Set the environment variable ANSIBLE_INVENTORY pointing to the AWS Inventory configuration file.
export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml
SSH PRIVATE KEY
Ansible requires SSH access to the instances and therefore the authentication is performed using a private key. We will use the private key pair created for the AWS instances (see Generating and using AWS Key Pairs with Terraform or OpenTofu).
export ANSIBLE_PRIVATE_KEY_FILE=${HOME}/keys/ditwl-kp-config-user-ecdsa
Previous Ansible versions accepted environment variable ANSIBLE_PRIVATE_KEY but seems to be deprecated now.
For simplicity, we are using the same private key for everything, but in a real scenario, it's recommended to have different keys for each combination of cloud, environment, and user.
For convenience write the configuration instructions in a script file as Ansible needs to be configured every time a new shell session is started.
Review the configuration from the file ansible/ditwl_pro.sh:
# source this file before using ansible
# source ditwl_pro.sh
#Set Private KEY for SSH
export ANSIBLE_PRIVATE_KEY_FILE=${HOME}/keys/ditwl-kp-config-user-ecdsa
#Set Ansible Inventory configuration
export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml
echo ANSIBLE_PRIVATE_KEY_FILE=$ANSIBLE_PRIVATE_KEY_FILE
echo ANSIBLE_INVENTORY=$ANSIBLE_INVENTORYBefore running any ansible commands, source the configuration file ditwl_pro.sh to set the appropriate environment variables.
$ source ditwl_pro.sh ANSIBLE_PRIVATE_KEY_FILE=/home/user/keys/ditwl-kp-config-user-ecdsa ANSIBLE_INVENTORY=/home/user/git/.../ansible/inventory.aws_ec2.yml
Get detailed inventory host information by running ansible-inventory for a specific host or review the whole inventory.
All this host info can be used inside Ansible Playbooks for selecting target hosts and for configuration.
$ ansible-inventory -i inventory.aws_ec2.yml --host ditwl-ec-front-end-001
{
"ami_launch_index": 0,
"ansible_host": "18.234.149.2",
"architecture": "arm64",
"block_device_mappings": [
{
"device_name": "/dev/sda1",
"ebs": {
"attach_time": "2023-12-13T09:02:55+00:00",
"delete_on_termination": true,
"status": "attached",
"volume_id": "vol-0c1547809a44aa5cd"
}
}
],
"boot_mode": "uefi",
"capacity_reservation_specification": {
"capacity_reservation_preference": "open"
},
"client_token": "terraform-20231213090254327700000001",
"cpu_options": {
"core_count": 2,
"threads_per_core": 1
},
"ebs_optimized": false,
"ena_support": true,
"enclave_options": {
"enabled": false
},
"hibernation_options": {
"configured": false
},
"hypervisor": "xen",
"image_id": "ami-03c0a7fe7530c7928",
"instance_id": "i-0e30e5c5969f14fea",
"instance_type": "t4g.micro",
"key_name": "ditwl-kp-config-user",
"launch_time": "2023-12-13T09:02:55+00:00",
"metadata_options": {
"http_endpoint": "enabled",
"http_protocol_ipv6": "disabled",
"http_put_response_hop_limit": 1,
"http_tokens": "optional",
"instance_metadata_tags": "disabled",
"state": "applied"
},
"monitoring": {
"state": "disabled"
},
"network_interfaces": [
{
"association": {
"ip_owner_id": "amazon",
"public_dns_name": "",
"public_ip": "18.234.149.2"
},
"attachment": {
"attach_time": "2023-12-13T09:02:55+00:00",
"attachment_id": "eni-attach-09234e6992f5e5c39",
"delete_on_termination": true,
"device_index": 0,
"network_card_index": 0,
"status": "attached"
},
"description": "",
"groups": [
{
"group_id": "sg-095f8ebf532627f53",
"group_name": "ditwl-sg-front-end"
},
{
"group_id": "sg-06293ee469299179f",
"group_name": "ditwl-sg-ssh-ec2"
}
],
"interface_type": "interface",
"ipv6_addresses": [],
"mac_address": "0a:8a:84:0f:aa:f7",
"network_interface_id": "eni-004e9402ef4e6e0ca",
"owner_id": "431960152202",
"private_ip_address": "172.21.1.133",
"private_ip_addresses": [
{
"association": {
"ip_owner_id": "amazon",
"public_dns_name": "",
"public_ip": "18.234.149.2"
},
"primary": true,
"private_ip_address": "172.21.1.133"
}
],
"source_dest_check": true,
"status": "in-use",
"subnet_id": "subnet-0e0726ade683c52af",
"vpc_id": "vpc-03546f2eaca41aead"
}
],
"owner_id": "431960152202",
"placement": {
"availability_zone": "us-east-1c",
"group_name": "",
"region": "us-east-1",
"tenancy": "default"
},
"platform_details": "Linux/UNIX",
"private_dns_name": "ip-172-21-1-133.ec2.internal",
"private_dns_name_options": {
"enable_resource_name_dns_a_record": false,
"enable_resource_name_dns_aaaa_record": false,
"hostname_type": "ip-name"
},
"private_ip_address": "172.21.1.133",
"product_codes": [],
"public_dns_name": "",
"public_ip_address": "18.234.149.2",
"requester_id": "",
"reservation_id": "r-0a681315893f5676c",
"root_device_name": "/dev/sda1",
"root_device_type": "ebs",
"security_groups": [
{
"group_id": "sg-095f8ebf532627f53",
"group_name": "ditwl-sg-front-end"
},
{
"group_id": "sg-06293ee469299179f",
"group_name": "ditwl-sg-ssh-ec2"
}
],
"source_dest_check": true,
"state": {
"code": 16,
"name": "running"
},
"state_transition_reason": "",
"subnet_id": "subnet-0e0726ade683c52af",
"tags": {
"Name": "ditwl-ec-front-end-001",
"app": "front-end",
"app_ver": "2.3",
"cost_center": "green-department",
"environment": "pro",
"os": "ubuntu",
"os_arch": "arm64",
"os_ver": "23.04",
"private_name": "ditwl-ec-front-end-001",
"public_name": "www"
},
"usage_operation": "RunInstances",
"usage_operation_update_time": "2023-12-13T09:02:55+00:00",
"virtualization_type": "hvm",
"vpc_id": "vpc-03546f2eaca41aead"
}Access all inventory data for the AWS infrastructure with the ansible-inventory --list command. As it can be a long list of properties, it is recommended to pipe the output to a file (inventory.json).
$ ansible-inventory --list > inventory.json
Inside the inventory and for each EC2 instance, a node is added to hostvars, the node contains properties and groups that will be available when running a playbook.
Example from a different tutorial showing two different groups holding PRE and PRO environment instances:
{
"_meta": {
...
"tag_environment_pre": [
"ditwl_ec2_pre_pub_abc01_1",
"ditwl_ec2_pre_pub_abc01_2",
"ditwl_ec2_pre_pub_abc01_3"
],
"tag_environment_pro": [
"ditwl_ec2_pro_pub_wp01_1"
]
...
}The Ansible Playbook is a YAML file that defines a set of tasks to be executed on one or more hosts.
Hosts are selected using Inventory Groups and Metadata, pattern selection allows for complex host selection.
| Description | Pattern | Example | Result |
|---|---|---|---|
| All hosts | all or * | all | ditwl-ec-front-end-001 ditwl-ec-back-end-123 |
| One host | name of host | ditwl-ec-front-end-001 | ditwl-ec-front-end-001 |
| Multiple Hosts | host1:host2 | ditwl-ec-front-end-001:ditwl-ec-back-end-123 | ditwl-ec-front-end-001 ditwl-ec-back-end-123 |
| One group | name of group | tag_cost_center_blue_department | ditwl-ec-back-end-123 |
| Multiple groups | group1:group2 | tag_cost_center_blue_department:tag_cost_center_green_department | ditwl-ec-front-end-001 ditwl-ec-back-end-123 |
| Excluding groups | group1:!group2 | tag_environment_pro:!instance_type_t4g_small | ditwl-ec-front-end-001 |
| Intersection of groups | group1:&group2 | tag_os_ubuntu:&tag_os_ver_23_04 | ditwl-ec-front-end-001 |
The file ansible/testing.playbook.yaml is used for testing host group selection.
- name: All hosts
hosts: all
remote_user: ubuntu
tasks:
- name: Ping all
ansible.builtin.ping:
- name: One host, by name
hosts: ditwl-ec-front-end-001
remote_user: ubuntu
tasks:
- name: Ping
ansible.builtin.ping:
- name: Multiple Hosts, by name
hosts: ditwl-ec-front-end-001:ditwl-ec-back-end-123
remote_user: ubuntu
tasks:
- name: Ping
ansible.builtin.ping:
- name: One group, hosts in group Cost Center blue_department (AWS hosts with tag cost_center=blue-department)
hosts: tag_cost_center_blue_department
remote_user: ubuntu
tasks:
- name: Ping
ansible.builtin.ping:
- name: Multiple groups, hosts in groups Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue-department or cost_center=green_department)
hosts: tag_cost_center_blue_department:tag_cost_center_green_department
remote_user: ubuntu
tasks:
- name: Ping
ansible.builtin.ping:
- name: Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g_small)
hosts: tag_environment_pro:!instance_type_t4g_small
remote_user: ubuntu
tasks:
- name: Ping
ansible.builtin.ping:
- name: Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04)
hosts: tag_os_ubuntu:&tag_os_ver_23_04
remote_user: ubuntu
tasks:
- name: Ping
ansible.builtin.ping:
Show the list of matching hosts for the patterns but not executing the playbook:
$ ansible-playbook testing.playbook.yaml --list-hosts
playbook: testing.playbook.yaml
play #1 (all): All hosts TAGS: []
pattern: ['all']
hosts (2):
ditwl-ec-front-end-001
ditwl-ec-back-end-123
play #2 (ditwl-ec-front-end-001): One host, by name TAGS: []
pattern: ['ditwl-ec-front-end-001']
hosts (1):
ditwl-ec-front-end-001
play #3 (ditwl-ec-front-end-001:ditwl-ec-back-end-123): Multiple Hosts, by name TAGS: []
pattern: ['ditwl-ec-front-end-001:ditwl-ec-back-end-123']
hosts (2):
ditwl-ec-front-end-001
ditwl-ec-back-end-123
play #4 (tag_cost_center_blue_department): One group, hosts in Cost Center blue_department (AWS hosts with tag cost_center=blue_department) TAGS: []
pattern: ['tag_cost_center_blue_department']
hosts (1):
ditwl-ec-back-end-123
play #5 (tag_cost_center_blue_department:tag_cost_center_green_department): Multiple groups, hosts in Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue_department or cost_center=green_department) TAGS: []
pattern: ['tag_cost_center_blue_department:tag_cost_center_green_department']
hosts (2):
ditwl-ec-front-end-001
ditwl-ec-back-end-123
play #6 (tag_environment_pro:!instance_type_t4g_small): Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g_small) TAGS: []
pattern: ['tag_environment_pro:!instance_type_t4g_small']
hosts (1):
ditwl-ec-front-end-001
play #7 (tag_os_ubuntu:&tag_os_ver_23_04): Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04) TAGS: []
pattern: ['tag_os_ubuntu:&tag_os_ver_23_04']
hosts (1):
ditwl-ec-front-end-001Run the playbook to test SSH connectivity:
$ ansible-playbook testing.playbook.yaml PLAY [All hosts] ********************************************************************************************************************************************************************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] ok: [ditwl-ec-back-end-123] TASK [Ping all] *********************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] ok: [ditwl-ec-front-end-001] PLAY [One host, by name] ************************************************************************************************************************************************************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] PLAY [Multiple Hosts, by name] ******************************************************************************************************************************************************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] ok: [ditwl-ec-back-end-123] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] ok: [ditwl-ec-front-end-001] PLAY [One group, hosts in Cost Center blue_department (AWS hosts with tag cost_center=blue-department)] ******************************************************************************* TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] PLAY [Multiple groups, hosts in Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue-department or cost_center=green-department)] ****************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] ok: [ditwl-ec-back-end-123] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] ok: [ditwl-ec-front-end-001] PLAY [Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g.small)] **************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] PLAY [Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04)] ************************************************************* TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] PLAY RECAP **************************************************************************************************************************************************************************** ditwl-ec-back-end-123 : ok=8 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ditwl-ec-front-end-001 : ok=12 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Ansible shows an empty inventory or the hosts list is empty when it has not been properly configured to use a dynamic inventory.
$ ansible-inventory --list
{
"_meta": {
"hostvars": {}
},
"all": {
"children": [
"ungrouped"
]
}
}
$ ansible-playbook playbook.yaml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'Check that the ANSIBLE_INVENTORY environment variable has been properly configured to point to the dynamic inventory configuration file and the file name ends in .aws_ec2.yml.
export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml
Make sure that the dynamic inventory is working, see the section on how to test the Ansible dynamic directory.
Ansible checks that the Private Key file permissions are properly configured.
WARNING: UNPROTECTED PRIVATE KEY FILE! Permissions 0660 for '/home/user/keys/ditwl-kp-config-user-ecdsa' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. Load key \"/home/user/keys/ditwl-kp-config-user-ecdsa\": bad permissions\ Permission denied (publickey)
See How to set the correct permissions for the Ansible Private Key file.
When running an ansible-playbook for the first time, Ansible opens an SSH connection to all targeted hosts.
The message "The authenticity of host can't be established" in SSH occurs when you try to connect to a server for the first time or if the server's host key has changed. This message serves as a security measure to warn you about a potential man-in-the-middle attack.
When you connect to a server for the first time, your SSH client doesn't have the server's host key stored locally. This host key acts like a digital fingerprint, uniquely identifying the server.
Without this stored key, your client cannot verify the server's identity and prevent the possibility of connecting to an imposter server.
The authenticity of host '3.94.214.81 (3.94.214.81)' can't be established. ED25519 key fingerprint is SHA256:mIBL6hfMMeEuaoxW+UL8JV3HaKShxFFLcgEYXaVD/3c. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
If you trust that the connection is being done to the correct server, answer yes to add the key fingerprint to your known_hosts file.
When a minimal Terraform configuration doesn't specify the availability zone for subnets, there is a possibility of randomly getting assigned an availability zone that doesn't have all AWS Services or instance types.
│ Error: creating EC2 Instance: Unsupported: Your requested instance type (t4g.micro) is not supported in your requested Availability Zone (us-east-1e). Please retry your request by not specifying an Availability Zone or choosing us-east-1a, us-east-1b, us-east-1c, us-east-1d, us-east-1f.
Set the availability zone for the subnet and run again.
# Subnet
resource "aws_subnet" "ditwl-sn-za-pro-pub-00" {
vpc_id = aws_vpc.ditlw-vpc.id
cidr_block = "172.21.0.0/23" #172.21.0.0 - 172.21.1.255
map_public_ip_on_launch = true
availability_zone = "us-east-1c"
tags = {
Name = "ditwl-sn-za-pro-pub-00"
}
}
IT Wonder Lab tutorials are based on the diverse experience of Javier Ruiz, who founded and bootstrapped a SaaS company in the energy sector. His company, later acquired by a NASDAQ traded company, managed over €2 billion per year of electricity for prominent energy producers across Europe and America. Javier has over 25 years of experience in building and managing IT companies, developing cloud infrastructure, leading cross-functional teams, and transitioning his own company from on-premises, consulting, and custom software development to a successful SaaS model that scaled globally.
Are you looking for cloud automation best practices tailored to your company?
Nice tutorial
[…] IT Wonderlab: How to Use Terraform, AWS, and Ansible Together ⁷ […]