# How to use Terraform, AWS, and Ansible Together > How to use Terraform to create AWS infrastructure with tags, and Ansible with its dynamic inventory plugin to configure the servers, linked by the tags. - Source: https://www.itwonderlab.com/terraform-aws-ansible/ - Published: 2023-12-13 - Updated: 2026-10-05 - Author: Javier Ruiz Jiménez (https://www.javierruizjimenez.com/) - Site: IT Wonder Lab (https://www.itwonderlab.com/) --- ## Linking Terraform AWS, and Ansible [Terraform](https://www.itwonderlab.com/tag/terraform/) is an Infrastructure as Code (IaC) tool used to create infrastructure and **should never be used to configure operating systems or applications**, instead, [Ansible](https://www.itwonderlab.com/tutorials/ansible/) is the de facto standard for the operating system and application configuration as code. This tutorial shows how to create infrastructure in AWS using Terraform and configure the operating system and applications using Ansible. ![Ansible AWS EC2 Terraform Tags](https://www.itwonderlab.com/media/tutorials/Terraform-AWS-Ansible/ITWL-Tutorials-Terraform-AWS-Ansible-Intro_2.svg) Linking the infrastructure created in AWS with Terraform is done using **infrastructure tags**. ### Prerequisites - [Install Terraform or OpenTofu](https://www.itwonderlab.com/how-to-install-opentofu/). - [Install Ansible](https://www.itwonderlab.com/how-to-install-ansible/). - AWS account, see [How to Create an AWS Account for demos](https://www.itwonderlab.com/create-an-aws-account-for-demos/). - AWS CLI profile, see [How to Install AWS CLI V2](https://www.itwonderlab.com/install-aws-cli/). - A Private Key Pair, See [Generating and using AWS Key Pairs with Terraform or OpenTofu](https://www.itwonderlab.com/aws-key-pairs-terraform/). - Download the [Terraform and Ansible Tutorial source code from the IT Wonder Lab GitHub repository](https://github.com/itwonderlab/ansible-aws-ec2-terraform-tags). ### Define the Infrastructure Tags All the AWS resources created with Terraform will have tags that follow a company-wide standard. Tags are used for provisioning, monitoring, and cost control. Check [AWS Resource Tagging best practices](https://www.itwonderlab.com/aws-resource-tagging/). | Tag Name | Description | Example | | --- | --- | --- | | Name | Unique name for the instance across the organization and the world (in case of mergers) | ditwl-ec-front-end-001 | | private\_name | The name assigned in the private DNS, Has detailed information and is usually the Name. | ditwl-ec-front-end-001 | | public\_name | The name assigned in the public DNS (hides releases, can be empty if not published on external DNS) | www | | app | Name of the application running on the instance | front-end | | app\_ver | The version of the application that should be deployed in the Instance | 1.2 | | os | Operating System of the Instance | ubuntu | | os\_ver | Release of the Operating System | 22.04 | | os\_arch | Hardware architecture of the Instance (and Operating System) | arm64 | | environment | Environment served by the application, usually dev, test, pre/qa, pro | pro | | cost\_center | Cost center assigned for the instance (who pays the instance cost) | blue-department | ### Create the Infrastructure in AWS using Terraform > [!WARNING] > **The infrastructure demonstrated in this Terraform and Ansible tutorial has been extremely simplified for educational purposes only and is not intended for production use**. It contains numerous security and design flaws that would pose significant risks in a real-world environment. Running the tutorial may incur costs for cloud resources, and it is the user’s responsibility to manage and terminate resources to avoid unnecessary charges. **Please be aware of the following limitations and risks:** - **Security Flaws:** The infrastructure is intentionally designed with security flaws for learning purposes. Implementing these configurations in production would expose your systems to vulnerabilities. - **Design Flaws:** The architecture may not be optimized for scalability, performance, or cost-efficiency. Using these configurations in production could lead to inefficient resource utilization and performance issues. - **Cost Considerations:** Running cloud resources associated with the tutorial may incur costs. Be sure to monitor and terminate resources promptly to avoid unnecessary charges. - **Limited Scope:** The tutorial is intended to provide a basic understanding of Terraform and Ansible. It does not cover advanced topics or production-ready configurations. **Before proceeding with this tutorial:** - Carefully review the provided code and configurations to understand the potential risks. - Only run the tutorial in a non-production environment. - Terminate all resources created during the tutorial when finished. - Do not use the generated infrastructure in any production environment. By proceeding with this tutorial, you acknowledge the risks and limitations outlined above and agree to take full responsibility for your actions. The Terraform file **terraform/aws-basic-vpc-ec2-tags.tf** contains all the code necessary to create the AWS infrastructure. You can either download the code from the [IT Wonder Lab GitHub repository](https://github.com/itwonderlab/ansible-aws-ec2-terraform-tags) or create the files manually. The following sections show the Terraform code used to create each AWS element. **terraform/aws-basic-vpc-ec2-tags.tf** #### Providers and AWS Provider configuration Defines the required Terraform version and the AWS required provider. ```yaml # Copyright (C) 2018 - 2023 IT Wonder Lab (https://www.itwonderlab.com) # # This software may be modified and distributed under the terms # of the MIT license. See the LICENSE file for details. # -------------------------------- WARNING -------------------------------- # IT Wonder Lab's best practices for infrastructure include modularizing # Terraform/OpenTofu configuration. # In this example, we define everything in a single file. # See other tutorials for best practices at itwonderlab.com # -------------------------------- WARNING -------------------------------- #Define Terrraform Providers and Backend terraform { required_version = "> 1.5" required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } } #----------------------------------------- # Default provider: AWS #----------------------------------------- provider "aws" { shared_credentials_files = ["~/.aws/credentials"] profile = "ditwl_infradmin" region = "us-east-1" } ``` #### Networking (VPC, Subnet and Route Table) Create a [VPC](https://www.itwonderlab.com/aws-vpc/) named `ditlw-vpc`, a single [subnet](https://www.itwonderlab.com/aws-subnets/) named `ditwl-sn-za-pro-pub-00` configured as a public subnet to assign Public IPs to its EC2 instances. ![Terraform AWS Ansible VPC Subnet Routing](https://www.itwonderlab.com/media/tutorials/Terraform-AWS-Ansible/ITWL-Tutorials-Terraform-AWS-Ansible-VPC-Subnet-Routing.svg) An Internet Gateway, named `ditwl-ig`, is created to allow EC2 instances with public IPs to access the Internet. A Routing table named `ditwl-rt-pub-main`, is created and assigned as the main or default routing table for the VPC, it includes a route to send all traffic directed to the Internet through the Internet Gateway. ```yaml # VPC resource "aws_vpc" "ditlw-vpc" { cidr_block = "172.21.0.0/19" #172.21.0.0 - 172.21.31.254 tags = { Name = "ditlw-vpc" } } # Subnet resource "aws_subnet" "ditwl-sn-za-pro-pub-00" { vpc_id = aws_vpc.ditlw-vpc.id cidr_block = "172.21.0.0/23" #172.21.0.0 - 172.21.1.255 map_public_ip_on_launch = true tags = { Name = "ditwl-sn-za-pro-pub-00" } } # Internet Gateway resource "aws_internet_gateway" "ditwl-ig" { vpc_id = aws_vpc.ditlw-vpc.id tags = { Name = "ditwl-ig" } } # Routing table for public subnet (access to Internet) resource "aws_route_table" "ditwl-rt-pub-main" { vpc_id = aws_vpc.ditlw-vpc.id route { cidr_block = "0.0.0.0/0" gateway_id = aws_internet_gateway.ditwl-ig.id } tags = { Name = "ditwl-rt-pub-main" } } # Set new main_route_table as main resource "aws_main_route_table_association" "ditwl-rta-default" { vpc_id = aws_vpc.ditlw-vpc.id route_table_id = aws_route_table.ditwl-rt-pub-main.id } ``` #### Network and Instance Security AWS Security groups hold the security rules that allow ingress and egress traffic to and from the EC2 instances. In this example, a basic group will be created to be assigned to all instances and the other two security groups will hold the rules specific to each instance. ![Terraform AWS Ansible Security Groups](https://www.itwonderlab.com/media/tutorials/Terraform-AWS-Ansible/ITWL-Tutorials-Terraform-AWS-Ansible-Security-Groups-2.svg) ##### Base Security Group: ditwl-sg-base-ec2 Creates a Security Group named `ditwl-sg-base-ec2` and the following rules: - `ditwl-sr-internet-to-ec2-ssh`: Rule Allow SSH to EC2 instances (ingress). - `ditwl-sr-internet-to-ec2-icmp`: Allow ICMP (Ping) from the Internet into the instances (ingress). - `ditwl-sr-all-outbund`: Allow all protocols from the instances to the Internet (egress). > [!NOTE] > Be aware that SSH access from the Internet is dangerous and is only used for demo simplicity. The Base Security Group will be assigned to all the EC2 instances. ```yaml # Create a "base" Security Group to be assigned to all EC2 instances resource "aws_security_group" "ditwl-sg-base-ec2" { name = "ditwl-sg-ssh-ec2" vpc_id = aws_vpc.ditlw-vpc.id } # DANGEROUS!! # Allow access from the Internet to port 22 (SSH) in the EC2 instances resource "aws_security_group_rule" "ditwl-sr-internet-to-ec2-ssh" { security_group_id = aws_security_group.ditwl-sg-base-ec2.id type = "ingress" from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] # Internet } # Allow access from the Internet for ICMP protocol (e.g. ping) to the EC2 instances resource "aws_security_group_rule" "ditwl-sr-internet-to-ec2-icmp" { security_group_id = aws_security_group.ditwl-sg-base-ec2.id type = "ingress" from_port = -1 to_port = -1 protocol = "icmp" cidr_blocks = ["0.0.0.0/0"] # Internet } # Allow all outbound traffic to Internet resource "aws_security_group_rule" "ditwl-sr-all-outbund" { security_group_id = aws_security_group.ditwl-sg-base-ec2.id type = "egress" from_port = "0" to_port = "0" protocol = "-1" cidr_blocks = ["0.0.0.0/0"] } ``` ##### Front-end Server Security Group: ditwl-sg-front-end Creates a Security Group named `ditwl-sg-front-end` with a single rule: - `ditwl-sr-internet-to-front-end-80`: Rule to allow access from the Internet to TCP port 80. This Security Group will be assigned to the front-end instance allowing access from the Internet to the instance port 80 (HTTP web server). ```yaml # Create a Security Group for the Front end Server resource "aws_security_group" "ditwl-sg-front-end" { name = "ditwl-sg-front-end" vpc_id = aws_vpc.ditlw-vpc.id } # Allow access from the Internet to port 80 in the EC2 instances resource "aws_security_group_rule" "ditwl-sr-internet-to-front-end-80" { security_group_id = aws_security_group.ditwl-sg-front-end.id type = "ingress" from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] # Internet } ``` ##### Back-end Server Security Group: ditwl-sg-back-end Creates a Security Group named `ditwl-sg-back-end` with a single rule that uses another Security Group as a source of traffic. - `ditwl-sr-front-end-to-mariadb`: Rule to allow access from the `ditwl-sg-front-end` security group to TCP port 3306. This Security Group will be assigned to the back-end instance allowing access from the front-end instance (because that instance will have the `ditwl-sg-front-end` security group assigned) to the MariaDB service. ```yaml # Create a Security Group for the Back-end Server resource "aws_security_group" "ditwl-sg-back-end" { name = "ditwl-sg-back-end" vpc_id = aws_vpc.ditlw-vpc.id } # Allow access from the front-end to port 3306 in the back-end (MariaDB) resource "aws_security_group_rule" "ditwl-sr-front-end-to-mariadb" { security_group_id = aws_security_group.ditwl-sg-back-end.id type = "ingress" from_port = 3306 to_port = 3306 protocol = "tcp" source_security_group_id = aws_security_group.ditwl-sg-front-end.id } ``` #### Private Key and AMIs Create a Private Key Pair for SSH Instance Authentication (see how to create the file in [Generating and using AWS Key Pairs with Terraform or OpenTofu](https://www.itwonderlab.com/aws-key-pairs-terraform/)). The public key of the pair will be included by AWS in the instance authorized access keys while running the initial cloud-init/cloud-config script. Ansible will use the same Private Key pair to access the instances for configuration from the management node. Find the AMIs for Ubuntu 22.04 ARM64 Server and Ubuntu 23.04 ARM64 Minimal that will be later used to spin up two EC2 instances (Ubuntu 23.04 has reached its end of life; use a supported release as explained in [AWS AMIs](https://www.itwonderlab.com/aws-terraform-tutorial-aws-ami/)). See [Terraform EC2 creation](#create-the-ec2-instances) below for private key and AMI assignment. ```yaml # Upload a Private Key Pair for SSH Instance Authentication resource "aws_key_pair" "ditwl-kp-config-user" { key_name = "ditwl-kp-config-user" public_key = file("~/keys/ditwl-kp-config-user-ecdsa.pub") } #Find AMI Ubuntu 22.04 ARM64 Server data "aws_ami" "ubuntu-22-04-arm64-server" { most_recent = true filter { name = "name" values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-arm64-server-*"] } filter { name = "virtualization-type" values = ["hvm"] } owners = ["099720109477"] # Canonical } #Find AMI Ubuntu 23.04 ARM64 Minimal data "aws_ami" "ubuntu-23-04-arm64-minimal" { most_recent = true filter { name = "name" values = ["ubuntu-minimal/images/hvm-ssd/ubuntu-lunar-23.04-arm64-minimal-*"] } filter { name = "virtualization-type" values = ["hvm"] } owners = ["099720109477"] # Canonical } ``` #### Create the EC2 Instances Two EC2 instances named `ditwl-ec-front-end-001` and `ditwl-ec-back-end-123` are created for different purposes with some common characteristics and some differences (e.g. instance types `t4g.micro` and `t4g.small`), tags and values are added to further classify and later on differentiate their purpose, instances have the same tags with some value differences. ![Shows two EC2 instances and its Terraform configuration and Tags](https://www.itwonderlab.com/media/tutorials/Terraform-AWS-Ansible/ITWL-Tutorials-Terraform-AWS-Ansible-EC2_1.svg) Later, Ansible will get an inventory with all instances, and their technical characteristics, like the assigned IPs, the Tags, and the assigned values. The inventory values are used to filter and apply different configurations to each set of instances. (e.g. change CPU limits to all instances running Ubuntu as the operating system, remove swap in instances with less than 1G of memory). In the Ansible inventory, all dashes and dots will be converted to underscore. | Tag Name | ditwl-ec-front-end-001
Front-end server | ditwl-ec-back-end-123
Back-end server | | --- | --- | --- | | Name | ditwl-ec-front-end-001 | ditwl-ec-back-end-123 | | private\_name | ditwl-ec-front-end-001 | ditwl-ec-back-end-123 | | public\_name | www | server | | app | front-end | back-end | | app\_ver | 2.3 | 1.2 | | os | ubuntu | ubuntu | | os\_ver | 23.04 | 22.04 | | os\_arch | arm64 | arm64 | | environment | pro | pro | | cost\_center | green-department | blue-department | ```yaml # Front end server running Ubuntu 23.04 ARM Minimal. resource "aws_instance" "ditwl-ec-front-end-001" { ami = data.aws_ami.ubuntu-23-04-arm64-minimal.id instance_type = "t4g.micro" subnet_id = aws_subnet.ditwl-sn-za-pro-pub-00.id key_name = "ditwl-kp-config-user" vpc_security_group_ids = [aws_security_group.ditwl-sg-base-ec2.id,aws_security_group.ditwl-sg-front-end.id] tags = { "Name" = "ditwl-ec-front-end-001" "private_name" = "ditwl-ec-front-end-001" "public_name" = "www" "app" = "front-end" "app_ver" = "2.3" "os" = "ubuntu" "os_ver" = "23.04" "os_arch" = "arm64" "environment" = "pro" "cost_center" = "green-department" } } # Back end server running Ubuntu 22.04 ARM Server. resource "aws_instance" "ditwl-ec-back-end-123" { ami = data.aws_ami.ubuntu-22-04-arm64-server.id instance_type = "t4g.small" subnet_id = aws_subnet.ditwl-sn-za-pro-pub-00.id key_name = "ditwl-kp-config-user" vpc_security_group_ids = [aws_security_group.ditwl-sg-base-ec2.id, aws_security_group.ditwl-sg-back-end.id] tags = { "Name" = "ditwl-ec-back-end-123" "private_name" = "ditwl-ec-back-end-123" "public_name" = "server" "app" = "back-end" "app_ver" = "1.2" "os" = "ubuntu" "os_ver" = "22.04" "os_arch" = "arm64" "environment" = "pro" "cost_center" = "blue-department" } } ``` ### Init, Plan, and Apply the Terraform Plan Initialize, plan, and apply the Terraform plan: ```shell $ tofu apply data.aws_ami.ubuntu-23-04-arm64-minimal: Reading... data.aws_ami.ubuntu-22-04-arm64-server: Reading... data.aws_ami.ubuntu-23-04-arm64-minimal: Read complete after 2s [id=ami-03c0a7fe7530c7928] data.aws_ami.ubuntu-22-04-arm64-server: Read complete after 2s [id=ami-05d47d29a4c2d19e1] OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create OpenTofu will perform the following actions: # aws_instance.ditwl-ec-back-end-123 will be created + resource "aws_instance" "ditwl-ec-back-end-123" { + ami = "ami-05d47d29a4c2d19e1" + instance_type = "t4g.small" + key_name = "ditwl-kp-config-user" + tags = { + "Name" = "ditwl-ec-back-end-123" + "app" = "back-end" + "app_ver" = "1.2" + "cost_center" = "blue-department" + "environment" = "pro" + "os" = "ubuntu" + "os_arch" = "arm64" + "os_ver" = "22.04" + "private_name" = "ditwl-ec-back-end-123" + "public_name" = "server" } ... } # aws_instance.ditwl-ec-front-end-001 will be created + resource "aws_instance" "ditwl-ec-front-end-001" { + ami = "ami-03c0a7fe7530c7928" + get_password_data = false + instance_type = "t4g.micro" + key_name = "ditwl-kp-config-user" + source_dest_check = true + tags = { + "Name" = "ditwl-ec-front-end-001" + "app" = "front-end" + "app_ver" = "2.3" + "cost_center" = "green-department" + "environment" = "pro" + "os" = "ubuntu" + "os_arch" = "arm64" + "os_ver" = "23.04" + "private_name" = "ditwl-ec-front-end-001" + "public_name" = "www" } ... } ... Plan: 16 to add, 0 to change, 0 to destroy. Do you want to perform these actions? OpenTofu will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes .. aws_instance.ditwl-ec-front-end-001: Creating... aws_instance.ditwl-ec-back-end-123: Creating... aws_instance.ditwl-ec-front-end-001: Still creating... [10s elapsed] aws_instance.ditwl-ec-back-end-123: Still creating... [10s elapsed] aws_instance.ditwl-ec-back-end-123: Creation complete after 16s [id=i-08727eee6f8452337] aws_instance.ditwl-ec-front-end-001: Creation complete after 16s [id=i-0e30e5c5969f14fea] Apply complete! Resources: 16 added, 0 changed, 0 destroyed. ``` ### Configure Ansible Dynamic Inventory [Ansible dynamic inventory](https://www.itwonderlab.com/ansible-dynamic-inventory/) is a feature in Ansible that allows you to generate inventory (host and group information) **dynamically** from the current cloud state rather than statically defining it in a static inventory file. The inventory file contains a list of the AWS instances and all their metadata, properties, tags, and values. #### Create an AWS Inventory Configuration File Create a file named `ansible/inventory.aws_ec2.yml` (the name can be changed **as long as the file ends in .aws\_ec2.yml**): The file defines: - **plugin**: the name of the plugin being configured `amazon.aws.aws_ec2` - **cache**: Store local information (set to false during development) - **use\_ssm\_inventory**: use metadata from the AWS SSM service to complement the inventory. - regions: configures the regions to query for available instances, we will only query the `us-east-1` region. - **profile**: the name of the AWS profile to use for authentication (see [Configure AWS CLI Security](https://www.itwonderlab.com/install-aws-cli/#configure-aws-cli-security)), `ditwl_infradmin`. - **keyed\_groups**: defines the Ansible Inventory Groups to create, we will create groups based on tags, and instance types. - **hostname**: the hostname is set to the value of the tag `private_name` for easy identification and tool traceability (e.g. `ditwl-ec-back-end-123` and `ditwl-ec-front-end-001`). It can include fallback values. - **ansible\_host**: defines what IP address, hostname, or FQDN to use for SSH access to the instance. We chose to use the `public_ip_address` as all our tutorial hosts have a public IP. In a production environment hosts will not have a public IP and instead, a VPN is used to access their private IP. **ansible/inventory.aws\_ec2.yml** ```yaml title="inventory.aws_ec2.yml" # AWS Ansible Inventory Plugin Configuration File # See options: https://docs.ansible.com/ansible/latest/collections/amazon/aws/aws_ec2_inventory.html plugin: amazon.aws.aws_ec2 cache: false use_ssm_inventory: true # Limit query to regions regions: - us-east-1 # A named AWS profile to use for authentication. profile: ditwl_infradmin # Create Ansible hosts groups using the following elements: keyed_groups: - prefix: tag key: tags - prefix: instance_type key: instance_type hostnames: - tag:private_name compose: ansible_host: public_ip_address ``` #### Test the Ansible Dynamic AWS Inventory Use the `ansible-inventory` command to list in a short graph format the AWS hosts using the Ansible Dynamic Inventory plugin for AWS: ```shell $ ansible-inventory -i inventory.aws_ec2.yml --graph @all: |--@ungrouped: |--@aws_ec2: | |--ditwl-ec-back-end-123 | |--ditwl-ec-front-end-001 |--@tag_environment_pro: | |--ditwl-ec-back-end-123 | |--ditwl-ec-front-end-001 |--@tag_cost_center_blue_department: | |--ditwl-ec-back-end-123 |--@tag_os_ver_22_04: | |--ditwl-ec-back-end-123 |--@tag_private_name_ditwl_ec_back_end_123: | |--ditwl-ec-back-end-123 |--@tag_os_ubuntu: | |--ditwl-ec-back-end-123 | |--ditwl-ec-front-end-001 |--@tag_Name_ditwl_ec_back_end_123: | |--ditwl-ec-back-end-123 |--@tag_app_back_end: | |--ditwl-ec-back-end-123 |--@tag_app_ver_1_2: | |--ditwl-ec-back-end-123 |--@tag_os_arch_arm64: | |--ditwl-ec-back-end-123 | |--ditwl-ec-front-end-001 |--@tag_public_name_server: | |--ditwl-ec-back-end-123 |--@instance_type_t4g_small: | |--ditwl-ec-back-end-123 |--@tag_private_name_ditwl_ec_front_end_001: | |--ditwl-ec-front-end-001 |--@tag_public_name_www: | |--ditwl-ec-front-end-001 |--@tag_Name_ditwl_ec_front_end_001: | |--ditwl-ec-front-end-001 |--@tag_cost_center_green_department: | |--ditwl-ec-front-end-001 |--@tag_os_ver_23_04: | |--ditwl-ec-front-end-001 |--@tag_app_front_end: | |--ditwl-ec-front-end-001 |--@tag_app_ver_2_3: | |--ditwl-ec-front-end-001 |--@instance_type_t4g_micro: | |--ditwl-ec-front-end-001 ``` The above list shows all the groups that will be available for Ansible Playbooks as host selectors. In the Ansible inventory, all dots and dashes in the AWS tags and their values are converted to underscore, except for the hostnames. For example: - The group `tag_os_ubuntu` includes all EC2 instances that run Ubuntu as the operating system (set by Terraform in the tag “os” with the value “ubuntu”). - The group tag\_os\_arch\_arm64 lists all hosts running ARM 64 architecture. - The group *boolean combination* `tag_os_ubuntu & tag_os_ver_22_04` includes all EC2 instances running Ubuntu version 22.04, in this example only the instance `ditwl-ec-back-end-123` ### Create a Script to set the Ansible Environment Ansible commands require specific configurations that can be set in different locations: - ANSIBLE\_CONFIG (environment variable if set) - ansible.cfg (in the current directory) - ~/.ansible.cfg (in the home directory) - /etc/ansible/ansible.cfg Additionally, many configuration settings can be set using session environment variables. For this tutorial, **we use environment variables as this approach allows code portability** (e.g. to be included in automated CI/CD DevOps flows). **ANSIBLE INVENTORY** Set the environment variable **ANSIBLE\_INVENTORY** pointing to the AWS Inventory configuration file. `export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml` **SSH PRIVATE KEY** Ansible requires SSH access to the instances and therefore the authentication is performed using a private key. We will use the private key pair created for the AWS instances (see [Generating and using AWS Key Pairs with Terraform or OpenTofu](https://www.itwonderlab.com/aws-key-pairs-terraform/)). `export ANSIBLE_PRIVATE_KEY_FILE=${HOME}/keys/ditwl-kp-config-user-ecdsa` Previous Ansible versions accepted the environment variable `ANSIBLE_PRIVATE_KEY`, which is no longer supported: use `ANSIBLE_PRIVATE_KEY_FILE`. > [!NOTE] > For simplicity, we are using the same private key for everything, but in a real scenario, it’s recommended to have **different keys** for each combination of cloud, environment, and user. #### Write the Ansible Configuration in a script file For convenience write the configuration instructions in a script file as Ansible needs to be configured every time a new shell session is started. Review the configuration from the file `ansible/ditwl_pro.sh`: ```bash title="ditwl_pro.sh" # source this file before using ansible # source ditwl_pro.sh #Set Private KEY for SSH export ANSIBLE_PRIVATE_KEY_FILE=${HOME}/keys/ditwl-kp-config-user-ecdsa #Set Ansible Inventory configuration export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml echo ANSIBLE_PRIVATE_KEY_FILE=$ANSIBLE_PRIVATE_KEY_FILE echo ANSIBLE_INVENTORY=$ANSIBLE_INVENTORY ``` Before running any ansible commands, source the configuration file `ditwl_pro.sh` to set the appropriate environment variables. ```shell $ source ditwl_pro.sh ANSIBLE_PRIVATE_KEY_FILE=/home/user/keys/ditwl-kp-config-user-ecdsa ANSIBLE_INVENTORY=/home/user/git/.../ansible/inventory.aws_ec2.yml ``` ### Access Detailed EC2 Inventory Host Information Get **detailed inventory host information** by running `ansible-inventory` for a specific host or review the whole inventory. All this **host info** can be used inside Ansible Playbooks for selecting target hosts and for configuration. #### Single host Inventory ```shell $ ansible-inventory -i inventory.aws_ec2.yml --host ditwl-ec-front-end-001 { "ami_launch_index": 0, "ansible_host": "18.234.149.2", "architecture": "arm64", "block_device_mappings": [ { "device_name": "/dev/sda1", "ebs": { "attach_time": "2023-12-13T09:02:55+00:00", "delete_on_termination": true, "status": "attached", "volume_id": "vol-0c1547809a44aa5cd" } } ], "boot_mode": "uefi", "capacity_reservation_specification": { "capacity_reservation_preference": "open" }, "client_token": "terraform-20231213090254327700000001", "cpu_options": { "core_count": 2, "threads_per_core": 1 }, "ebs_optimized": false, "ena_support": true, "enclave_options": { "enabled": false }, "hibernation_options": { "configured": false }, "hypervisor": "xen", "image_id": "ami-03c0a7fe7530c7928", "instance_id": "i-0e30e5c5969f14fea", "instance_type": "t4g.micro", "key_name": "ditwl-kp-config-user", "launch_time": "2023-12-13T09:02:55+00:00", "metadata_options": { "http_endpoint": "enabled", "http_protocol_ipv6": "disabled", "http_put_response_hop_limit": 1, "http_tokens": "optional", "instance_metadata_tags": "disabled", "state": "applied" }, "monitoring": { "state": "disabled" }, "network_interfaces": [ { "association": { "ip_owner_id": "amazon", "public_dns_name": "", "public_ip": "18.234.149.2" }, "attachment": { "attach_time": "2023-12-13T09:02:55+00:00", "attachment_id": "eni-attach-09234e6992f5e5c39", "delete_on_termination": true, "device_index": 0, "network_card_index": 0, "status": "attached" }, "description": "", "groups": [ { "group_id": "sg-095f8ebf532627f53", "group_name": "ditwl-sg-front-end" }, { "group_id": "sg-06293ee469299179f", "group_name": "ditwl-sg-ssh-ec2" } ], "interface_type": "interface", "ipv6_addresses": [], "mac_address": "0a:8a:84:0f:aa:f7", "network_interface_id": "eni-004e9402ef4e6e0ca", "owner_id": "431960152202", "private_ip_address": "172.21.1.133", "private_ip_addresses": [ { "association": { "ip_owner_id": "amazon", "public_dns_name": "", "public_ip": "18.234.149.2" }, "primary": true, "private_ip_address": "172.21.1.133" } ], "source_dest_check": true, "status": "in-use", "subnet_id": "subnet-0e0726ade683c52af", "vpc_id": "vpc-03546f2eaca41aead" } ], "owner_id": "431960152202", "placement": { "availability_zone": "us-east-1c", "group_name": "", "region": "us-east-1", "tenancy": "default" }, "platform_details": "Linux/UNIX", "private_dns_name": "ip-172-21-1-133.ec2.internal", "private_dns_name_options": { "enable_resource_name_dns_a_record": false, "enable_resource_name_dns_aaaa_record": false, "hostname_type": "ip-name" }, "private_ip_address": "172.21.1.133", "product_codes": [], "public_dns_name": "", "public_ip_address": "18.234.149.2", "requester_id": "", "reservation_id": "r-0a681315893f5676c", "root_device_name": "/dev/sda1", "root_device_type": "ebs", "security_groups": [ { "group_id": "sg-095f8ebf532627f53", "group_name": "ditwl-sg-front-end" }, { "group_id": "sg-06293ee469299179f", "group_name": "ditwl-sg-ssh-ec2" } ], "source_dest_check": true, "state": { "code": 16, "name": "running" }, "state_transition_reason": "", "subnet_id": "subnet-0e0726ade683c52af", "tags": { "Name": "ditwl-ec-front-end-001", "app": "front-end", "app_ver": "2.3", "cost_center": "green-department", "environment": "pro", "os": "ubuntu", "os_arch": "arm64", "os_ver": "23.04", "private_name": "ditwl-ec-front-end-001", "public_name": "www" }, "usage_operation": "RunInstances", "usage_operation_update_time": "2023-12-13T09:02:55+00:00", "virtualization_type": "hvm", "vpc_id": "vpc-03546f2eaca41aead" } ``` #### Ansible AWS Full Inventory Access all inventory data for the AWS infrastructure with the `ansible-inventory --list` command. As it can be a long list of properties, it is recommended to pipe the output to a file (inventory.json). ```shell $ ansible-inventory --list > inventory.json ``` Inside the inventory and for each **EC2 instance**, a node is added to **hostvars**, the node contains properties and groups that will be available when running a playbook. Example from a different tutorial showing two different groups holding PRE and PRO environment instances: ```json title="ansible-aws-ec2-terraform-tags$ inventory/ec2.py" { "_meta": { ... "tag_environment_pre": [ "ditwl_ec2_pre_pub_abc01_1", "ditwl_ec2_pre_pub_abc01_2", "ditwl_ec2_pre_pub_abc01_3" ], "tag_environment_pro": [ "ditwl_ec2_pro_pub_wp01_1" ] ... } ``` ### Create the Ansible Playbook The Ansible Playbook is a YAML file that defines a set of **tasks** to be executed on one or more **hosts**. Hosts are selected using Inventory Groups and Metadata, pattern selection allows for complex host selection. | **Description** | **Pattern** | **Example** | **Result** | | --- | --- | --- | --- | | All hosts | all or \* | all | ditwl-ec-front-end-001
ditwl-ec-back-end-123 | | One host | name of host | ditwl-ec-front-end-001 | ditwl-ec-front-end-001 | | Multiple Hosts | host1:host2 | ditwl-ec-front-end-001:ditwl-ec-back-end-123 | ditwl-ec-front-end-001
ditwl-ec-back-end-123 | | One group | name of group | tag\_cost\_center\_blue\_department | ditwl-ec-back-end-123 | | Multiple groups | group1:group2 | tag\_cost\_center\_blue\_department:tag\_cost\_center\_green\_department | ditwl-ec-front-end-001
ditwl-ec-back-end-123 | | Excluding groups | group1:!group2 | tag\_environment\_pro:!instance\_type\_t4g\_small | ditwl-ec-front-end-001 | | Intersection of groups | group1:&group2 | tag\_os\_ubuntu:&tag\_os\_ver\_23\_04 | ditwl-ec-front-end-001 | | | | | | #### Testing Playbook The file `ansible/testing.playbook.yaml` is used for testing host group selection. ```yaml - name: All hosts hosts: all remote_user: ubuntu tasks: - name: Ping all ansible.builtin.ping: - name: One host, by name hosts: ditwl-ec-front-end-001 remote_user: ubuntu tasks: - name: Ping ansible.builtin.ping: - name: Multiple Hosts, by name hosts: ditwl-ec-front-end-001:ditwl-ec-back-end-123 remote_user: ubuntu tasks: - name: Ping ansible.builtin.ping: - name: One group, hosts in group Cost Center blue_department (AWS hosts with tag cost_center=blue-department) hosts: tag_cost_center_blue_department remote_user: ubuntu tasks: - name: Ping ansible.builtin.ping: - name: Multiple groups, hosts in groups Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue-department or cost_center=green_department) hosts: tag_cost_center_blue_department:tag_cost_center_green_department remote_user: ubuntu tasks: - name: Ping ansible.builtin.ping: - name: Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g_small) hosts: tag_environment_pro:!instance_type_t4g_small remote_user: ubuntu tasks: - name: Ping ansible.builtin.ping: - name: Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04) hosts: tag_os_ubuntu:&tag_os_ver_23_04 remote_user: ubuntu tasks: - name: Ping ansible.builtin.ping: ``` #### Ansible Hosts Pattern Testing Show the **list of matching hosts for the patterns** but not executing the playbook: ```shell $ ansible-playbook testing.playbook.yaml --list-hosts playbook: testing.playbook.yaml play #1 (all): All hosts TAGS: [] pattern: ['all'] hosts (2): ditwl-ec-front-end-001 ditwl-ec-back-end-123 play #2 (ditwl-ec-front-end-001): One host, by name TAGS: [] pattern: ['ditwl-ec-front-end-001'] hosts (1): ditwl-ec-front-end-001 play #3 (ditwl-ec-front-end-001:ditwl-ec-back-end-123): Multiple Hosts, by name TAGS: [] pattern: ['ditwl-ec-front-end-001:ditwl-ec-back-end-123'] hosts (2): ditwl-ec-front-end-001 ditwl-ec-back-end-123 play #4 (tag_cost_center_blue_department): One group, hosts in Cost Center blue_department (AWS hosts with tag cost_center=blue_department) TAGS: [] pattern: ['tag_cost_center_blue_department'] hosts (1): ditwl-ec-back-end-123 play #5 (tag_cost_center_blue_department:tag_cost_center_green_department): Multiple groups, hosts in Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue_department or cost_center=green_department) TAGS: [] pattern: ['tag_cost_center_blue_department:tag_cost_center_green_department'] hosts (2): ditwl-ec-front-end-001 ditwl-ec-back-end-123 play #6 (tag_environment_pro:!instance_type_t4g_small): Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g_small) TAGS: [] pattern: ['tag_environment_pro:!instance_type_t4g_small'] hosts (1): ditwl-ec-front-end-001 play #7 (tag_os_ubuntu:&tag_os_ver_23_04): Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04) TAGS: [] pattern: ['tag_os_ubuntu:&tag_os_ver_23_04'] hosts (1): ditwl-ec-front-end-001 ``` #### Run the Ansible Test Playbook Run the playbook to test SSH connectivity: ```shell $ ansible-playbook testing.playbook.yaml PLAY [All hosts] ********************************************************************************************************************************************************************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] ok: [ditwl-ec-back-end-123] TASK [Ping all] *********************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] ok: [ditwl-ec-front-end-001] PLAY [One host, by name] ************************************************************************************************************************************************************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] PLAY [Multiple Hosts, by name] ******************************************************************************************************************************************************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] ok: [ditwl-ec-back-end-123] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] ok: [ditwl-ec-front-end-001] PLAY [One group, hosts in Cost Center blue_department (AWS hosts with tag cost_center=blue-department)] ******************************************************************************* TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] PLAY [Multiple groups, hosts in Cost Center blue_department or in green_department (AWS hosts with tag cost_center=blue-department or cost_center=green-department)] ****************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] ok: [ditwl-ec-back-end-123] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-back-end-123] ok: [ditwl-ec-front-end-001] PLAY [Excluding groups, hosts in environment pro not running instance type t4g_small (AWS hosts with tag environment=pro but not using an instance of type t4g.small)] **************** TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] PLAY [Intersection of groups, all hosts running Ubuntu version 23.04 (AWS hosts with tag os=ubuntu AND tag os_ver=23_04)] ************************************************************* TASK [Gathering Facts] **************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] TASK [Ping] *************************************************************************************************************************************************************************** ok: [ditwl-ec-front-end-001] PLAY RECAP **************************************************************************************************************************************************************************** ditwl-ec-back-end-123 : ok=8 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ditwl-ec-front-end-001 : ok=12 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ``` ### Troubleshooting #### Empty Ansible Inventory Ansible shows an empty inventory or the **hosts list is empty** when it has not been properly configured to use a dynamic inventory. ```shell $ ansible-inventory --list { "_meta": { "hostvars": {} }, "all": { "children": [ "ungrouped" ] } } $ ansible-playbook playbook.yaml [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' ``` Check that the `ANSIBLE_INVENTORY` environment variable has been properly configured to point to the dynamic inventory configuration file and the file name ends in .aws\_ec2.yml. ```bash export ANSIBLE_INVENTORY=$(pwd)/inventory.aws_ec2.yml ``` Make sure that the dynamic inventory is working, see the section on [how to test the Ansible dynamic directory](#test-the-ansible-dynamic-aws-inventory). #### Unprotected Private Key File Ansible checks that the Private Key file permissions are properly configured. ``` WARNING: UNPROTECTED PRIVATE KEY FILE! Permissions 0660 for '/home/user/keys/ditwl-kp-config-user-ecdsa' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. Load key \"/home/user/keys/ditwl-kp-config-user-ecdsa\": bad permissions\ Permission denied (publickey) ``` See [How to set the correct permissions for the Ansible Private Key file](https://www.itwonderlab.com/aws-key-pairs-terraform/#set-the-right-permission-to-the-file). #### The authenticity of host ‘IP/NAME’ can’t be established When running an ansible-playbook for the first time, Ansible opens an SSH connection to all targeted hosts. The message “The authenticity of host can’t be established” in SSH occurs when you try to connect to a server for the first time or if the server’s host key has changed. This message serves as a security measure to warn you about a potential man-in-the-middle attack. When you connect to a server for the first time, your SSH client doesn’t have the server’s host key stored locally. This host key acts like a digital fingerprint, uniquely identifying the server. Without this stored key, your client cannot verify the server’s identity and prevent the possibility of connecting to an imposter server. ``` The authenticity of host '3.94.214.81 (3.94.214.81)' can't be established. ED25519 key fingerprint is SHA256:mIBL6hfMMeEuaoxW+UL8JV3HaKShxFFLcgEYXaVD/3c. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes ``` If you trust that the connection is being done to the correct server, answer yes to add the key fingerprint to your `known_hosts` file. #### Instance type (name) is not supported in your requested Availability Zone When a minimal Terraform configuration doesn’t specify the availability zone for subnets, there is a possibility of randomly getting assigned an availability zone that doesn’t have all AWS Services or instance types. ``` │ Error: creating EC2 Instance: Unsupported: Your requested instance type (t4g.micro) is not supported in your requested Availability Zone (us-east-1e). Please retry your request by not specifying an Availability Zone or choosing us-east-1a, us-east-1b, us-east-1c, us-east-1d, us-east-1f. ``` Set the availability zone for the subnet and run again. ```hcl # Subnet resource "aws_subnet" "ditwl-sn-za-pro-pub-00" { vpc_id = aws_vpc.ditlw-vpc.id cidr_block = "172.21.0.0/23" #172.21.0.0 - 172.21.1.255 map_public_ip_on_launch = true availability_zone = "us-east-1c" tags = { Name = "ditwl-sn-za-pro-pub-00" } } ```