AWS Systems Manager Session Manager with Terraform: SSH Without Open Ports

Access instances without SSH #

The traditional way to administer an EC2 instance in a private subnet is a bastion host, an open SSH port and key pairs (key pairs, SSH). AWS Systems Manager Session Manager replaces all of it:

• No inbound ports: the instance opens an outbound HTTPS connection to the SSM service.
• No SSH keys or bastion host to manage.
• Access is controlled with IAM, and every session can be logged to CloudWatch or S3 and appears in CloudTrail.

Requirements #

1. The SSM Agent on the instance (preinstalled on Amazon Linux and recent Ubuntu AMIs).
2. An instance profile with the AmazonSSMManagedInstanceCore policy.
3. Network access to the SSM endpoints: through a NAT gateway or through VPC endpoints (ssm, ssmmessages and ec2messages).
4. The user needs IAM permission ssm:StartSession, and the AWS CLI Session Manager plugin installed.

Instance role and profile #

data "aws_iam_policy_document" "ec2_assume" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["ec2.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "ssm" {
  name               = "ditwl-ssm-instance"
  assume_role_policy = data.aws_iam_policy_document.ec2_assume.json
}

resource "aws_iam_role_policy_attachment" "ssm_core" {
  role       = aws_iam_role.ssm.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

resource "aws_iam_instance_profile" "ssm" {
  name = "ditwl-ssm-instance"
  role = aws_iam_role.ssm.name
}

Instance with no SSH access #

resource "aws_security_group" "app" {
  name   = "ditwl-app"
  vpc_id = aws_vpc.main.id
  # No ingress rules: SSH is not needed
}

resource "aws_vpc_security_group_egress_rule" "https" {
  security_group_id = aws_security_group.app.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "tcp"
  from_port         = 443
  to_port           = 443
}

resource "aws_instance" "app" {
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = "t3.micro"
  subnet_id              = aws_subnet.private[0].id
  vpc_security_group_ids = [aws_security_group.app.id]
  iam_instance_profile   = aws_iam_instance_profile.ssm.name

  metadata_options {
    http_tokens = "required"   # IMDSv2
  }

  tags = {
    Name = "ditwl-app"
  }
}

The security group has no inbound rules at all, and no key_name. Outbound 443 is enough for the agent. See security groups.

Connect #

$ aws ssm describe-instance-information \
    --query "InstanceInformationList[].[InstanceId,PingStatus]" --output table
$ aws ssm start-session --target i-0abc123def4567890

The first command lists instances registered with SSM, and PingStatus must be Online. If the instance does not appear, check the instance profile, the network path to the three endpoints and that the agent is running.

Port forwarding to a private database #

$ aws ssm start-session --target i-0abc123def4567890 \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{"host":["mydb.abc.eu-west-1.rds.amazonaws.com"],"portNumber":["5432"],"localPortNumber":["15432"]}'
$ psql -h localhost -p 15432 -U app mydb

It reaches an RDS database in a private subnet through the instance, with no bastion. You can also use ProxyCommand to run normal ssh over Session Manager.

Log the sessions #

resource "aws_cloudwatch_log_group" "sessions" {
  name              = "/ssm/sessions"
  retention_in_days = 90
}

resource "aws_ssm_document" "session_prefs" {
  name            = "SSM-SessionManagerRunShell"
  document_type   = "Session"
  document_format = "JSON"

  content = jsonencode({
    schemaVersion = "1.0"
    description   = "Session Manager preferences"
    sessionType   = "Standard_Stream"
    inputs = {
      cloudWatchLogGroupName      = aws_cloudwatch_log_group.sessions.name
      cloudWatchEncryptionEnabled = false
      idleSessionTimeout          = "20"
    }
  })
}

Setting the document named SSM-SessionManagerRunShell replaces the account's default preferences, so create it only once per account and region.

Restrict who can connect #

Allow ssm:StartSession only on instances with a tag, which keeps developers out of production:

data "aws_iam_policy_document" "developers" {
  statement {
    actions   = ["ssm:StartSession"]
    resources = ["arn:aws:ec2:*:*:instance/*"]

    condition {
      test     = "StringEquals"
      variable = "aws:ResourceTag/Environment"
      values   = ["dev"]
    }
  }

  statement {
    actions   = ["ssm:TerminateSession", "ssm:ResumeSession"]
    resources = ["arn:aws:ssm:*:*:session/$${aws:username}-*"]
  }
}

The $$ escapes the AWS policy variable so Terraform does not interpret it. See IAM roles and policies.

Beyond sessions #

The same agent runs commands and patches at scale: Run Command, State Manager associations and Patch Manager. Combine it with scheduled automation and use it instead of provisioners to configure servers.

Cost #

Session Manager has no additional charge. You pay for the VPC endpoints if you use them (endpoints and costs) or for the NAT gateway.