Terraforming your Infrastructure

This is not a fully fledged tutorial. This is merely an introduction to the benefits that Terraform can offer you.

What is Terraform

Terraform is a Cloud Orchestration language. It offers a simple syntax to define infrastructure for cloud providers such as AWS, Azure, and GCP. Cloud providers offer lots of tools for setting up and provisioning your systems. Unfortunately, they're usually not compatible with each other. Leaving you locked-in their platform!

In order to keep the option to switch providers as smooth as possibly, we need a tool that can provision our infrastructure in every available cloud provider. Furthermore, we want to define our infrastructure as code. That way, we can monitor our infrastructure changes through a version control system. The tool we've chosen is Hashicorp's Terraform. It's popular, well-maintained, and works great with their other product Packer.

Syntax

# Configure the AWS Provider
provider "aws" {
  region = "<your region>"
  # Please refer to the Terraform docs for more authentication options
  shared_credentials_file = "/Users/<username>/.aws/creds" # Use /home/<username>/.aws/creds for Linux
  profile                 = "default"
}
# Define a constant
locals {
  mongos_cluster_name = "mongos-rs0"
}
resource "aws_instance" "EC2Cluster" {
  # Count 1 means it will be treated as an array
  count = 1
  # You can use the custom AMI you've made in our previous blog post
  ami = "<insert custom AMI ID here>"
  # Define the instance type
  # This needs to be an instance with the same architecture as your AMI. In our case that is ARM
  instance_type = "t4g.micro"
  iam_instance_profile = "<Your IAM profile>"
  tags = {
    # Tag your instances dynamically based on a local variable and resource count
    Name = "${local.mongos_cluster_name}-${count.index}"
    Cluster = local.mongos_cluster_name
    # Set some metadata
    Environment = "test"
    "Application Role" = "database"
    MongoRole = "mongos"
    Port = 27017
  }
}

This basic configuration is enough to launch an EC2 instance in AWS. Of course, you can create instances using the AWS console manually or using CloudFormation. But Terraform has a few massive benefits. For one, Terraform's syntax can be used for most large cloud providers, and even allows for custom plugins for any provider not yet integrated. This means your team only needs to know one syntax to launch instances across multiple platforms.

Secondly, Terraform's configurations can be maintained in a version control system. Which is important, because we can have a repository with all of our configurations. We can then monitor any changes we make to the infrastructure. This is the basics of "Infrastructure-as-Code".

Dynamic features

The Terraform configuration syntax is just that, a configuration language. It's not a programming language by any means, but that doesn't mean we can't do anything dynamically. Locals can be used to define constants that can then be used in our configuration. This is great for defining cluster names that you want to use for multiple resources.

locals {
  mongos_cluster_name = "mongos-rs0"
}

Variables are similar but can be overwritten when running terraform apply. These are great for when you need to be able to quickly switch between instance counts or regions when deploying your infrastructure.

variable "ec2_instance_count" {
  default = 2
}
variable "aws_region" {
  default = "eu-central-1"
}

Besides variables, you can also use any information from existing resources in new resources.

The following configuration creates new EBS volumes for as many instances as you launch. There's no need to edit this configuration based on the amount of instances that you want. This is great when you quickly want to spin up a cluster in another region.

# Create a volume based on a backup
resource "aws_ebs_volume" "data_backup" {
  # Create as many volumes as you would do instances
  count             = var.ec2_instance_count
  # Assign them the same availability zone as the instance you would attach them to
  availability_zone = "${element(aws_instance.EC2Cluster.*.availability_zone, count.index)}"
  type              = "gp3"
  # Grab the snapshot from a variable
  snapshot_id       = var.ebs_snapshot_id"
  # Tag them accordingly
  tags = {
    Name = "data_volume_${count.index}"
  }
}
# Attach volume to instance
resource "aws_volume_attachment" "data_backup" {
  device_name = "/dev/sdh"
  # Again, use the same amount of attachments as instances
  count = var.ec2_instance_count
  # Use the index to map a volume to the right instance
  volume_id   = "${aws_ebs_volume.data_backup.*.id[count.index]}"
  instance_id = "${element(aws_instance.EC2Cluster.*.id, count.index)}"
}

Notice how we use metadata from the aws_instance resource to define a configuration for the volumes.

Another great example is writing to the SSM Parameter Store. You could create a parameter which contains a list of IP addresses of your cluster, which can be great for auto-discovery in applications.

resource "aws_ssm_parameter" "ReplicaSets" {
  name        = "/test/cluster/${local.shard_svr_0_name}/replicas"
  description = "List of servers"
  type        = "StringList"
  value       = join(",", aws_instance.MongoCluster.*.private_ip)
  tags = {
    environment = "test"
  }
}

Conclusion

Terraform is a great tool to create a single source of truth for your infrastructure over multiple platforms. It allows us to keep track of complex infrastructure changes and adapt quickly. I hope this introduction has inspired you to try it out for yourself! Check it out on their Getting Started tutorial.

As always, don't feel shy to contact us if you have any question, suggestion or correction to this post. Get in touch via our Support Channel.