Infrastructure as Code with Terraform

Instead of a manual process to setup and configure our technology stack, we want to automate this proces through software. Hashicorps’ Terraform (2014) enables a declarative (vs. imperative/procedural) approach to Infrastructure as Code.

Although it is claimed to be cloud agnostic, Terraform is certainly not. The configuration will depend strongly on the naming and organization of the specific provider. But we can make at least the top-level infrastructure more cloud-agnostic by using the following two-level approach:

base-level: all our networking (VPC, TransitGateway, Load Balancers), virtual infrastructure (Kubernetes!), DNS and Certificate Enrollment;
operations-level: backing services: storage, IAM roles, Kubernetes Services such as Monitoring.

We will walk through the setup of Kubernetes for a couple of cloud providers. In case you want to walk through the manual deployment, find instructions on the JupyterHub project page.

Best practices

A best-practice checklist:

all Terraform code is in a git-based repository that automatically triggers a CICD pipeline (Jenkins, Gitlab CI, CircleCI, etc.) for the stages validate and plan and provides the options to run apply and destroy per environment;
use a workspace per environment to minimize the risk of state corruption and allow simultaneous deployment to all environments;
the state is externally stored in a bucket (GCP/S3) and ensure locking during deployment (GCP/DynamoDB);

Manage multiple versions

Managing multiple versions of Terraform is simple with tfenv; installed with Homebrew:

brew install tfenv

This allows you to easily discover, install and activate any Terraform version:

$ tfenv list-remote
0.13.0-beta1
0.12.28
0.12.27
0.12.26
...
$ tfenv install 0.12.28
Installing Terraform v0.12.28
Downloading release tarball from https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_darwin_amd64.zip
################################################################################################################################################# 100.0%
Downloading SHA hash file from https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_SHA256SUMS
No keybase install found, skipping OpenPGP signature verification
Archive:  tfenv_download.ygXhca/terraform_0.12.28_darwin_amd64.zip
  inflating: /usr/local/Cellar/tfenv/2.0.0/versions/0.12.28/terraform  
Installation of terraform v0.12.28 successful. To make this your default version, run 'tfenv use 0.12.28'

Now switch to a different version with:

$ tfenv use 0.12.28
Switching default version to v0.12.28
Switching completed

Project structure

To target a specific environment, it is good practice to separate each in its own folder. Also, make use of modules to avoid repitition.

README.md
.gitignore
modules
  ├── iamrole
  └── loadbalancer
environments
  ├── develop
  │   ├── backend.tf
  │   ├── main.tf
  │   ├── outputs.tf
  │   └── variables.tf
  ├── acceptance
  │   ├── backend.tf
  │   ├── main.tf
  │   ├── outputs.tf
  │   └── variables.tf
  └──production
      ├── backend.tf
      ├── main.tf
      ├── outputs.tf
      └── variables.tf

You can find a .gitignore example here.

Infrastructure as Code with Terraform#

Best practices#

Manage multiple versions#

Project structure#

Infrastructure as Code with Terraform

Best practices

Manage multiple versions

Project structure