Earlier I posted about my hobby cluster on GKE which I want to keep under an affordable budget. Unfortunately Google Cloud will start charging a management fee from june 2k20 of 10$ct per hour (=$73/mnth) just like AWS. If they unilaterally change the rules, let’s get out of here!

I’m thinking of moving to a self-managed Kubernetes cluster on AWS with spot instances:

  • 1 x 1GiB master-node (t2.micro spot instance, $2.920/mnth)
  • 2 x 2GiB worker-nodes (t3.small spot instance, $5.256/mnth)

With a total estimated monthly cost of $13.43 (~€15.10 incl. VAT). So, let’s deploy a self-managed Kubernetes cluster on AWS using Kops.

Preparations

  1. Install Kops and configure AWS with IAM credentials;
  2. Setup of a Route 53 domain;
  3. Create an S3 bucket to store the clusters’ config and state, e.g. aws s3 mb s3://cluster.dobken.nl.

Create the cluster

It is not possible to create a cluster with spot instances directly from the kops command-line. What we do instead is generate a cluster manifest, change the configuration manually and use this modified manifest to create the cluster.

Create the cluster manifest in the file cluster.dobken.nl.yaml with settings for master and node instances:

kops create cluster \
    --name cluster.dobken.nl \
    --zones eu-central-1a \
    --master-size t2.micro \
    --node-size t3.small \
    --state s3://cluster.dobken.nl \
    --ssh-public-key ~/.ssh/kops_dobdata.pub \
    --dry-run --output yaml | tee cluster.dobken.nl.yaml

In the created yaml file, edit the InstanceGroups by adding lines of maxPrice: "0.20" just above maxSize:

  ...
  machineType: t2.micro
  maxPrice: "0.20"
  maxSize: 1
  minSize: 1
  ...
  machineType: t3.small
  maxPrice: "0.20"
  maxSize: 2
  minSize: 2
  ...

Create the cluster with the following commands:

kops create -f cluster.dobken.nl.yaml
kops create secret sshpublickey admin -i ~/.ssh/kops_dobdata.pub --name cluster.dobken.nl
kops update cluster --yes --name cluster.dobken.nl

Now, if you wait a minute or two, you’ll have acces to the new cluster:

$ kubectl get no
NAME                                             STATUS   ROLES    AGE     VERSION
ip-172-20-36-345.eu-central-1.compute.internal   Ready    node     48s     v1.16.7
ip-172-20-45-521.eu-central-1.compute.internal   Ready    master   2m38s   v1.16.7
ip-172-20-50-887.eu-central-1.compute.internal   Ready    node     59s     v1.16.7

Changes configuration

You can make changes to the yaml file and apply these changes to the cluster with:

kops replace -f cluster.dobken.nl.yaml
kops update cluster --yes
kops rolling-update cluster --yes

Resources