Kubernetes Best Practices

Below is the list of Kubernetes Best Practices that have been implemented in the EKS infrastructure of the project.

Activate Kubernetes Metrics Server #

Kubernetes Metrics Server is essential since it allows access to Kubernetes metrics that can be retrieved by a metrics collector like Prometheus, and is used by Kubernetes specific tools such as VPA and HPA.

To activate it, use this simple command:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Uniform replicas distribution across Nodes #

The ideal in a cluster is to distribute the replicas evenly across all the Nodes. For that, add this in the Deployment specs:

affinity {
  pod_anti_affinity {
    preferred_during_scheduling_ignored_during_execution {
      weight = 1
      pod_affinity_term {
        topology_key = "topology.kubernetes.io/zone"
        label_selector {
          match_expressions {
            key      = "app"
            operator = "In"
            values   = ["product-hunting"]
          }
        }
      }
    }
  }
}

Horizontal Pod Autoscaler implementation #

📃 See Horizontal Pod Autoscaler implementation file →

HPA allows to autoscale the application pods horizontally (number of replicas). It has been implemented using a Terraform configuration file.

Vertical Pod Autoscaler implementation #

📃 See Vertical Pod Autoscaler implementation file →

VPA allows to autoscale the application pods vertically (compute resources allocated). It has been implemented using a yaml file (Terraform does not support VPA).

In production, the update mode must be Off in order not to let VPA autoscale the application: instead, VPA just monitors the application and gives a recommendation (dry run) on the compute resource to provide. The recommendation is visible either by doing a kubectl describe on the VPA or by using a tool such as Goldilocks.

Update strategy #

A Rolling Update strategy has been added to the Deployment so that the application is automatically updated by the Deployment when it is pushed to production:

strategy {
  type = "RollingUpdate"
  rolling_update {
    max_surge       = 1
    max_unavailable = 0
  }
}

When the application is updated, the Deployment creates a second Replicaset that takes care of replacing one by one each old Replica with a new one.

Mechanism to perform rollbacks #

To perform a rollback, simply use the kubectl rollout command:

kubectl rollout undo deployment product-hunting

The application will revert to the previous revision, which is essential when the last update makes the application unstable or even unusable.

This functionality is integrated into the pipeline via a job, where all it takes is to press a button.

Healthchecks with Probes #

Kubernetes has its own healthcheck system, as it provides Probes (Readiness, Liveness, Startup).

Two readiness probes have been deployed in the Deployment specs:

• One for the website port 443:

readiness_probe {
  http_get {
    scheme = "HTTP"
    path   = "/"
    port   = 443
  }
  initial_delay_seconds = 10
  period_seconds        = 15
  success_threshold     = 2
  failure_threshold     = 3
}

• And one for the API port 5000:

readiness_probe {
  http_get {
    scheme = "HTTP"
    path   = "/health"
    port   = 5000
  }
  initial_delay_seconds = 10
  period_seconds        = 15
  success_threshold     = 2
  failure_threshold     = 3
}