The Horizontal Pod Autoscaler operator reloaded

One of the main advantages of the Pipeline platform is that it allows users to use their infrastructure cost effectively; Telescopes helps with cluster and machine instance recommendations, Hollowtrees enables SLA-aware cost reduction using spot instances, and autoscalers allow for multi-dimensional autoscaling based on custom metrics. This post will highlight some new features of the Banzai Cloud Horizontal Pod Autoscaler Kubernetes Operator and the advanced automation supported by Pipeline - a new, forward-thinking way to operate Kubernetes clusters and autoscale deployments.

Learn more about the different types of autoscaling features supported and automated by the Banzai Cloud Pipeline platform platform:

• Autoscaling Kubernetes clusters
• Vertical pod autoscaler
• Horizontally autoscale Kubernetes deployments on custom metrics
• Horizontal Pod Autoscaler Kubernetes Operator
• The Horizontal Pod Autoscaler operator reloaded
• Horizontal Pod Autoscaling based on custom Istio metrics

tl;dr:

• An introduction to redesigned HPA annotations in the Horizontal Pod Autoscaler Kubernetes Operator
• The Deletion of HPA resources in the event that annotations are completely removed
• The HPA & Metrics Server recently released for Amazon EKS available in Pipeline
• An introducion to the new Pipeline HPA API

Redesigned HPA annotations

We redesigned the annotation naming scheme in order to provide us greater flexibility and to comply with Kubernetes annotation naming standards. Let's see an example of the new annotation system in action:

hpa.autoscaling.banzaicloud.io/minReplicas: "1",
hpa.autoscaling.banzaicloud.io/maxReplicas: "3",
cpu.hpa.autoscaling.banzaicloud.io/targetAverageUtilization:
"70"
memory.hpa.autoscaling.banzaicloud.io/targetAverageValue:
"1000Mi"
pod.hpa.autoscaling.banzaicloud.io/custom.metric.name:
"1000Mi"

As you can see, there's a separate subdomain for cpu, memory and custom pod metrics. Now you can specify percentage or absolute quantity values for both cpu and memory metrics.

• cpu.hpa.autoscaling.banzaicloud.io/targetAverageUtilization: "{targetAverageUtilizationPercentage}" - adds a Resource type metric for CPU with targetAverageUtilization set as specified, where targetAverageUtilizationPercentage should be an int value between [1-100]
• cpu.hpa.autoscaling.banzaicloud.io/targetAverageValue: "{targetAverageValue}" - adds a Resource type metric for CPU with targetAverageValue set as specified, where targetAverageValue is a Quantity.
• memory.hpa.autoscaling.banzaicloud.io/targetAverageUtilization: "{targetAverageUtilizationPercentage}" - adds a Resource type metric for memory with targetAverageUtilization set as specified, where targetAverageUtilizationPercentage should be an int value between [1-100].
• memory.hpa.autoscaling.banzaicloud.io/targetAverageValue: "{targetAverageValue}" - adds a Resource type metric for memory with targetAverageValue set as specified, where targetAverageValue is a Quantity.
• autoscale.pod/custom_metric_name: "{targetAverageValue}" - adds a Pods type metric with targetAverageValue set as specified, where targetAverageValue is a Quantity.

HPA & Metrics Servers available on Amazon EKS

Amazon just announced that they've enabled aggregations on the API Server, thus the [HPA & Metrics Server is now available on EKS] (https://aws.amazon.com/blogs/opensource/horizontal-pod-autoscaling-eks/). Unsurprisingly, we've been keen to add this to Pipeline as well. As usual, the whole process is automated and our end users don't need to do anything, since we install the Metrics Server automatically. This is not a straight forward process, so here are a couple of notes for those of you who are going to attempt this manually, instead of using Pipeline:

• be sure to use the latest (0.3.0) version of gcr.io/google_containers/metrics-server-amd64
• don't forget to configure security settings in order to allow the cluster control plane to communicate with pods running extension API servers on port 443, and, vice-versa, to allow pods running extension API servers on port 443 to receive communication from the cluster control plane as well. Checkout our latest Cloud Formation template for reference.

Pipeline HPA API

If you'd like to autoscale your Kubernetes deployments, by default, you'll need to add the correct annotations manually. This is consistent with Pipeline as it existed a few months ago, and the first version of the Horizontal Pod Autoscaler Kubernetes Operator relied on these manual inputs as well. The process has been simplified in the current Pipeline user interface, as it generates these annotations, but the cluster operator still has to copy generated annotations into the right locations in the values.yaml files. This process is better than the default HPA (especially now that we're generating proper YAML snippets), but based on feedback from our customers, we've realized that there's still room for improvement. So, we decided to create a separate page to manage HPAs for all Pipeline deployments, and, in order to automate that process, we designed a new Horizontal Pod Autoscaler API, which actually creates, updates and manages annotations on all Kubernetes Deployments and StafulSets. Let's examine a sample of how we might manage HPA for a deployment:

• query resources contained in a released Helm chart, for example a K8s deployments:

  GET
  {{url}}/api/v1/orgs/{{orgId}}/clusters/{{cluster_id}}/deployments/quaffing-dachshund/resources?resourceTypes=deployment

• query HPAs bound to this deployment specifying deployment name in scaleTarget:

  GET
  {{url}}/api/v1/orgs/{{orgId}}/clusters/{{cluster_id}}/hpa?scaleTarget=morbid-catfish-zeppelin

• create and update HPAs

  POST
  {{url}}/api/v1/orgs/{{orgId}}/clusters/{{cluster_id}}/hpa
  
  { "scaleTarget": "morbid-catfish-zeppelin", "minReplicas":
  1, "maxReplicas": 10, "cpu": { "targetAverageValueType":
  "percentage", "targetAverageValue": "60" }, "memory": {
  "targetAverageValueType": "quantity", "targetAverageValue":
  "1100Mi" }, "customMetrics": { "customMetricName": { "type":
  "pod", "targetAverageValue": "700m" }, "customMetricName2":
  { "type": "pod", "targetAverageValue": "700m" } } }

• delete HPA

  DELETE
  {{url}}/api/v1/orgs/{{orgId}}/clusters/{{cluster_id}}/hpa?scaleTarget=morbid-catfish-zeppelin

Our main goal with the Horizontal Pod Autoscaler Kubernetes Operator is to bypass the complexities of HPA. Pipeline platform users should only have to think about the best metrics for autoscaling based on their deployments, and Pipeline will automate the rest. Happy Autoscaling!

About Banzai Cloud Pipeline

Banzai Cloud’s Pipeline provides a platform for enterprises to develop, deploy, and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operations teams alike. Strong security measures — multiple authentication backends, fine-grained authorization, dynamic secret management, automated secure communications between components using TLS, vulnerability scans, static code analysis, CI/CD, and so on — are default features of the Pipeline platform.