Skip to content

HPA and Prometheus

Kubernetes

HPA

Horizontal Pod Autoscaler automatically scales the number of pods in a replication controller, deployment, replica set or stateful set based on observed CPU utilization (or on some other application-provided metrics). To use this you need to have the label metrics_server_enabled=true.

As examples are available upstream, we will skip the introductory part. This is a stepping stone for the example that follows on HPA with custom metrics.

Prometheus-Adapter

To use custom metrics as a source for the HPA we need to expose metrics in the kubernetes custom.metrics.k8s.io or external.metrics.k8s.io endpoints. Prometheus-adapter is a component that works as an add-on to the prometheus server and, as such, depends on the monitoring_enabled=true label. Refer to the prometheus monitoring to set up the required infrastructure. Prometheus-Adapter comes enabled by default and is configurable through the following labels:

  • prometheus_adapter_enabled (default: true) - if the adapter is enabled or disabled
  • prometheus_adapter_configmap - the name of the configmap to use (this overwrites the default configmap installation)

Let's check what metric endpoints we have available with:

$ kubectl api-versions | grep metrics
custom.metrics.k8s.io/v1beta1 (this metrics come from the prometheus-adapter)
metrics.k8s.io/v1beta1 (this metrics come from the metrics-server)

You can list all the default available custom metrics with:

$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1"
{"kind":"APIResourceList","apiVersion":"v1","groupVersion":"custom.metrics.k8s.io/v1beta1","resources":[]}

To create a HPA definition based on custom metrics you can follow this tutorial.

Exposing Custom Metrics

Some times, the defined default custom metrics are not enough and you need to add a diferent grouping or a diferent time window for example. In this situation you can consult the prometheus-adapter documentation on how to configure and make available the desired metric.


Last update: February 24, 2020