Kubernetes Metrics API Goes Stable: What It Means for DevOps

Kubernetes v1.37 has finally brought the metrics.k8s.io API to a stable release (v1), making it a more dependable source for CPU and memory usage data for nodes and Pods. This API, crucial for commands like kubectl top and resource-metrics-based autoscaling, now comes with all the stability guarantees you'd expect from a stable Kubernetes API. The good news? The v1 API keeps the same resource types and fields as v1beta1, so there's no need to worry about changes to the data you collect or return. For anyone focused on consistent performance monitoring and autoscaling, this is a pretty big deal.

Why This Matters

Stabilizing the Metrics API is a significant milestone for both cluster operators and app developers. With the API now stable, you can count on it for reliable performance monitoring and autoscaling tasks. This is really useful for keeping solid DevOps practices on track, as it fortifies the backbone for monitoring and scaling applications in production. Given its long use in production by tools like the HorizontalPodAutoscaler (HPA) and kubectl top, its stability is well-tested. This also reduces the headache of maintaining compatibility since its structure hasn’t changed.

A Long-Lived API Reaches Stability

Let's take a trip down memory lane. The Metrics API made its debut in Kubernetes v1.6 as an alpha feature, advanced to beta in v1.8, and has continued to prove itself in production environments. Now it graduates to stable in v1.37, showing just how reliable it has become. It offers two vital resource types:

• NodeMetrics: Gives you CPU and memory usage metrics for a node. It's crucial for assessing the load and performance of each node in your cluster.
• PodMetrics: Provides CPU and memory usage metrics for a Pod, with detailed usage per container in its containers field. Particularly handy for apps running multiple containers in one Pod, allowing for precise monitoring and resource allocation.

This setup makes it a key tool for monitoring and autoscaling, enabling users to collect necessary data without the need for a full-blown monitoring solution.

What Changed?

The leap to stable is notable, but changes are minimal. The API surface mirrors v1beta1 exactly; no new or renamed fields here. So, if you’re already using v1beta1, you can move to v1 without touching your code. For example, here’s how you'd pull node metrics from the stable endpoint:

kubectl get --raw /apis/metrics.k8s.io/v1/nodes

And for fetching pod metrics in a namespace, it’s as simple as:

kubectl get --raw /apis/metrics.k8s.io/v1/namespaces/default/pods

Did I mention that kubectl top supports both API versions? It will choose v1 if it's available and will automatically revert to v1beta1 in clusters not yet updated to v1, ensuring backward compatibility. This fallback is pretty crucial when not all systems transition at the same time.

What to Do on Monday Morning

As a cluster operator or developer, there's no need to enable any new feature gates. The Metrics API runs through the API aggregation layer, typically using an implementation like the metrics-server. To get the v1 metrics API up and running, make sure your implementation supports v1.metrics.k8s.io and that the corresponding APIService is registered. Here’s a quick checklist for a smooth transition:

1. Check if your current metrics-server supports v1. If not, think about upgrading to a compatible version.
2. Register the APIService for v1.metrics.k8s.io in your cluster, making sure your API aggregation layer is setup to serve the new version.
3. During this shift, it’s important to maintain both v1 and v1beta1 for compatibility with older clients. Verify the versions your cluster serves with:

kubectl get --raw /apis/metrics.k8s.io/ | jq .

4. After your metrics setup supports v1, confirm APIService availability using:

kubectl get apiservice v1.metrics.k8s.io

Following these steps will ensure your monitoring setup is fully functional and ready to scale. In my experience, supporting both versions during transitions helps minimize any disruptions and keeps everything running smoothly.

Limitations and Trade-offs

The Metrics API remains intentionally streamlined, focusing on resource metrics necessary for autoscaling and basic inspection. It's not a replacement for a comprehensive monitoring pipeline or the custom metrics (custom.metrics.k8s.io) API. If you need detailed monitoring beyond CPU and memory, you'll have to use additional tools or APIs. This is especially relevant where detailed application-level metrics are vital for performance tuning and debugging.

Also, the HPA controller currently only supports v1beta1. Although there's talk of adding discovery-based selection between v1 and v1beta1, it’s not an option in Kubernetes v1.37. So, while your setup can handle both versions, automatic selection isn’t possible yet. This might mean manual intervention or scripting to choose the version, a crucial point for automation workflows relying on seamless API transitions.

Engineering Judgement

From my perspective, adopting stable APIs when they become available can boost the reliability and predictability of your systems. The Metrics API has been stable in practice for years, so this official graduation is a great signal to lean on it even more. However, it’s smart to keep an eye on how things develop, especially with HPA support and custom metrics integration. Keeping tabs on Kubernetes release notes and community discussions will offer insights into future changes and their potential impact on your setup.

For DevOps folks, integrating the stable Metrics API can lead to more efficient resource management and possibly even lower costs through smarter autoscaling. If you’re involved in a migration or deploying new services, this API stability can simplify the process by offering reliable metrics for decision-making. Ensuring your team understands the API's capabilities and limitations is crucial to harnessing its full potential.

Get Involved

The Metrics API is in the hands of SIG Instrumentation. If you're interested in contributing or learning more, consider joining the #sig-instrumentation channel on Kubernetes Slack or attending a SIG Instrumentation meeting. Engaging with the community can give you insights into best practices and upcoming features that might affect your implementation. It also offers a chance to influence the API's direction and tackle any issues you encounter.