What AI Skills Should an AKS Observability SRE Learn First?

A very practical first project would be to run a vLLM pod on a spot GPU node in AKS and collect GPU metrics with the NVIDIA DCGM exporter. You’ll quickly learn how GPU memory pressure and KV-cache growth affect the workload, sometimes before a regular HPA reacts. It’s a good way to get familiar with GPU provisioning, scheduling, monitoring, and the failure modes that are specific to inference.

The vLLM and GPU exporter approach is a good starting point, but don’t focus only on hardware metrics. Treat the model server like any production service: monitor request latency, time to first token, queue depth, failed requests, retries, throughput, and cost per request when available. Correlate those signals with GPU utilization, memory, and Kubernetes events. Breaking the deployment under load and then building dashboards that explain what happened will teach you more than following a generic AI roadmap.

I’d learn GPU scheduling and monitoring in AKS first, then explore a serving framework such as KServe to understand how inference requests move through Kubernetes. For observability, build on your existing distributed-tracing knowledge and learn how to connect model-level metrics with infrastructure signals. The tooling is still evolving, so being able to separate useful application signals from noisy or incomplete telemetry is especially valuable.