What AI skills should an AKS observability SRE learn first?

A very practical first project would be running a vLLM pod on a spot GPU node in AKS and collecting GPU telemetry with the NVIDIA DCGM exporter and Prometheus. You’ll quickly encounter real issues such as GPU memory pressure and KV-cache growth that may appear before a conventional HPA reacts. Building the metrics pipeline, then load-testing and troubleshooting it, will teach you more than starting with abstract AI roadmaps.

Since you already understand Kubernetes and observability, I wouldn’t start with broad AI theory. Run an inference workload and approach it as an SRE problem. Deploy something like vLLM, put it under load, instrument it, and see how the behavior differs from a normal service. In addition to CPU, memory, and request latency, you’ll need to understand time to first token, tokens per second, queue time, concurrency, GPU utilization and memory, KV-cache usage, out-of-memory failures, and cost. Test what happens as concurrency increases, when requests begin queueing, and which resource becomes the bottleneck. That hands-on work will make GPU scheduling, autoscaling, model serving, capacity planning, and tracing much easier to understand. Your existing SRE experience is the foundation; you mainly need to learn the operational characteristics of inference workloads.

I’d begin with GPU scheduling and monitoring in AKS, since those are often the hardest infrastructure constraints for AI workloads. Then learn how a serving framework such as KServe handles the inference path in Kubernetes. For observability, combine ordinary service signals with AI-specific ones: latency, time to first token, queue depth, failed requests, retries, tokens per second, GPU saturation, and cost per request. Distributed tracing is still evolving in this area, so the useful skill is learning to correlate model-serving metrics with Kubernetes and infrastructure signals. Treat the model endpoint like any production service: deploy it, stress it, break it, and make the dashboards explain why it failed.