I'm using EKS with Karpenter and Datadog. I'd like one view of which nodes were consolidated and where their pods ended up, but the Karpenter logs give me a pod count rather than pod identities. Kubernetes events show evictions and scheduling, though replacement pods can have different names. Is there a practical way to put this together?
Can I see Karpenter consolidation and pod replacements in one dashboard?
Asked by
p3+ratchet_716
Answers
You can put the Karpenter logs and Kubernetes events on the same Datadog dashboard, but the pod mapping will need correlation rather than a simple join. Extract node names and timestamps from the disruption logs, then look for eviction and scheduling events around that window.
A pod is not literally moved. It is terminated and a replacement may be created elsewhere with a new name and UID. The old UID can identify the evicted pod, but it won't match the replacement. Owner references and timing can help associate both pods with the same workload, though that won't always prove a one-to-one replacement.
Reply to this answer
StatefulSets and Deployments need different treatment here. StatefulSet pod names are stable across replacements, so the name can make a before and after view easier. Deployment replacement pods usually have different names, so grouping by their owning workload is more useful than relying on pod names alone.
Reply to this answer
What decision is the dashboard meant to support? If the concern is disruption, I'd start with application health and consolidation frequency rather than trying to reconstruct every pod replacement. A perfect movement map could take a lot of work without making the underlying problem clearer.
p3+ratchet_716:
The mapping would help connect consolidation activity with workload disruption. It sounds like a workload-level view may be enough for that.
Reply to this answer
For a quick node-level view, eks-node-viewer can show nodes being added or removed, pod counts and cost estimates. Grafana dashboards are another option for visualizing cluster activity, though neither suggestion by itself establishes which replacement pod corresponds to an evicted one.
Reply to this answer
If the main need is troubleshooting rather than a full migration map, consider sending Karpenter events into the same notification stream as deployment or reconciliation events. Seeing node recycling alongside a failure can narrow down what to investigate, even if it doesn't provide a precise pod-to-pod link.
Reply to this answer
Add your answer
Answers are reviewed before publication.
nearly-on-purpose:
That distinction matters when several replicas are replaced close together. A workload-level view may be more reliable than claiming each old pod has one definite successor.