Hey everyone! I'm looking for advice on how to keep track of costs associated with my workloads running in Kubernetes clusters on AWS. We have a number of clusters, but I'm struggling to pinpoint exactly how much each namespace, deployment, job, or standalone pod is costing us. Additionally, I'm curious about identifying idle nodes since my clusters usually show CPU and memory usage below 20%. Even with tools like Karpenter and Spot instances in place, there seems to be a lot of potential for cost-saving improvements. Any tips would be appreciated! Cheers!
1 Answer
To get a clear picture of your spending, I'd recommend checking out OpenCost or Kubecost—these tools are great for breaking down your expenses by namespace or labels. AWS also offers a CUDOS Dashboard, but it won’t give you the granular details you’re looking for at the pod level.
However, as you've noticed, the real issue is your low utilization rate. Karpenter does excel at bin-packing, but it can only work with the requests that your pods specify. If your pods are over-requesting resources, you might end up with nodes that sit underutilized. To tackle that, you'll want to consider some rightsizing strategies. You might want to look into tools like VPA for adjusting pod requests, but be mindful that it can disrupt workloads. Workload-aware bin packing tools would also help optimize those 'unevictable' pods.
In my experience, automating rightsizing and actively managing node fragmentation are key to reducing costs without sacrificing performance. It sounds like you’re on the right path; just keep refining your inputs!
Thanks for the thorough breakdown! I’ve been considering options like PerfectScale for real-time adjustments, but it seems like integration with Karpenter isn’t seamless. I'm on the same page as you with regards to focusing on rightsizing rather than just visibility. It’s tough to justify extra costs for what might be temporary insights. If you're comfortable, could you share your experience comparing OpenCost and Kubecost? They seem to be the main contenders out there!