Hey everyone! I'm currently managing a Kubernetes setup on DigitalOcean, which doesn't have Karpenter support. As a result, I'm using the Cluster Autoscaler and handling a lot of tasks manually, like capacity planning, node rightsizing, and topology design. My current strategy involves analyzing workload patterns, checking CPU and memory requests against actual usage, and categorizing workloads to create node pools that match these profiles for better binpacking. However, this approach is quite manual and can lead to errors. I'm looking for any tools or workflows that can help automate or streamline node rightsizing, binpacking strategies, and topology planning within the Cluster Autoscaler, especially for those of us on platforms like DO. I'd love to hear your tips or recommendations! Thanks for any help!
3 Answers
You're doing a lot right already! Here are some tips:
- Consider using message queues instead of direct HTTP requests; this simplifies scaling since you can autoscale based on the queue size.
- Try to limit the number of node groups, as having too many can slow down the Cluster Autoscaler significantly.
- Aim for fewer nodes in each Availability Zone to avoid noisy neighbor issues. The right number can depend on your specific use case.
- Larger nodes might offer better binpacking and reduced overhead for things like daemon sets, but they can be less flexible with autoscaling adjustments.
- If you can, implementing KEDA could make a big difference!
Have you checked out Cast AI? They provide node autoscaling options, even though DO isn't directly supported. You can still optimize your HPA and VPA configurations for your workloads and monitor costs across workloads and namespaces.
Make sure you're using the right metrics for scaling. Relying solely on CPU and memory might not give you the complete picture. Depending on your applications, you might need to monitor additional metrics that can be critical for scaling decisions.
You're spot on! We actually scale differently based on the service type. For instance, we scale single-threaded applications using HPA with CPU metrics, while databases get vertically scaled based on VPA recommendations, and we have web servers scaled by HTTP worker processes. It’s not perfect, but it’s been manageable. Thanks for the reminder to keep an eye on our assumptions!
Thanks for the solid guidance! We're already using RabbitMQ for background tasks, and KEDA has really improved our scaling based on queue length. I had no idea too many node groups could be a problem; I'll definitely look into consolidating them.