Hey everyone! We're facing a unique situation where we need to keep the Horizontal Pod Autoscaler (HPA) active, but we want to control how we scale down our pods. Specifically, we only want to scale down pods that are not currently doing any calculations, which can take up to 20 minutes. From my research, it seems like Kubernetes' HPA doesn't support this kind of targeted scaling, and I've found that KEDA doesn't handle it either. Has anyone implemented a Custom Pod Controller to tackle this issue? Thanks for your help!
4 Answers
You might want to give KEDA's scaled jobs a try. They work great for scaling workloads based on events and states, which could align well with your needs.
If you have a setup where one component handles calculations and another needs to stay running, consider separating them into different pods. As someone already mentioned, you can use Jobs for the calculation tasks and keep your main service as a Deployment.
It sounds like you're trying to manage scheduled jobs. Have you looked into how Kubernetes Jobs work? You can create a job template that gets triggered when certain conditions are met. This might be just what you need! 😊
Absolutely, I was blind. Thanks!
Have you checked out any resource management platforms on a free trial basis? I can't say for all options, but ScaleOps can definitely scale workloads using numerous factors, detailed policies, and even scheduled timings. It could be worth a try if you're interested!
Good point. We are using Kafka, so it will be easy to trigger the scaled jobs.