I'm exploring the idea of creating a couple of tools for Kubernetes that could make it easier to handle unschedulable pods in a multi-cloud environment. The traditional methods—like resizing nodes in one cloud or manually spinning up VMs—aren't ideal for cost optimization or multi-cloud scenarios.
My concept includes two tools:
1. Tool A, which I'll call the "price brain," gathers VM types and hourly pricing from various cloud providers and normalizes them into one currency. It would provide recommendations based on user-defined constraints like min vCPU, region, and max price.
2. Tool B is a Kubernetes controller that monitors for unschedulable pods. It requests the cheapest matching instance from Tool A and provisions it, allowing those pods to run. Once the workload decreases, it would handle deprovisioning.
I'm curious if this idea of a cost-conscious, multi-cloud bursting solution resonates with you. Would it be practical for real-world scenarios, or is it more of an academic exercise? What concerns or use cases should I consider?
2 Answers
I think this is a solid approach, but you might want to consider existing solutions like Karpenter. It optimizes workload efficiency by automatically watching for unschedulable pods and provisioning the right nodes based on resource requests. Your concept seems a bit too similar unless you have a unique angle, like multi-cloud capabilities.
One thing to keep in mind is that egress costs can really add up in a multi-cloud setup. The outbound traffic from a cloud provider can really spike compared to traffic between nodes in one zone. Also, latency could be a concern. It seems like multi-cloud bursts could work better for specific workloads, like batch jobs, that are self-contained, where you're mostly looking at one-time data transfers instead of constant communication between nodes.
Thanks for pointing that out! I agree that Karpenter handles some of this, but my focus is on multi-cloud, where nodes could come from various providers depending on cost. I see it as a sort of cost broker for Kubernetes, rather than just another autoscaler.