I've heard Kubernetes can run just about anywhere, from bare metal to the cloud, and even in some wild scenarios. My real question, though, is about workload distribution: Is it feasible to have a Kubernetes cluster where the master nodes are on-premises while the worker nodes run in cloud providers like AWS or GCP? Or would that just lead to insane latency issues? Should I be considering separate clusters and multi-cluster management instead? I'm looking for a clear answer beyond the usual hype.
4 Answers
Honestly, asking why you'd do that is valid. The Kubelet talks a lot to the control plane, and if your nodes are spread out too far, you're just asking for trouble in production. It might be fun to experiment with, but I wouldn't rely on it for anything serious.
Yes, you can set this up without massive latency issues, as long as the data centers aren't too far apart. However, watch out for ingress fees; those can add up quickly!
Yes, you can mix them up, but I have to ask why you'd want to! I once tried linking two locations with Wireguard for a cluster setup, and let’s just say it didn’t go well.
It's definitely possible to do this for your specific workload. You could try running more control planes in different clouds and configure your network plugin to connect them. With a few adjustments, grouping workloads should work fine! Check out some resources about multi-cloud setups to get started: Kubermatic, Google Cloud's Kubernetes Engine multi-cloud docs, or some Terraform tutorials.
I get that! But think about it: if some of your workloads aren't time-sensitive, like batch processing, or if there are seasonal demands, then it could make sense. You might not want to commit to on-prem capacity when spot instances can save the day.