I'm in the process of planning a migration from a legacy on-prem datacenter to a Kubernetes-based infrastructure. The goal is to transition various Windows Servers running IIS to three on-prem Kubernetes clusters and ideally one cloud-based cluster to create a hybrid environment for failover. I'm considering using GitOps for deployment management through tools like ArgoCD or Flux, with a preference for ArgoCD given my brief experience with both.
I've got three powerful bare metal servers available and I'm debating whether to dedicate them as control planes and worker nodes or to install Proxmox on them, allowing each server to host one control plane and multiple worker nodes in VMs while still using Talos. The intention is to manage everything remotely, which I think Proxmox could facilitate, making scaling and monitoring easier during upgrades.
I'm also thinking about how to structure my environments—having separate clusters for development, staging, and production seems more straightforward with VMs rather than using namespaces. For networking, I have two ISPs and plan to direct traffic to the Kubernetes clusters using Traefik/MetalLB, with SSL termination handled at this point.
Am I overthinking this? Is the trade-off of running on Proxmox worth it for ease of management, considering the redundancy from having three separate physical machines?
3 Answers
I second the VM approach. With Proxmox, you can easily manage your nodes remotely which is a huge advantage during cluster operations. And yes, definitely set up separate clusters for dev, staging, and production environments. Using GitOps initially might complicate things, especially as you're learning, so make sure you have a good grasp of the basics first. Also, don't overlook a solid storage solution—something like Ceph could be invaluable for handling stateful applications.
You might find that using VMs is a better fit for your needs now. It allows for easy scaling and management. With Proxmox, you can create new clusters quickly without having to add more hardware. As for your networking, I'd keep it straightforward and stick with something like Flannel or Kube-router in the beginning until you're more comfortable with Kubernetes. Just make sure your network policies are solid before you expand the setup!
I've been managing on-prem Kubernetes for over six years, and I'd definitely recommend going the VM route with Proxmox. You'll sacrifice a tiny bit of performance, but the manageability and flexibility you gain make it worthwhile. Instead of dealing with physical servers, which can be a hassle, VMs are much quicker to work with. Just remember to implement GitOps right from the start—it's a lifesaver for managing your workloads. Also, consider using Cilium over MetalLB for your networking; the BGP configuration works well with it. Hope this helps!
Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures