I'm planning to transition a legacy on-prem datacenter to a Kubernetes-based setup. The goal is to move several Windows Servers running IIS into three on-prem Kubernetes clusters, with an additional cloud-based cluster for hybrid and failover scenarios. I'm considering using GitOps tools like ArgoCD, but I'm still deciding between that and Flux since I've dabbled in both.
I have three powerful bare metal servers to work with. Initially, I thought about setting up the control plane and worker nodes on each server using Talos, but I'm rethinking that for a production setup. Now, I'm weighing options: should I go for six physical servers (three for control plane and three for workers), or could I use Proxmox to manage one control plane and several worker nodes on each of the three bare metal servers? Using Proxmox could make it easier to manage remotely, which sounds attractive since I want to have development, staging, and production environments. This might lead me to use separate clusters for each rather than just namespaces, which would limit the need for more physical servers.
For networking and ingress, I have two ISPs, and I'm planning to route traffic through Traefik/MetalLB, setting up SSL termination at that point and managing SSL certs automatically.
Am I overthinking the whole VMs versus bare metal debate? I feel like the benefits of using VMs with Proxmox could outweigh the drawbacks, especially since I will have three physical machines available for redundancy.
4 Answers
I've been leaning towards using bare metal for my clusters, but I see the perks of virtualization. How many VM nodes are feasible on each physical server? I worry about the risk of having multiple VMs share a single point of failure. Scaling could be tricky without the right planning.
Since you're still figuring this all out, I advise optimizing for flexibility and go with VMs. If you have Proxmox ready to roll, that’s a great option. I recommend setting up separate clusters for dev, staging, and production instead of using namespaces alone; it simplifies upgrades and management significantly. And for networking, stick with something straightforward, like flannel, which pairs nicely with Talos.
I've been managing on-prem Kubernetes for over six years, and my advice is to go for virtualization. While there’s a minor performance hit with VMs, the benefits in terms of manageability and flexibility are huge. Dealing with physical servers can be a headache, and virtual machines are much quicker to work with. I'd also suggest starting with GitOps; it's invaluable and can save you a lot of headaches down the road. Check out Talos, too—it's a great choice for an on-prem environment.
I'm with you on using VMs! I've been running clusters in a virtual environment on Proxmox lately, and it’s been smooth sailing. You’ll need a solid storage solution though—Ceph is really useful, especially for stateful applications. Getting GitOps in place from the start can really streamline your operations and reduce overall complexity.
Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures