We're facing a significant increase in our licensing costs—up 300% even after some negotiation. This has led us to consider alternatives to minimize our expenses. Currently, we have Kubernetes running on Linux virtual machines in our vSphere environment using Rancher, and while we have some workloads in Tanzu, they're not critical. I'm curious if there's a solid reason we shouldn't convert part of our ESX servers to run Debian and deploy Kubernetes directly on bare metal. This change could potentially save us a couple of hundred thousand dollars annually. Has anyone else had experience with this kind of transition?
5 Answers
Have you thought about switching to another hypervisor, like Proxmox? It might offer you some cost savings too.
Keep in mind, the amount you're saving could equal the salary of a competent engineer. Sometimes, it’s cheaper to outsource than to manage everything in-house, especially with the associated risks and costs.
You should consider treating bare metal like cattle instead of pets. Implementing strategies like PXE, Redfish, or IPMO for deployment can help. Also, template your OS installs, and consider using tools like Packer for your pipelines. I personally recommend Lenovo's XClarity Administrator for managing physical servers—it’s really effective. Once you treat bare metal as you do in the virtual realm, it makes deploying Kubernetes more manageable.
If you're going bare metal, remember that you’ll need to manage the node lifecycles for Kubernetes upgrades. Kairos is currently the only open-source distribution that handles that well. So, think about how you'll manage upgrades if you decide to make the switch.
You might want to explore Harvester if you’re considering switching up your infrastructure. It’s been working well for others. It's worth checking out!
I switched from VMware to Harvester a year back and honestly, I have no regrets. It's been great!