Hey everyone! I'm looking for insights on various Kubernetes distributions because we've been facing a lot of issues with our current setup using Tanzu. To give you some context, we've encountered a series of frustrating problems, especially during updates. We recently had a production cluster go down, and it took over two months to resolve the issue, only to find out that the root cause wasn't even identified. So, we're considering switching to something else and have narrowed down our options to Thalos, k3s, Rancher, Open Nebula with OneKE, and Kubermatic. We're primarily operating in an on-premise data center with VMware and we're looking for something that's stable, easy to manage, and has good support. I'd love to hear your experiences with these platforms and any recommended best practices. Thanks in advance!
9 Answers
I've had great experiences with Kubermatic; the team behind it is very knowledgeable. I've seen them at CNCF events and they really know their stuff. If you're thinking about it, definitely ask for a demo to see how it performs in your setup.
If you haven't heard of Cozystack, it's also worth checking out. It's built on Kubernetes and uses Talos as its base. It can spawn multiple tenant Kubernetes clusters, which could be helpful for your needs.
I've used Rancher in both production and testing environments, and it's been quite reliable. Upgrades have been simple for me. That said, I don’t have personal experience with their support, which might indicate that I've had no major issues - which is a good sign for the product!
Just a heads up, upgrades can be tricky in Rancher. I ran into some obscure bugs when upgrading from 2.5 to 2.6 that took some digging to fix, but overall the usability is great.
Talos has been a breeze to manage on my Proxmox cluster. After just a couple hours of setup, it's incredibly easy to keep everything running smoothly!
I had a similar experience! It was surprisingly simple to get going.
Just go with Talos. It simplifies everything—you won't have to manage package dependencies or tweak OS settings for Kubernetes deployment. It feels like having an appliance for your Kubernetes nodes.
I've been using Rancher in production for the past eight years and I wouldn't switch. Just be prepared—some of the others you’re considering actually utilize Rancher under the hood, which is interesting!
I’m hesitant about migrating from RKE1 to RKE2, especially if they plan another shift to RKE3.
I noticed that too. It makes me wonder about the stability of these alternatives.
We had a bumpy transition from Rancher 1.x to 2.x. Has it improved in the newer updates?
I've been using Talos in my home lab and pushed for it at work, but they opted for OpenShift instead. My experience with Talos has been amazing; everything from setup to upgrades has been seamless. I can't speak for their enterprise support, but they do actively engage in forums, which is a plus. I'd recommend checking them out!
Talos is also my top pick. I set up a small cluster and was really impressed by its straightforwardness. I think having a strong community is key if you're worried about support!
K3S has been my go-to for a while now. It's super lightweight, open-source, and I can upgrade my entire cluster with just one command. I also use it for demos to teach others about networking in Kubernetes, and the automation makes everything so much easier.
Talos along with Omni are fantastic if you're deploying quickly. I had everything up and running in just a week after setting everything up!
Is Omni part of the free offerings? I’d love to deploy that on bare metal.
I had Rancher for a year and their support was lacking, unfortunately. Just something to consider.