Should I Choose Managed Kubernetes or Bare Metal for My Clusters?

I've managed both cloud and on-prem clusters, and though EKS is easier to work with, the Amazon-specific quirks can be a headache. On the other hand, bare metal clusters tend to perform better on a core basis. Honestly, for businesses not doing complex operations, managed services often save a lot of stress and resources, making them worthwhile even with a slightly higher price tag.

I think if you have a solid team, running Kubernetes on bare metal can actually be cost-effective and scalable. But you definitely need a dedicated team for it to be successful. For companies that have a lot of externally-facing workloads, the cloud might be easier to scale up, but there are security concerns with managed services that you have to think about too, especially in regions like the EU.

It really depends on your workloads. While managed Kubernetes can offload a lot of the control plane management and simplify upgrades, costs can vary significantly based on how much you're actually using it. If you opt for a managed service, you might save on the engineering side since a smaller team can handle multiple clients more easily. However, bear in mind you might be trading off cost for control and flexibility.

If you're looking purely at costs, bare metal does have the upper hand. However, managing physical servers can become quite complex and may require significant effort, which is what many organizations prefer to avoid by opting for managed services. So, it's a balance between cost and the convenience of having someone else manage the infrastructure.

From my experience deploying on bare metal, once you set everything up, it runs smoothly. We often use basic distributions like RHEL or Ubuntu Server with k3s. Sure, it's air-gapped and we only access it via VPN, but it works without much hassle. The clients handle the patching of the servers while we focus on the app side.