Hey folks! I'm currently planning to transition from a Docker-based setup to a Kubernetes cluster on bare metal, and I could really use some insights from anyone who's made a similar move. Here's a brief overview of my existing setup: I have a few independent Linux servers managing multiple proxy servers, a PostgreSQL database running several Docker containers for different environments, a Windows server for MS SQL, a monitoring/logging server, a dedicated GitLab Runner for CI/CD, and an Odoo CE system. Lately, I've noticed that this structure has become quite fragmented, which has been causing downtime and scalability issues.
I'm aiming to build a unified bare-metal Kubernetes cluster with 6 nodes. The goal is to centralize services, improve observability, and learn more about Kubernetes. Importantly, we want to do this without using cloud services, and the budget is zero.
The planned cluster will have 1 control plane node and 5 worker nodes, all with 32GB RAM. We intend to deploy multiple Spring Boot, Django, Laravel apps, alongside the Odoo system and a few other tools. What I'm stuck on is deciding between k3s and full Kubernetes (k8s). I've heard that k3s is lightweight and easier to manage, which could fit our smaller team well, but full k8s might be better for long-term growth and experience. I'd love your thoughts on this:
- Is k3s a good fit for our needs considering potential growth?
- Any issues I should be aware of when deploying k3s or k8s on bare metal?
- Suggestions for tools, monitoring stacks, and networking options would be greatly appreciated!
- Is there anything essential that I'm missing in my evaluation?
- Any posts or links you recommend for further reading?
2 Answers
Have you checked out Talos Linux? It's fantastic for bare-metal clusters! I personally love using Cilium for the networking stack, and they have a solid guide for integration. If you're looking for a lighter GitLab alternative, you might want to look into Gitea. Just my two cents!
I’m also on board with Talos; it’s been working wonders for my setup!
I'd definitely recommend Talos as well. It simplifies a lot of the kubernetes setup for bare metal, especially with its lifecycle management features! For your setup, consider having at least 3 control plane nodes for high availability, which will help with etcd replication. Also, don’t forget about effective storage solutions; Rook-Ceph could be a great fit. Don't host things like Vault in your cluster itself. And utilize external secrets management for better security.
Can you explain more about how you use the system-upgrade-controller with Talos? That sounds intriguing!
Totally agree with you! I switched from a mixed setup to Talos, and it’s been a game changer for me.