I'm relatively new to Kubernetes and have been working with k3s for a couple of months after outgrowing my Docker Compose/Swarm setup. I've grasped the basics and successfully incorporated FluxCD and Cilium into my k3 cluster. Currently, I'm developing a WebRTC app with a few services like Postgres and NATS, and I'm also using STUNner thanks to the k8s ecosystem. I know some might argue that I could continue using Docker Compose or Swarm, but I'm hoping to future-proof my application. As a one-person team, cost optimization is very important to me.
The big question I have is whether I should stick with a lightweight and flexible self-managed k3s stack or transition to Google Kubernetes Engine (GKE). On one hand, k3s offers me full control, potential cost savings (like moving to Hetzner), and the ability to keep my prod and non-prod clusters similar in design. On the other hand, using GKE could lead to significant costs as I scale up, and I'm concerned about losing the nimbleness of k3s.
I'm curious to know—how difficult and time-consuming is it really to manage a self-hosted k3s setup? I've tried GKE and already feel the pressure to control costs due to things like external load balancers and monitoring expenses. Should I focus on setting up high availability and disaster recovery with k3s, or am I being naive about the issues that self-management might bring? Any insights or reality checks would be greatly appreciated!
1 Answer
Honestly, focusing on future-proofing during development might not be the best use of your time. The fact that you’re using containers and a service-oriented architecture already sets you up well for transitioning to k8s when it’s necessary. At this stage, sticking with a straightforward container host could actually be more beneficial.
I get your point, but it feels like a chicken-and-egg dilemma. I’d rather learn k8s now when I have time instead of stressing about it later under different pressures. Given that a single VM could suffice, could a single-node k3s setup be a reasonable compromise?