I'm trying to figure out the best way to set up a small Kubernetes architecture for internal use. I've got just one physical server, but I want to make sure it's reasonably reliable for supporting a small to medium-sized company with about 50 users. Here are the hardware specs I'm working with:
- **CPU:** Intel Xeon Silver 4210R (10 cores/20 threads, 2.4GHz, Turbo, Hyper-Threading)
- **RAM:** 128GB (4 x 32GB RDIMM 2666MT/s)
- **Storage:**
- **HDD:** 4 x 12TB 7.2K RPM NLSAS (planning for RAID 10)
- **SSD:** 2 x 480GB SATA SSD (planning for RAID 1 for OS/VM storage)
- **RAID Controller:** PERC H730P (2GB NV Cache)
I'm considering two different approaches for Kubernetes:
1. **Option 1:** Set up 6 VMs on Proxmox: 3 control plane nodes and 3 worker nodes, potentially using Longhorn for distributed storage (even though all nodes would be on one physical host).
2. **Option 2:** Create a single control plane and worker node VM or just do a bare-metal install and run all pods there to utilize all available hardware resources.
My questions are:
1. With just one physical machine, should I virtualize multiple control planes and worker nodes, or keep it simple with a single node cluster?
2. Is RAID 10 (HDD) + RAID 1 (SSD) a good combination, or do you suggest a different approach?
3. For Kubernetes storage, is Longhorn the best option, or is there a more lightweight alternative for single-host reliability and performance?
Thanks for your help!
5 Answers
I'd just install Kubernetes via Snap on Ubuntu. It's been reliable in my setups, and it makes adding nodes later a breeze. Plus, patching is as easy as one command: `snap refresh --channel=1.34-classic/stable k8s`. This could be a great way for you to go.
Given that it’s a single machine, I think it's best to go with a straightforward setup using k3s. Why complicate things with multiple nodes when they all run on the same box? You'll just add unnecessary complexity without any real gain in reliability.
If it’s for a local environment, I'd suggest deploying everything directly onto the server or again, use k3s for simplicity.
I wouldn't trust the hardware RAID controller much since recovery can be tricky. I’d suggest using software RAID with mdadm instead. For your setup, install Talos Linux as your minimal Kubernetes OS on the RAID 1 SSDs. If Talos seems too unfamiliar, you can always use k3s on a stable Linux distro, like Debian LTS. Once Kubernetes is up, try using GitOps with ArgoCD for managing the rest of your stack. I'd also recommend using the 4 HDDs for Longhorn configuration.
If it’s just for local development, I’d throw everything on that server or stick with the lightweight k3s option. It’s easy and gets the job done.
I didn't know about Talos before; I'll definitely check it out!