I'm building a home lab on a single physical machine and want to run a 3–4 node Kubernetes cluster as virtual machines, such as one control plane and three workers. Besides learning cluster operations, I'd like to use the setup for local LLM inference and possibly some fine-tuning or small-scale training.
What hardware would make this practical? I'm especially interested in recommended CPU core counts, RAM capacity, storage, and GPU or NPU options. I'd also appreciate specific mini PC or compact-build suggestions, preferably with workable GPU passthrough for isolating AI workloads inside a VM. I'm trying to stay budget-conscious but would spend more if the additional hardware provides a meaningful improvement for inference or experimentation. Real-world experience with similar setups would be very helpful.
5 Answers
A typical mini PC usually won’t include a useful dedicated GPU. For AI workloads, a small-form-factor desktop may be a better fit, or you could use a system with an OCuLink port and attach an external GPU. An RTX 3060 or better is a reasonable starting point because its VRAM can handle many 7B-class models, while fine-tuning and larger models benefit from substantially more VRAM. GPU passthrough can work, but it may be fiddly on compact systems, so running inference on the host and exposing it to the cluster may be simpler. A compact build with an RTX 5070 is suitable for inference and image-generation workloads, but limited VRAM can still make training difficult.
Running several virtual nodes on one host makes sense if the goal is learning Kubernetes scheduling, upgrades, networking, and cluster behavior. It doesn’t provide much real fault tolerance, though, because one host failure takes everything down. For maximum performance, you could also run k3s directly on the machine and avoid virtualization overhead.
If you plan to distribute one large LLM across multiple nodes, ordinary Ethernet will become a serious bottleneck. Model parallelism needs much more bandwidth and lower latency than most home networks provide. A single GPU with enough VRAM is generally preferable for inference; use the cluster for orchestration rather than splitting one model across several machines.
For a hands-on virtualized lab, Proxmox with VMs running Talos or k3s would be a good foundation. You can add a Proxmox-aware autoscaling provider to experiment with creating or removing worker nodes based on workload demand. Just remember that four VMs need enough memory for the operating systems, Kubernetes services, monitoring, storage components, and the AI workload itself. I’d target at least 64 GB of RAM, with 128 GB being much more comfortable.
A Minisforum MS-02 Ultra with a 235HX has worked well for my worker workloads. Mine has 32 GB of RAM and runs several Kubernetes workers with services such as Cilium, Longhorn, Prometheus, Argo CD, Valkey, and llama.cpp in a VM with the GPU attached. The CPU usually stays around 25% and only rises significantly during builds. The system supports up to 256 GB of DDR5, so expanding the memory is an option. I keep the control plane on a separate low-power N100 system, which leaves the larger machine focused on workers and workloads. For your single-host design, I’d prioritize expandable RAM, fast NVMe storage, and a discrete GPU with adequate VRAM over an NPU, since current AI software support is generally better on established GPUs.
The main goal is to simulate a small cluster and practice the orchestration side. I understand that separate physical machines would be needed for meaningful host-level redundancy.