Hey everyone! I'm trying to set up a home lab where I can run some clusters on mini PCs using baremetal solutions like k3s, k0s, or Talos. My goal is to easily teardown and rebuild my clusters, similar to how it works in a virtual environment. So far, I'm considering using a PXE server that would allow each node to boot with a fresh image. I'm leaning towards Talos with machine configurations stored on the PXE, but I'm also exploring the idea of using a mutable distribution along with Ansible for bootstrapping and managing configurations. I would really appreciate any thoughts or recommendations you might have!
6 Answers
I personally use Cluster API (CAPI) with Metal3 as my baremetal provider, which pairs nicely with Image Builder for creating immutable node images. It's been effective for my needs.
For my setup, I simply went with Fedora, installed kubeadm, and took it from there. It was straightforward and worked well for me!
Before diving too deep, I'd suggest trying this out in a hypervisor like libvirt. Trust me, you might end up in a reboot loop, and that could be quite a hassle. You could also check out Tinkerbell, which is great for Kubernetes-native bare-metal provisioning. Good luck!
If you choose Talos, just run this command when you want to reset everything: `talosctl reset --system-labels-to-wipe EPHEMERAL,STATE --reboot --graceful=false --wait=false -n `. This wipes the node clean except for the OS, making it easy to start fresh!
I set up Ubuntu and used Terraform to bootstrap everything. It’s been a productive way to learn about infrastructure as code!
We offer a PXE boot utility called 'booter' which you can find on our GitHub page. It makes the process pretty smooth. Here’s a video demonstrating how it functions!
We’ve also got infrastructure providers that handle both bare metal (like IPMI) and virtual options (like Proxmox, kubevirt, etc.) if you’re looking for centralized management and UI.