I'm looking to set up a Kubernetes cluster in a remote, air-gapped environment where we'll be handling the hardware and bootstrapping ourselves—no infrastructure team support this time. The cluster is physically located in areas with limited connectivity, specifically satellite links, so being able to operate offline is crucial. I've been exploring options like Talos, k0s, and RKE2/Rancher. Have any of you worked in similar environments? What technologies would you recommend? I'm particularly worried about Talos and its troubleshooting challenges compared to traditional Linux distributions. Any insights or experiences would be greatly appreciated!
5 Answers
I would ditch Talos, k0s, and RKE2 if you want to keep things simple. Just go with kubeadm; it's reliable and gives you total control without unnecessary abstractions. You can write simple bash scripts to automate the process. Plus, you can maintain everything on a USB and transfer it when you're back at the site.
Funny how straightforward solutions sometimes get overlooked for the fancier tools, isn't it?
Consider looking into using RKE2 with ansible playbooks for your offline setup. There's good documentation out there, and it can help you create a more efficient air-gapped environment without adding too much complexity. Plus, MetalLB can help with accessibility for your cluster.
Thanks for the tip! I’ll definitely check out the ansible playbooks for better automation.
Yeah, RKE2 also seems like a solid choice if you want to keep things a bit less hands-on.
I definitely see the appeal of Talos, but I'd caution against it if you're looking for something straightforward. The troubleshooting can be more complex compared to standard distros, especially when you're in a sticky situation with an air-gapped setup. Managing everything manually might feel safer in those scenarios.
I'm glad you brought that up! It's my main concern—if the system fails, will I be left in the dark with Talos?
Running a setup similar to yours using Ubuntu with microk8s—it's been solid for about a year now. But with Canonical's recent plans, I’m thinking of switching to something else like RKE. We also need to comply with various regulations, which can complicate things further.
Good to know microk8s can handle air-gapped environments well. I might need to look at my next steps based on Canonical's changes.
You might also want to check out the Linux Foundation's Eve-OS project designed for edge computing. It could offer some useful lifecycle management features for your remote hardware.
Thanks for the suggestion! I wasn’t aware of Eve-OS and will definitely give it a look.
This approach seems legit! I've done something similar and it really keeps things manageable.