We're setting up a Kubernetes fleet on CloudStack and need to build our own node images. For now, we're using Kubernetes image-builder with Ubuntu as the base, but vulnerability scans report a large number of findings—many of them appear to be noise, such as obsolete Linux headers that were left installed. We're using Cluster API and want to establish a sensible hardening process. What tools and base operating systems are you using for node images, and which hardening steps have worked well for you?
5 Answers
Bottlerocket is also worth evaluating if its supported Kubernetes features and CloudStack integration meet your requirements. Its minimal, locked-down design reduces the amount of traditional image hardening you need to manage yourself.
I’ve had good results with Debian for golden Kubernetes images. Packer runs an Ansible playbook that applies the common Kubernetes configuration, updates the kernel, removes anything not required, sets the desired kernel parameters, and installs containerd and the kubelet. I also generate an SBOM as the final build step. The resulting image is under 700 MB, or roughly 300 MB compressed, and can join a cluster through cloud-init.
A practical approach is Packer with the CIS Ansible playbooks. They handle most of the common hardening tasks without requiring you to build everything yourself. Ubuntu 22.04 LTS is a reasonable base; after the initial setup, we remove unnecessary Snap packages and cloud-init components. For the stale-header findings, explicitly purge unused linux-image and header packages near the end of the build. That usually eliminates a lot of scanner noise.
Diskimage-builder is another option if you only need a disk artifact such as a QCOW2 image. It can produce the image directly instead of requiring a separate cloud environment to boot and prepare it, which may make the pipeline simpler and faster.
If you want a minimal operating system with very little to maintain, Talos is worth considering. Its immutable, purpose-built design removes much of the usual OS-hardening work. The main thing to verify is whether the current Talos integration and provider support fit your Cluster API setup.
That’s useful, although I’ve seen that some of the Talos provider options for Cluster API have changed, so I’d need to confirm the current integration before switching.
A 20–30 minute build doesn’t seem especially problematic if the images are rebuilt infrequently, but a direct disk-image workflow could still be worthwhile when iteration speed matters.