I'm working on ensuring compliance with CIS-hardened containers, but we're facing challenges with our current Alpine/distroless strategy. Whenever developers need specific packages, it creates issues, leaving us stuck between overly large compliant images that increase our CVE counts and minimal images that can't meet audit requirements. I'm curious if anyone has found a good compromise? I'm looking for ways to start with minimal images while still adding the necessary packages without compromising our hardening efforts. Daily rebuilds help keep things updated, but they don't solve the foundational compatibility issues. What strategies have you implemented where you need to satisfy both a minimal attack surface and specific compliance benchmarks?
5 Answers
We've had success by creating custom hardened base images using minimal Ubuntu and Docker multi-stage builds. In the first stage, we install only the required packages with specific apt pinning for versions. The second stage copies the binaries and strips out development dependencies. A key part of our process is automating CIS benchmark testing in our CI/CD pipeline using tools like Docker Bench and Trivy. This way, we can generate compliance reports automatically so auditors see we're meeting the benchmarks even with custom images. We also maintain a 'golden image' registry with pre-approved base images that developers can expand on. If we encounter package conflicts, we document exceptions with risk assessments and compensating controls, emphasizing that auditors appreciate knowing our risks over achieving perfect compliance. Monthly rebuilds keep us on top of patches, which is a pain to maintain, but it's better than dealing with auditor issues every quarter.
This makes a lot of sense. Auditors need to see the implications of any exceptions so they can properly frame their reports. It’s key to show that you understand the risks involved.
We operate primarily in a Red Hat environment, so we find it natural to start with UBI (Universal Base Images). However, if your needs extend to RPMs that aren't included by default, things can get tricky.
> Well, if you need RPMs that aren't already baked in, it works if you build and run on licensed hosts. But things get really messy with something like Google’s Container-Optimized OS.
You might also consider purchasing images if the budget allows.
If you want to stick with Alpine as your base and face package shortages, consider opening an issue on the Alpine Linux GitLab for any missing packages. Alternatively, you can contribute APKBUILDs for better control if you're willing to maintain them. It's no guarantee that the packages will be created, but it will at least alert someone to the need.
Have you thought about breaking down what makes the applications compliant and creating your containers based on those benchmarks? It seems there's a disconnect, as compliance should be factored in early in the development process. These requirements should be established with development teams upfront to ensure a smoother workflow.
Thanks, this is giving me a whole new perspective.