I'm facing pressure from compliance for CIS-hardened containers, but our use of Alpine and distroless images complicates things when developers need specific packages. We're caught in a dilemma between using larger compliant images that increase our CVE count and smaller images that don't meet audit requirements. Has anyone found a balance? I'm looking for strategies that allow us to start with minimal images but add necessary packages while still maintaining a strong security posture. Our daily rebuilds help keep patches up to date, but they don't solve compatibility issues. What solutions have worked for your organization when auditors require both a minimal attack surface and adherence to compliance standards?
5 Answers
One effective method we've used is creating custom hardened base images, particularly with a minimal Ubuntu setup and leveraging Docker multi-stage builds. The first stage installs only the necessary packages using apt with specific version pinning, and the second stage copies over the binaries while stripping away dev dependencies. We've automated CIS benchmark testing in our CI/CD using tools like Docker Bench and Trivy, which allows us to generate compliance reports automatically. This way, auditors can see that we're meeting the benchmarks with our custom images. We also keep a "golden image" registry of pre-approved base images for developers to extend as needed. For package conflicts, we document exceptions and conduct risk assessments. Auditors tend to care more about understanding your risks than about achieving perfect compliance. If a specific package compromises hardening, it's crucial to document why it's necessary and how you're mitigating the associated CVEs.
Exactly! Auditors appreciate knowing the context of any exceptions. They might not be experts, but they value a thorough risk analysis.
Have you thought about redefining how compliance is approached? Instead of just trying to meet compliance after development, consider working with approved packages from the start. It should be part of a DevSecOps process where developers only have access to vetted stacks and libraries. Compliance needs to align with the product team to shift discussions earlier in the development process.
If you're keen on sticking with Alpine, you might consider opening an issue on their GitLab page to request missing packages. Alternatively, you could submit APKBUILDs for more control, although it does mean you'll have to keep track of them yourself.
Another option is to simply purchase pre-built compliant images, which could save you a lot of hassle in meeting audit requirements.
Since we're a Red Hat environment, using Universal Base Images (UBIs) makes a lot of sense for us. That said, if you need RPMs that aren't already included, it could get tricky.
True, that can be messy, especially if you're aiming to run something like Google’s Container-Optimized OS on your cluster.
Thanks for sharing that, it really reshapes my approach to this issue!