Kubernetes clusters typically have robust pre-deployment controls in place, but runtime threats still find a way to infiltrate, particularly through service accounts and dependencies. I'm curious to hear how others are monitoring the behavior of their live clusters to address these risks. What tools or strategies are you using?
4 Answers
We've realized we haven't been doing enough for runtime security, so we're enhancing our model with security admission control, better base images, and network policies. We're also focusing on orphan detection to ensure all resources in our clusters are managed through GitOps. After implementing these steps, we'll pivot to actual runtime monitoring using Aqua, Splunk, and Cilium.
For monitoring unexpected processes that execute inside containers or for detecting privilege escalation, tools like Cilium and Tetragon are fantastic. They allow you to track commands like curl or wget through logs too.
I highly recommend using Falco. It's pretty straightforward to set up and run, and it does a great job of monitoring for runtime threats.
If you're looking for a solution, Redhat ACS (formerly Stackrox) is a solid choice for runtime security.
That sounds interesting! Are you utilizing any open-source solutions for detecting orphans (I've heard them referred to as zombies)? I find it tricky since I heavily use controllers, and it seems more complex than just enabling prune, especially when not all resources track ownership.