Hey everyone! I'm curious about what happens in Kubernetes when you don't have liveness or readiness probes set for your pods. How does Kubernetes manage situations where resource usage is really high? If a microservice is overwhelmed, how does eviction work in that case? I know kubelet has some built-in mechanisms, but I've also seen events that suggest probes are failing even when they aren't configured. Would love to hear your insights!
3 Answers
Good question! There are a couple of layers to this. First, if you don’t have CPU/memory requests or limits set, Kubernetes can have a tough time managing resources effectively. You can run into situations where pods hog resources, potentially disrupting others. Check out the Kubernetes docs on resource requests to understand how it affects scheduling. As for probes, without them, Kubernetes won’t know if your pod is healthy or not—it won’t report any failures because there’s nothing to check. I’ve learned this the hard way when a rogue pod took down my entire cluster!
Yeah, I had a similar experience too! An unrestricted pod took out a whole node before I put limits in place.
Just to clear up a point: if you're seeing probe failure events, it’s likely that there are checks happening. If no probes are defined in your pod spec, Kubernetes won’t record any success or failure events. Check if your app has internal health checks or if there’s another layer, like a sidecar proxy, affecting that. It seems like that’s what happened in your case with Istio.
Exactly! If you have a sidecar like an Istio proxy, it could be failing while not directly reflecting issues in your main application. It’s crucial to separate those checks to get a clear picture of what’s going on.
I get that not using limits can be tempting, but don’t you think having requests set helps with avoiding noisy neighbor issues?