I'm trying to improve the CPU and memory settings for several Kubernetes services, including Java applications, and would like to compare this approach with how others run production workloads.
My current CPU requests are around 200–500m, but steady-state usage is often closer to 20m, so I suspect the requests are unnecessarily increasing my node count. CPU limits are around 800m–1000m, and they seem to cause CPU throttling during normal traffic as well as slow or unstable Java startup while the JVM is doing class loading, JIT compilation, and framework initialization.
Memory requests of roughly 600Mi–1000Mi appear to match observed usage. However, my memory limits are higher than the requests, and I'm questioning whether that provides useful burst capacity. Since memory is not reclaimable in the same way as CPU—especially for JVM heap—extra allocation could increase the risk of evicting or OOM-killing other workloads on the node.
I'm considering lowering CPU requests to around 30–50m and either setting CPU limits very high, perhaps 3–4 cores, or removing them entirely. For memory, I'm considering setting the request based on real usage and keeping the limit close to or equal to the request.
Does this reasoning hold up? In particular, if a pod has no CPU limit, can it consume the whole node during a spike and starve other workloads, or do CPU requests provide enough protection? I'd also appreciate practical guidance on monitoring and periodically adjusting these values, especially for Java services running without CPU limits.
4 Answers
A common starting point is to size requests around the workload’s p90 or p95 usage, then revise them as production behavior changes. Set alerts for workloads that stay above their requests for a sustained period, such as 30–60 minutes, and periodically check whether the requests are consistently much higher than actual usage. Workers, queue consumers, and jobs often need separate treatment because their usage patterns are less predictable.
For many services, CPU limits are omitted to avoid throttling. Memory is different: setting the memory request equal to the limit gives the pod a more predictable QoS classification, while allowing a large memory burst can make node-level OOM situations harder to control. Some teams intentionally allow extra memory headroom, but that is a stability tradeoff rather than a free benefit.
Removing a CPU limit does not automatically let one pod permanently take the entire node. The scheduler reserves each pod’s CPU request, and the Linux CPU scheduler gives workloads with requests appropriate shares of CPU time. If other pods have requests configured, they retain access to those reserved amounts while unused CPU can be shared during bursts.
The node still has finite capacity, so a CPU-intensive pod can use spare cycles and may compete with workloads that have very small requests. For stronger isolation, use sensible requests, separate node pools, priority settings, or application-level concurrency controls. Also remember that CPU requests should reflect more than just the idle average if the application needs predictable latency during normal traffic.
If you require guaranteed QoS and the strongest protection against memory pressure, matching requests and limits is the conservative option, but it can require more capacity and therefore cost more. A practical compromise is to use measured memory requests, keep memory limits close to them for important services, and monitor working-set usage, OOM kills, restarts, and node memory pressure before deciding how much headroom is justified.
Tools such as Goldilocks or the Vertical Pod Autoscaler can use historical metrics to suggest or adjust requests and limits. They’re useful for finding obvious overprovisioning, but recommendations still need review for Java startup bursts, scheduled jobs, queue workers, and other workloads with unusual patterns. It’s also worth coordinating any right-sizing automation with horizontal autoscaling and node autoscaling so the systems do not constantly react to one another.
I’ve found these tools most useful as a recommendation and review mechanism first. I’d be cautious about fully automatic changes until workloads with unusual spikes have been excluded or tuned.
That makes sense. I’m mainly trying to avoid paying for oversized CPU requests while still keeping memory failures predictable.