I'm currently auditing the resource limits for nodes in my EKS cluster, specifically looking at around 40 applications. I need to identify areas where resources are being wasted and determine the right limits and requests for these nodes. I'm curious about what percentage above the average usage I should set the resource limits. For example, if an application typically uses about 531m of memory, but the limit is set at 1000m, that limit definitely needs adjustment. I'm wondering where to set it without being too close—would 600m be too low? Is there a general guideline to follow? The same goes for CPU; one service averages 10.1m cores, but its limit is at 1 core. Although CPU throttling won't crash an application, I want to ensure there's enough room for wiggle. Any recommendations would be greatly appreciated!
1 Answer
From my experience, a good approach is to set the requests just slightly above your app's average usage and keep the limits quite a bit higher, or perhaps set them very high or not at all. This is crucial because if the OOM killer starts looking for memory to reclaim, apps using more than they're allocated are first on the chopping block. For CPU, find a reasonable baseline for requests and then leave the limits higher to allow for burst usage. Vertical Pod Autoscaler and Goldilocks can provide useful insights into these settings. Just remember, workloads can vary greatly, and some need more careful tuning than others.
Absolutely, you want to be careful with limits since they can add costs. The idea is to not overshoot averages too much. You’re spot on—requests ensure a guaranteed minimum amount of resources, right?