I'm running a GitLab CI pipeline using a Kubernetes executor on AWS with auto-scaling groups. Each node has a total of 8 CPU cores and I've limited all CI job pods to request 2 CPU cores and limit to 4, meaning I can run 4 jobs at once per node. However, I've been noticing some performance issues with my CI, and I'm considering adjusting the CPU settings. If I set my CI job pods to request 2 cores and limit them to 4, what can I expect in terms of performance? I suspect that under light loads, jobs might run faster, but during heavy loads, performance may not change much. I'm aware that CPU behaves differently than RAM, and that Kubernetes employs throttling rather than hard limits. I'd love to hear your thoughts on how these settings would perform under varying node loads.
5 Answers
Your gut instinct sounds right. When the node isn't heavily utilized, your containers could take advantage of additional cores. But during peak loads, performance will likely stay the same. A common tip is to avoid setting hard CPU limits unless absolutely necessary.
It might be better to remove the CPU limits entirely so your pods can utilize any extra CPU available on the nodes. That way, if one job needs more resources, it can burst without being throttled.
I'd also suggest tracking CPU throttle seconds. This will show you how limits are affecting the performance of your containers. Additionally, consider checking out Kubernetes Quality of Service (QoS) concepts—it’ll give you insight into prioritizing resources based on your configurations.
Setting CPU limits could actually be a good idea for cost management and security. In case of compromised containers, limits can help prevent resource hogging that triggers autoscalers.
You're onto something! If your CPU resources are being fully used by your 4 jobs at around 2 cores each, increasing the limits won't boost output. But if there's variation in usage, higher limits can let heavier jobs take advantage of any unused capacity from lighter jobs.
Related Questions
How To: Running Codex CLI on Windows with Azure OpenAI
Set Wordpress Featured Image Using Javascript
How To Fix PHP Random Being The Same
Why no WebP Support with Wordpress
Replace Wordpress Cron With Linux Cron
Customize Yoast Canonical URL Programmatically