I'm working at a tech company that has a significant presence on AWS, and we set up a single EKS cluster for each region where we deploy products. Our goal is to achieve maximum bin packing efficiency. In our larger regions, we usually maintain clusters with over 2,000 nodes (think 12-48xl instances) and often have more than 20,000 pods running. When workload demands increase, we scale up to nearly double this amount. I'm curious about how common it is for a single EKS cluster to reach this scale. We've experienced some API server demands issues occasionally, but it's not a regular problem. I wonder how much larger we can or should scale before we need to split into multiple clusters.
4 Answers
You're approaching the limits for Kubernetes. Just a heads up on the rough thresholds:
- Maximum 110 pods per node
- No more than 5,000 nodes
- Up to 150,000 total pods
- A cap of 300,000 total containers
These limits are outlined in the Kubernetes best practices, so keep those numbers in mind!
I see large clusters fairly often in my work at AWS. However, scaling isn’t just tied to node and pod counts; it’s heavily influenced by how much load is on the API server. For instance, if you have high pod churn or use resource-heavy tools, that can create complications. My colleague gave a detailed presentation on this topic at KubeCon last year if you're interested in diving deeper!
Clusters of that size are pretty uncommon, from my experience. Most I’ve seen typically range between 10 to 100 nodes, often managed with multiple clusters per region or even different accounts with one cluster each per region. Sure, AWS has autoscaling for control plane components, but I bet there are practical limits you’ll hit eventually.
If you're planning to exceed 5,000 pods, it might be wise to consult with AWS. They could help you ensure your core services scale appropriately to prevent any mishaps.
Not extremely common, but there are a few clusters I've seen that handle around 80-100k CPUs. We're working on projects to reduce scale for reliability and cost, however, like rewriting core apps and distributing workloads across more clusters. By the way, what’s your typical CPU utilization at that scale?
If you don't mind me asking, what’s your job title?
In AWS EKS, you can manage about 220 or even 230 pods per node if you're using larger node sizes.