I'm curious to know how many Kubernetes clusters and nodes others have in their organizations, especially those of you working in similar-sized companies. We have over 100,000 customers in a global B2B SaaS setting, and I'm in a bit of a debate with our product engineering team about Kubernetes administration. I'm looking for real-world examples of how many clusters you guys are running and how many services each cluster supports (not including infrastructure services like ingress or DNS). I've shared my previous experiences and data, but they think my information is biased. I'd love to hear different perspectives on this!
5 Answers
We operate quite a bit more customers than that, but across various environments, I’d estimate a few hundred clusters overall. The largest can scale up to a few thousand nodes.
I think this question isn't as straightforward as it sounds. For instance, my 100,000 users may be heavily read-oriented, while yours might be write-heavy with a lot of background processing. You could manage 100k requests per second on just a 2-node cluster with 4 pods for an API, or you might need a massive 1500-node cluster. It really varies based on the use case.
And what does 'customer' even mean here? Are we talking about individual users or firms with hundreds of users? It really changes the context.
I have around 20 clients, each with two or three clusters. Most of those clusters support about five services, but one customer has been known to run 20 on each cluster.
From my experience at a university, we had hundreds of clusters. But when I worked at Disney Animation, we had thousands, and at Disney+, it jumped to tens of thousands. Size really doesn't dictate effectiveness or need in clusters.
Haha, that's what she said! But honestly, I'm curious about the actual use too.
We run two EKS clusters, each hosting around 25-30 microservices. We have about 30 nodes in each cluster of varying sizes, and we support 20 million customers monthly. Just to note, this question feels a bit off base.
Yeah, we have clusters for each customer, totaling over 60 across more than 20 AWS accounts. The specifics heavily depend on what you need for your use case.