Hey everyone, I'm part of a small Platform team (just three of us) that supports five major clients, each with their own namespace on our single RKE2 cluster. Right now, everything is running smoothly, and we manage the complexity without much trouble. However, three of our clients are interested in deploying their own operators on the cluster. We're a bit hesitant because the existing operators we have in place are shared and responsible for tasks that affect all client namespaces, like Kyverno.
We're concerned that allowing additional operators could increase maintainability issues. We've considered shifting the responsibility of managing these operators to the clients themselves, but they want to concentrate on their application development. One idea is to only allow new operators that benefit all five clients, but that still adds to the complexity of our platform. Another option could be to split our one cluster into five separate ones, but that raises new complications, especially if one cluster requires a specific operator.
I'd love to hear how you all deal with situations like this or if you've ever faced similar challenges. Looking forward to your insights!
4 Answers
You might want to consider a multi-tenant setup. Keeping one main RKE2 cluster while deploying Kamaji can give you individual control planes for each client. This way, you can tailor resources as needed while maintaining control over the overall environment.
It's great that your clients are interested in operators, but I think a crucial step is to clarify what they want to achieve with them. Instead of just jumping in, ask them about the specific problems they’re trying to solve. You already have Kyverno for some governance, but other tools like network policies or service meshes may be necessary down the line depending on how complex their needs get.
I totally get where you're coming from. In a shared environment, multi-tenancy can really complicate things. Have you thought about setting up vClusters for your clients? They can install their own CRDs without affecting others, but you might want to prepare for debugging challenges. Ideally, you should consider a managed Kubernetes service for your clients if the demands keep growing. It helps abstract away a lot of the complexity!
Definitely look into using OPA Gatekeeper! It can help block the creation of cluster-scoped resources, keeping operations confined to your clients' namespaces. Also, sensible RBAC policies will go a long way to maintain security and organization. This way, you can allow some flexibility while minimizing risk.
Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures