Hey there! I'm a DevOps/Platform Engineer with a few years of experience provisioning multi-tenant infrastructure manually using Terraform. It's pretty straightforward to create a tenant with modules, but coordinating with regional managers who are sometimes eight hours behind can be a hassle. To solve this, I built an internal 'Tenant Operator' that automates tenant creation and has been in production for about two years now. I recently published a cleaner version, Tenant Operator v1, which watches an external registry to create Kubernetes Tenant CRs automatically, uses Go templates for resource rendering, and tracks dependencies. I would love to hear suggestions or feedback on it, especially regarding any gaps you might find. For anyone who wants to test it out, there's a Minikube QuickStart guide available!
4 Answers
That sounds really cool! Have you considered using something like ArgoCD for templating? It could let your tenant registry create Helm applications instead, which might simplify things a bit.
Those integration guides you linked are great! They pretty much cover everything most setups will need.
If your tenants are not solely Kubernetes resources, using the Terraform Operator could be a viable solution. The custom operator you've created can adapt quickly to various needs with CR or label/annotation extensions, so there’s flexibility there!
I've worked on something similar! I built a [DB Query Operator](https://github.com/konnektr-io/db-query-operator) and found it extremely useful as I was dealing with the same multi-tenancy issues. It's awesome to see someone else tackling this problem.
That’s interesting! My Tenant Operator has been handling over 100 tenants for a couple of years now, so I'm curious about your design decisions compared to mine.
I decided to move away from multi-tenant clusters because they seemed to compromise user experience. Instead, we created a cluster vending machine with k0smotron that provisions separate environments for users. However, I still think the controller pattern for tenancy management is pretty fascinating!
I haven't used ArgoCD, but I think it could work! With tenant operators, you might already be able to define pre-rendered templates using the 'manifests' field, so it might be worth checking out.