I'm developing a multi-tenant platform using Kubernetes where users can launch their own pods. My challenge is finding the best pattern to track these pods while keeping the business state and Kubernetes state interconnected. Should I use a Controller with Postgres, or is an Operator with Custom Resource Definitions (CRD) the better approach? I'm looking for something similar to what platforms like vast.ai or runpod.io do. Any insights on the best practices here?
4 Answers
Honestly, stick with the Operator + CRD approach for tracking. It's a smart way to tie together Kubernetes state with your business metrics without having to solely rely on a database like Postgres for everything.
You might want to consider using namespaces for each tenant to maintain isolation. Alongside that, labels and annotations are perfect for adding metadata to your pods. But remember, while they help with organization, they won't replace the need for a solid business logic framework to keep everything aligned.
Namespaces per tenant really do help with isolation, especially in a multi-tenant setup. Just keep in mind that while those labels and annotations are metadata helpers, they won't provide context for business logic on their own. You definitely want that Operator functionality if you need to keep both state levels in sync.
For tracking user-launched pods in a multi-tenant setup, I'd recommend using the Operator pattern along with Custom Resource Definitions (CRDs). This approach allows you to monitor and react to changes in the Kubernetes state, which is essential for syncing with your business logic.
That's a great point! It sounds like combining namespaces for isolation with an Operator for state management could be a solid path forward.