I'm currently on the hunt for successful case studies from companies that have developed internal AI platforms, specifically for creating AI agents. I'm interested in knowing which tools you all have used! Here's a quick rundown of my requirements: I need support for OIDC and OAuth2, data isolation per namespace, and an easy-to-use UI for quick prototyping and testing. Additionally, I'm looking for a customer-friendly UI, similar to ChatGPT, and MCP server support for integration with tools like VS Code and Cursor. Ideally, I would prefer open-source solutions, but it's not a strict necessity. So far, I've discovered LangFlow, but it lacks OAuth support (I see this feature is under PR). Any alternative suggestions?
4 Answers
There are a few tools you can consider, but none seem to tick all the boxes perfectly. LangFlow has a nice UI for prototyping, but it’s still waiting on OIDC/OAuth support. Flowise is lighter and easier to customize, but it lacks multi-tenant isolation and enterprise authentication. Haystack is great for building custom agent pipelines, though you’ll need to figure out your own UI and authentication. For something more infrastructure-focused, NudgeBee, Resolve AI, and Neubird offer stronger Kubernetes integration. If you're looking to manage scaling and namespace isolation, KubeRay coupled with Ray Serve could be a good fit, but it does require some extra work on the front end and for authentication. So it really depends on whether your priority is a quick UI or more infrastructure control!
I’m actually working on something that fits your description at [Okteto](https://okteto.com/ai), though it’s a commercial product with a free tier for small teams (not open-source). If you're interested in your own development, you’re on the right track. A few additional things to consider could be workload isolation at the namespace level, quota management, and features like garbage collection to help manage unused resources. Also, think about integration options for GitHub or GitLab, as well as a CLI or admin panel for managing agents and monitoring usage.
If your OAuth needs are just for a frontend UI, you might find oauth2proxy helpful. It works well enough and is worth checking out. Here's their [GitHub page](https://oauth2-proxy.github.io/oauth2-proxy/) for more details!
From what I've seen in building internal platforms, having OIDC on the cluster, strict namespace isolation, and a solid default infrastructure setup (like ingress, certs, storage, and metrics) can lead to success. We’ve actually set up a rough prototype here if you're interested in experiencing a basic version: [Reliable AI Platform](https://reliable.luthersystemsapp.com/?utm_source=chatgpt.com). Any feedback, even the tough stuff, is welcome!
We actually use oauth2proxy with Istio, and it does the job decently. I know there might be fancier options out there, but I had to move fast to beat the more 'creative' solutions being tossed around at work.