Which gateway would you pick for internal LLM demos?

For non-production use, LiteLLM sounds like a reasonable starting point. I’d keep the evaluation focused on whether it handles your provider mix and gives you the access controls and usage visibility you actually need, rather than optimizing for production features you won’t use yet.

If you don’t need the surrounding MLOps features, TrueFoundry may be more than this use case calls for. Portkey is another option if you want a lighter setup for routing and usage tracking. I’d try the simplest one that meets your access requirements before committing to a bigger platform.