How would you handle LLM traffic from Kubernetes apps?
We’re considering a way to manage LLM calls from Kubernetes workloads, including provider routing and keys, team budgets, audit logs, and checks for sensitive data or disallowed topics…
We’re considering a way to manage LLM calls from Kubernetes workloads, including provider routing and keys, team budgets, audit logs, and checks for sensitive data or disallowed topics…