Azure budgets seem to alert after usage has already happened. What can actually limit the damage from a stolen credential or an agent stuck in a loop?
What actually stops runaway Azure OpenAI spend?
Answers
For agent loops, put a quota in the request path. API Management has an llm-token-limit policy that can enforce a token quota per key over an hour, day or month. Once the quota is exhausted, requests can be rejected.
That only covers traffic passing through the gateway. Make sure callers cannot bypass it and reach the model endpoint directly. A deployment tokens-per-minute limit is useful too, but it limits the rate, not the total. A loop can keep consuming at that rate all day.
Reply to this answer
Deal with the stolen key problem separately. Disable local key authentication and use Microsoft Entra authentication with managed identities where possible. You can enforce the setting with Azure Policy, and restrict network access to the applications that need the endpoint.
Inventory your key-based callers first. Check request logs, app settings and Key Vault references, then migrate them before turning key auth off. Otherwise you can break legitimate apps with authentication failures. Also, disabling local auth is not necessarily an instant cutoff. Old keys may continue working for a while after the change.
Reply to this answer
A budget alert can trigger an action group and automation that blocks access or shuts down a resource. That gives you a backup kill switch rather than just an email.
Don't treat it as a precise spending cap though. Cost data and budget evaluations lag behind consumption, so a lot of usage can happen before the automation fires. Test the shutdown and recovery path too, since legitimate traffic will be affected.
_v67~parcelcrank.4:
Key rotation can disrupt key-based callers, but it also breaks legitimate ones until they get the replacement. It is a recovery measure, not a substitute for a request-path quota.
Reply to this answer
Watch for direct client access when designing the gateway setup. A private endpoint or firewall restriction can help keep traffic on the intended path, but that can be awkward when end-user tools need to call the API directly.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Brinley:
Foundry AI gateway also offers project-level token quotas and rate limits. Check that the limits fit the traffic you actually expect before enforcing them.