Looking for a Gateway/Proxy to Handle Spending Limits on Azure OpenAI

January 9, 2026 2 answers
Asked by TechWizard42
I'm currently using Azure OpenAI for a couple of projects and have encountered a major concern: Azure doesn't provide a built-in hard spending limit on the resource or API level that would automatically disable the service once a defined dollar amount is reached. The Budget Alerts in Azure Cost Management only notify you but don't cut off the service in real-time. I'm searching for a self-hosted or open-source gateway or proxy that I could place between my applications and the Azure OpenAI endpoint to help manage this. Here's what I need: - A hard spending limit that allows me to set a maximum budget (for example, $50/month) and gives an error response (like 429 or 402) when that limit is reached. - Compatibility with Azure OpenAI API by supporting specific headers and deployment routing. - The ability to estimate costs in real-time based on usage (prompt and completion tokens). - Ideally, it should be lightweight enough to run within a Docker container or as a lightweight service using Go, Node.js, or Python. It would be great if it could also: - Manage different budgets for various API keys or users. - Include a simple dashboard for tracking current month-to-date spending. - Preferably, be open source with an MIT or Apache license.

Answers

DataDreamer
Have you looked into Litellm? It seems to fulfill quite a few of the requirements you have listed, including cost management functionalities.

TechWizard42: Perfect! I'll explore that option, thanks a lot!

Reply to this answer
DevGuru99
You might want to check out Microsoftโ€™s Azure API Management. It seems like a solid option for your needs. API Management can serve as the intermediary between your apps and the OpenAI endpoint, and from what I understand, it has a quota-by-key policy that can help manage spending.

BudgetBoss87: Does this allow for setting a hard spending limit? Like stopping once I exceed $1,000 in a month? Iโ€™ve looked at the docs but didnโ€™t see that feature mentioned.

CleverCoder33: Absolutely! All you need to do is set up your API behind APIM, and youโ€™re good to go!

Reply to this answer

Add your answer

Answers are reviewed before publication.

Related Discussions