I'm developing a product on Azure that utilizes Azure OpenAI for legal and compliance document reviews. Since regulatory compliance requires me to stick with Azure OpenAI, I can't switch to using OpenAI directly. I'm a small startup with funding, so I can consider more serious contract options if necessary. My application deals with heavy workloads, and during customer reviews, token usage can spike significantly. To run smoothly in production, I need around 1.5 million to 2 million tokens per minute on the o4-mini model. However, under my current pay-as-you-go subscription, my deployments only reach about 200,000 tokens per minute. While the Microsoft documentation mentions cap limits of around 1 million tokens per minute for certain contracts, I can't seem to access that level in the portal. I've tried filling out the quota increase form multiple times, logged support tickets (but support said they can't help with quota approvals), and talked to Microsoft reps, who only offered apologies without concrete solutions. I'm looking for insights from anyone who has managed to reach high token-per-minute limits with Azure OpenAI. Specifically: 1) Are you running Azure OpenAI at 1 million+ tokens per minute? How did you achieve this? 2) Did you need to switch to an MCA, Enterprise, or another contract type? 3) Was there a specific role or team at Microsoft that provided assistance, like an account manager or a special Azure OpenAI team? 4) Did you have to commit to a certain spending amount or contract term to unlock higher limits? 5) Are the token-per-minute figures in the documentation realistic for small businesses, or are they only applicable to larger organizations? I'm not looking for marketing responses or links to public documents; I want real experiences from people who have scaled Azure OpenAI effectively.
6 Answers
If your tenant is in Europe, I can assist with increasing your quota. Just reach out to me directly, and I'll see what I can do!
Consider utilizing the global standard option—it allows for up to 30 million tokens per minute! It's a solid way to expand your processing power if you're looking for options.
I’ve found that my biggest limitation comes from the embedding model, where I’m capped at just 350,000 tokens per minute in eastUS. It’s frustrating that there's no batch API for embedding models like there is for inference models. That might be something to consider reviewing.
What region are you currently deployed in? If you're looking for a quick fix, I'd recommend getting a Priority Treatment Unit (PTU), if you haven’t already. There might be more capacity available after the Black Friday season ends as major retailers will no longer reserve theirs. Also, make sure you’re not on a standard deployment—this could limit your access.
The region you choose really makes a difference! We moved our deployment to Sweden specifically for OpenAI and managed to hit 10 million tokens per minute on versions 4.1 and 5.0. If congestion is an issue in your current area, a change might be necessary.
One way to boost your token capacity is to implement Azure API Management (APIM) and route requests to multiple backends in different regions or subscriptions. This horizontal scaling through regional distributions can help. I can share our Terraform code when I’m off mobile if you're interested!
I'd love to see that Terraform code when you’re ready!