GPT 5.6 on Azure is generating around 5 tokens per second for me. Is there something I should check in my setup, or is this a capacity or model issue?
Why is GPT 5.6 so slow on Azure?
Answers
First, separate the wait for the first token from the speed of the output after it starts. Those are different delays. There have also been reports of long waits for first output on very short prompts, even with low reasoning effort.
Try the same short prompt several times with the same settings on 5.6 and another model. Record time to first output, total response time, and output token count. That should help show whether the problem is startup latency, generation speed, or both.
Reply to this answer
Check for capacity errors as well as slow responses. A reported no_capacity error mentioned high demand and suggested Provisioned Throughput for more reliable capacity. That points to load as a possible factor, though it doesn't establish why every slow request is slow.
Reply to this answer
Prompt caching may be a separate issue worth checking if you're sending long, repeated inputs through the Responses API. There are reports of caching not working there on Azure, alongside a report that it worked through the direct provider API. That difference doesn't tell us where the fault is, but it's a useful comparison for a support ticket.
Compare the reported cached token counts for otherwise equivalent requests. Caching affects repeated input processing, so don't assume it explains slow output generation too.
Reply to this answer
If you're using a coding client, test a plain API request too. One report mentioned having to disable multi-agent mode with this model, but there's no evidence that setting caused the slow generation. It would at least help narrow down whether the client setup is involved.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Related Discussions