Why is GPT-5.6 so slow on Azure?
Asked by
MellowPine47
GPT-5.6 was released a few days ago, and I tried running it through Azure because I have substantial platform credits available. The model initially produced output at roughly 5 tokens per second, with even simple prompts taking 24–50 seconds before showing the first response. Older models such as GPT-5.5 were much faster under similar settings. Is this a temporary capacity or rollout issue, or could something be wrong with my configuration? It eventually improved several days after release, but it still averages around 2–4 times slower than the official API and frequently returns response.failed errors. I have also noticed missing prompt caching, occasional no_capacity errors, and unreliable streaming behavior.
Answers
The situation improved after several days, but performance remains inconsistent: sometimes it is acceptable, while other requests are two to four times slower than the official API and fail intermittently. Older model versions can be a reasonable temporary fallback until capacity, caching, and streaming are stable.
Reply to this answer
There also seem to be product-level issues beyond raw inference speed. Responses API streaming has reportedly buffered an entire completion instead of emitting incremental chunks, and prompt caching was unavailable for some users. Those behaviors make the service feel even slower and can increase cost and waiting time.
Reply to this answer
The no_capacity errors strongly suggest demand exceeded the available deployment capacity. The service may recommend Provisioned Throughput during peak periods, though switching to that is not always practical. A cloud migration is also a major operational change, so alternatives are not an instant solution.
SilverKite73: Provisioned capacity could improve reliability, but it is a costly workaround for an API that should already handle normal enterprise traffic.
Reply to this answer
Azure has historically been slower for many hosted models than the official API and other providers. It was often considered the practical choice for certain OpenAI models because of enterprise availability and cloud integration, but that does not guarantee the best latency. The new model may simply have had insufficient regional capacity during launch.
RiverQuartz58: It may still be the most convenient option for these models, but convenience and enterprise support do not necessarily translate into better response speed.
Reply to this answer
The initial latency appears to have been a capacity and rollout problem rather than a local configuration issue. Several people saw simple prompts take half a minute or longer, followed by temporary improvements. Even after the partial fix, some requests still take around 9 seconds or effectively hang on longer prompts.
CopperLynx31: Nine seconds is still far too slow for a trivial request, and longer prompts can remain unusable.
Reply to this answer
Add your answer
Answers are reviewed before publication.
FrostyWillow29: The caching issue may be related to an active service incident rather than an intentional permanent limitation, since it appeared around the same time as the other failures.