GPT-5.6 in Azure Foundry has been extremely slow for me, especially when using the Responses API. Time to first token can be around a minute, making the model nearly unusable, while output speed afterward seems normal. Prompt caching also appears not to work reliably, even though several programming features require this API. Has anyone figured out whether the problem is a caching bug, Azure capacity constraints, or something else? Is a proper fix or redeployment expected soon?
3 Answers
Some users are reporting that performance has improved recently, possibly because caching is now being applied more consistently. It may be worth testing again, but caching behavior still needs to be verified separately for implicit and explicit cache modes.
Prompt caching may be affected by an upstream API bug rather than by the model weights. However, Azure hosts and serves its own deployments, so the Azure and direct API implementations can behave differently. The fact that caching works through the direct endpoint but not consistently in Azure suggests the Azure Responses API integration still needs attention.
Exactly—the same request works with the direct API, including both implicit and explicit caching, so it’s difficult to treat this as only an upstream issue.
Be careful when evaluating costs and caching. Unexpectedly large charges can result from selecting provisioned throughput instead of a standard serverless deployment, so check the deployment type and billing configuration before assuming the model has no caching. Also monitor usage closely while the service is unstable.
Has anyone confirmed that both implicit and explicit caching are actually working in Azure, rather than just seeing faster responses from general capacity improvements?