How does Foundry AI Gateway compare with setting up APIM yourself?

November 20, 2026 5 answers
Asked by Nuvtaro-3p

I'm looking at Azure Bicep deployment, Foundry and MCP connections, policies, and traffic logging. What should I check before choosing Foundry AI Gateway over a self-managed APIM setup?

Answers

M.284!drelvani

Start with a small deployment and trace a request through the whole setup. Connect a Foundry endpoint, configure the policies you need, and check that the traffic appears in the logs. Do the same for any MCP tools you plan to expose.

Pay particular attention to the Foundry association. Test authentication and routing for each model endpoint rather than assuming the association handles every case the same way. Bicep can cover some of the setup, but check which configuration changes you can actually manage through your existing deployment workflow.

Reply to this answer
5k+parcelwicket_836

The estimated spend per caller policy is worth a close look if you need to track usage by consumer. Enable it early and compare its estimates in the traffic logs with actual Foundry billing. An estimate is useful for visibility, but it isn't necessarily the invoice.

_d46!plorvani.8:

Also check whether that policy is available in your APIM setup before treating it as a reason to switch. It may be a distinguishing AI Gateway feature, but the comparison isn't settled here.

Reply to this answer
Mivreno+7j

I'd compare the operational workflow, not just the backend. AI Gateway uses APIM underneath, but that doesn't guarantee the same feature set or the same GitOps experience. If your team manages APIM through APIOps, check how much of the gateway configuration can be kept in code versus done through the portal.

There is a possibility that the products will diverge as more AI-specific features are added, but that's a prediction, not a reason to assume the features you need are there today.

Reply to this answer
W.417!Melnari

Make the cost case before adding another gateway layer. A shared AI Gateway can serve multiple Foundry accounts, which may make more sense than deploying one for every project. Either way, the gateway only earns its keep if you actually configure policies and retain the logs you need.

Reply to this answer
_n54~drivquelo.3

If Responses API support or file uploads matter to your application, include both in the proof of concept. The supplied discussion indicates that a PDF upload to the Responses endpoint worked, but that's a narrow test, so check your own file types and request flow.

Reply to this answer

Add your answer

Answers are reviewed before publication.

Related Discussions