I'm deciding how to run AI agents in production on Azure and would appreciate advice from people who have deployed them at scale. The main options I'm considering are Azure AI Foundry Agent Service, with managed threads, tool calling, identity, and state, or a self-hosted agent running in AKS, Azure Container Apps, or App Service. Which approach has worked better for you, and what trade-offs around flexibility, vendor lock-in, operations, cost, and scaling should I consider?
4 Answers
There’s also a middle ground: self-host the agent in Azure Container Apps and use its sandbox or Dynamic Sessions capabilities for isolated code execution. That gives you more control over your orchestration and deployment model while avoiding the need to build agent execution sandboxing from scratch.
Self-hosting on Azure Container Apps is usually the better fit when you need a custom orchestration loop, LangGraph or Semantic Kernel, unusual memory and retrieval behavior, or detailed control over latency and cost. It also keeps your application architecture more portable across model providers. The trade-off is additional work for deployment, identity, state, observability, evaluations, scaling, and security. For a small internal tool with a couple of tools, the managed service is probably the faster and simpler option; for a product that will evolve for years, self-hosting may be worth the operational overhead.
Don’t treat Foundry’s container-based agents as automatically more locked in than a self-hosted deployment. They rely on much of the same container-oriented Azure technology, so the practical difference may be smaller than expected. The bigger decision is whether you want Foundry to manage more of the agent-specific identity, state, evaluation, and monitoring features, or whether you need complete control over those components.
Foundry Agent Service is a strong choice when you have a fairly standard setup—RAG, tool calling, and conversational threads—and you’re already committed to Azure. It provides features such as Entra ID integration, Content Safety, monitoring, evaluations, agent identity, and state management without requiring you to build and operate all of that yourself. The newer container- and code-based agent options also provide more flexibility than the earliest versions, while still handling much of the serverless and agent-specific infrastructure for you.
That’s helpful. My main concern was losing portability, so the container-based approach may be a reasonable compromise if the underlying deployment model stays familiar.