Are people running FastAPI in production with a large user base or heavy request volume? I'm interested in real-world experience, including how you deploy it, keep async code from blocking, benchmark performance, handle background or CPU-heavy work, and scale the surrounding services and infrastructure.
5 Answers
FastAPI itself usually is not the limiting factor. Python services can run at very large scales when they are split across multiple processes and instances, with sensible timeouts, backpressure, caching, and properly sized database and connection pools. The real bottlenecks are often downstream services, monolithic startup time, or too much work in a single request.
At moderate traffic, FastAPI can be very inexpensive to run. The main performance improvements tend to come from optimizing the whole application: caching frequently used data, using a faster JSON serializer when profiling justifies it, offloading CPU-bound work to process pools, and replacing blocking libraries with async alternatives. If throughput requirements become extreme, Go or Rust may offer better efficiency, but FastAPI is capable of scaling far beyond a small prototype.
For production, use a process manager or deployment setup with multiple ASGI workers rather than relying on a single development-style server process. Add rate limiting, response compression where appropriate, application-level caching, and monitoring. Lightweight background tasks are fine for small jobs, but substantial work should go through a durable queue and separate worker.
FastAPI can handle substantial traffic, but getting there requires careful async design. In one setup, a worker handled close to 10,000 requests per second, although reaching that level took significant tuning. The main issue was making sure every I/O operation was non-blocking and that CPU-heavy work was moved to separate processes.
For benchmarking, measure the complete request path rather than just checking whether functions use async def. Track p50, p95, and p99 latency, event-loop lag, in-flight requests, thread-pool usage, connection-pool waits, and downstream service latency. Load-test with realistic payloads and dependency behavior.
It has worked well for services handling a few hundred requests per second. The important part is avoiding blocking calls inside async endpoints, since one accidental synchronous database, storage, or HTTP call can stall the event loop. Horizontal scaling with multiple worker processes is usually more useful than trying to make one process do everything.
A common example is the standard cloud-storage SDK, which may block even when it is called from an async endpoint. Use an async-compatible client or explicitly move the synchronous operation to a thread pool.
That makes sense. I was especially wondering whether the framework itself becomes the bottleneck, or whether the surrounding database, storage, and application code usually matter more.