How should I compare Python API frameworks beyond JSON benchmarks?

November 16, 2026 5 answers
Asked by b.635~orbitcabbage

I'm looking at FastAPI, Litestar, Django REST Framework, Django Ninja and Bolt for APIs with database queries and authentication. What should I look for in a benchmark besides raw JSON throughput?

Answers

Velnar-2d

Look at each endpoint separately rather than picking a winner from the fastest JSON response. In one test with a one CPU container and PostgreSQL, Bolt led the 1 KB JSON and simple database read tests, while Litestar with Granian led the paginated articles test. Those rankings don't tell you which will be fastest for your particular queries.

The test used 100 connections for 10 seconds per endpoint and reported the median of five container starts. That's useful context, but I'd also want to see latency and longer runs before using the numbers for capacity planning.

Reply to this answer
d.718!ferrulemarker

Check what's actually being compared. The frameworks used different authentication libraries, and Litestar and Bolt used native msgspec serialization. Even with identical response bytes, the work needed to produce them can differ.

It would be interesting to rerun the JSON tests with orjson where supported, then compare the database and auth endpoints too. Faster serialization might matter a lot less when queries or other I/O dominate. Likewise, compare ORM queries with raw SQL and manually built dictionaries before assuming how much the ORM costs.

_e93+thunderpocket.6:

For the auth tests, I'd also check that each implementation does equivalent user lookup and validation work. Different libraries can make a throughput comparison tricky.

Reply to this answer
ClatterTheory

Server choice matters too. The reported DRF results differed substantially between Gunicorn and Granian, so calling one number the framework's speed is a bit misleading. Keep the server configuration visible when comparing results, and test the combination you would actually deploy.

Reply to this answer
_m63!portwinder.5

Throughput isn't the whole framework decision. DRF can be convenient for CRUD and business APIs, while a lighter framework may be a better fit for a smaller service. Check the integrations and development tradeoffs you need, then benchmark the realistic endpoints that matter to you.

Reply to this answer
4x!baudcaddy_682

There's room to extend the comparison. Robyn would be interesting to test against the same database and auth workloads, not just JSON. For Bolt, I'd also check its Django integration and deployment setup before treating a fast result as enough reason to switch.

Reply to this answer

Add your answer

Answers are reviewed before publication.

Related Discussions