I'm diving into using SQLite for a FastAPI project and I've noticed that it usually involves creating a new connection to the database for each change or query. I'm curious why it's considered a bad practice to use a single connection for all database requests. Even though a cursor needs to be created for each connection, is there something inherently wrong with reusing the same connection with multiple cursors? Also, does using multithreading in FastAPI affect the decision about using a global connection?
3 Answers
The reason behind needing multiple connections boils down to concurrency. If only one caller is executing queries, a single connection might suffice. However, for systems that handle multiple simultaneous requests, one global connection won't work as it would serialize operations, blocking other requests. Creating a new connection for each request is inefficient, but implementing a database connection pool is a smart solution to manage multiple requests efficiently.
When you query a database, you're essentially dealing with two components: the database context and the underlying connection. Each time you establish a new context, it might seem like you're opening a new connection, but many database providers utilize connection pooling behind the scenes. If you're managing connections manually, it's crucial to implement pooling to avoid slow connections. If you’re using an ORM, it may handle pooling for you, which is the preferred way to manage connections.
Connections maintain state relevant to your requests. For simple queries, that state might not be critical, but for operations like updates or inserts, it's crucial. This is because multiple threads could be trying to modify the database simultaneously. Starting a transaction for updates ensures the database state remains consistent while you make changes. If there’s a conflict due to another connection altering the same data, your commit might fail, which is why maintaining separate connections is necessary.

If connections are context-specific, why do libraries like Python's sqlite3 strongly suggest using cursors for actions?