I'm running multiple application pods that connect to PostgreSQL through SQLAlchemy. Each pod has its own connection pool, so scaling from three replicas to ten creates new pools while the existing pods keep their current connections open.
If PostgreSQL read replicas sit behind a Kubernetes Service or proxy, new connections may be routed to the newer replicas, but connections already held by the pools will remain attached to whichever database instance accepted them originally.
Who is normally responsible for redistributing those existing connections after a scale-out, and how is this usually handled in practice?
4 Answers
A common design is to put a database connection pooler such as PgBouncer or PgDog between the application pods and PostgreSQL. The application connects to the pooler, while the pooler manages a smaller set of database connections and can route new backend connections as database instances change. This separates application scaling from database connection management, although it still does not magically move an active session in the middle of a transaction.
Your assumption is right: scaling the application does not rebalance connections that are already open. SQLAlchemy’s pool keeps using those connections until they are returned, recycled, invalidated, or closed. A Service or load balancer generally only influences where new connections go. In practice, the application’s pool settings—such as connection lifetime, recycling, health checks, and pool size—control how quickly connections naturally move around.
If the pools are already configured with relatively short lifetimes—often minutes or tens of minutes—the issue may resolve on its own. New pods connect to the current topology, while old pods gradually replace their idle or expired connections. Immediate rebalancing only becomes necessary when the existing connections are very long-lived or a replica is being removed and needs to drain quickly.
Many teams handle this through connection lifecycle management rather than trying to force immediate redistribution. Set a maximum connection age or pool recycle interval, use pre-ping or equivalent health checks, and close or invalidate pools during planned topology changes. Over time, old connections disappear and newly created ones are distributed according to the current routing setup.
That does move the pooling layer, but it is usually easier to operate there. Pooler instances can be scaled independently, and existing client connections can remain connected while backend database connections are recycled or reassigned when they become idle.