Does Kubernetes HPA provide a way to rebalance connection pools after scaling?
Asked by
MellowCedar42
When the Horizontal Pod Autoscaler adds or removes application or database pods, existing long-lived connections can remain attached to the original pods while newly created pods receive few or no connections. Does HPA expose a lifecycle event or hook that applications can use to invalidate or gradually rebalance their connection pools after scaling? If not, what is the usual Kubernetes approach: lifecycle hooks, watching endpoint changes, finite connection lifetimes, or a database-aware proxy?
Answers
A load balancer or service mesh may distribute new connections across the available endpoints, but existing persistent connections usually are not moved automatically. Keepalive settings can limit how long they remain pinned, but they do not replace application-level pool recycling or database-side pooling.
Reply to this answer
HPA only changes the replica count; it does not know anything about application connections or connection pools. The application has to handle this itself, usually by setting a finite connection lifetime or idle timeout so connections are recycled naturally. Watching endpoint changes and rotating the pool can also work, but it adds application complexity. A proxy can handle similar behavior when changing the application is difficult.
Reply to this answer
The bigger scaling risk is often the total pool size. If every pod maintains a pool of 10 connections, increasing from 3 replicas to 20 raises the potential database connections from 30 to 200. That can exhaust the database before stale connections become a problem. Use a finite max lifetime for normal recycling, and size each per-pod pool based on the database's actual connection budget.
Reply to this answer
A database-aware connection pooler is often the cleanest solution. The application pods connect to the pooler, and the pooler manages a smaller, shared set of database connections. Options include RDS Proxy for supported AWS databases, PgBouncer for PostgreSQL, and other PostgreSQL poolers. The pooler operates continuously rather than waiting for a scaling event, so it is less sensitive to how many application replicas are running.
Reply to this answer
Add your answer
Answers are reviewed before publication.
QuietOrbit31: A proxy is a reasonable option when the application cannot safely rebuild its own pool, though it is still important to configure connection and idle timeouts appropriately.