We run several Spring Boot microservices on EKS, and every service currently scales with memory-based HPA. I'm starting to question that setup because the JVM can retain heap after garbage collection and doesn't necessarily return it to the operating system quickly. That means memory usage may stay high even when traffic is low, while a busy service might still have enough allocated heap and fail to trigger scaling based on memory alone.
I'm considering choosing the scaling signal based on the service's workload:
- HTTP services: CPU initially, or possibly requests per second and p95 latency for a more direct demand signal
- Asynchronous workers: queue depth or queue age, potentially using KEDA with SQS
Does this approach make sense? For HTTP services, should I begin with CPU and only introduce RPS or latency-based scaling if CPU does not correlate well with demand? For queue consumers, is KEDA with SQS backlog the usual approach?
I'd also like to verify each service's actual runtime behavior instead of relying only on architecture documentation. What are good ways to confirm whether a service really receives HTTP traffic, consumes from SQS, or has CPU usage that correlates with workload? Are ingress and service configuration, queue metrics, application telemetry, and controlled load tests enough to classify the services before selecting an HPA metric?
One concern is downstream capacity: scaling out application pods may not help if the database or another dependency is already the bottleneck. How should that factor into the choice of scaling metric and scale limits?
1 Answer
The overall direction is reasonable, but don’t treat the scaling signal in isolation. If PostgreSQL or another downstream dependency is saturated, adding more Java pods can increase contention without improving throughput. CPU is still a useful starting metric for many HTTP services, especially when it tracks request work reasonably well, but set sensible maximum replicas and monitor database connections, query latency, error rates, and saturation alongside it. For queue workers, backlog is generally a better demand signal than memory, though queue age can be more informative than raw message count when messages take different amounts of time to process.
Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures