We run several Spring Boot microservices on EKS, and every service currently uses memory utilization for horizontal pod autoscaling. I'm reconsidering that because the JVM may keep heap allocated after garbage collection, meaning memory usage may stay high even when traffic is low. Conversely, a busy service might still have enough allocated heap that memory-based scaling does not react quickly.
I'm considering choosing metrics based on workload type: CPU, request rate, or p95 latency for HTTP services, and queue depth for asynchronous workers, possibly using KEDA with SQS. For HTTP services, would it be better to begin with CPU and only introduce request-based custom metrics if CPU is a poor proxy? For queue consumers, is SQS backlog-based scaling the usual approach?
I'd also like advice on how to verify each service's actual workload empirically rather than relying only on architecture documentation. For example, can I confirm whether a service receives ingress traffic, whether it is genuinely driven by SQS, and whether CPU or memory correlates with work during load tests before assigning an autoscaling metric?
1 Answer
The general direction makes sense, but the metric should reflect the real bottleneck rather than just the service category. CPU is often a good first metric for HTTP services and is simpler and more reliable than immediately adding custom request-rate metrics. If request volume, latency, or concurrency clearly predicts saturation better than CPU, then a Prometheus-based metric can be worthwhile.
For queue consumers, scaling from backlog or estimated backlog-per-worker is a common approach, including KEDA with SQS. Just remember that adding workers will not fix a downstream bottleneck. If PostgreSQL or another dependency is already saturated or slow, scaling the Java pods may increase contention and make the system worse. Validate the dependency capacity and observe CPU, latency, throughput, queue age, and error rates together.
To verify the classification, inspect actual ingress and service-routing configuration, application access logs, SQS consumer metrics, and dependency telemetry. Then compare CPU, heap usage, request rate, latency, throughput, and queue depth during representative load tests. The metric that rises predictably before saturation—and produces a useful scaling response—is generally a better choice than memory alone.
Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures