I'm currently using the Kong Ingress Controller on our AKS clusters, set up with 3 replicas and some configurations for pod anti-affinity and topology spreading. We also have a Pod Disruption Budget in place to ensure at least one instance is always available. With a minimum of 15 nodes, scaling up to 150-200 in production, I'm wondering if it makes sense to implement Horizontal Pod Autoscaler (HPA) instead of sticking with static replicas. I've enabled HPA for application workloads, but not for platform components like Kong, Prometheus, or ExternalDNS. Is it considered good practice to enable HPA for these resources? I'm personally hesitant due to the potential added complexity, but I'm curious if others have tried this in similar situations.
1 Answer
For ingress controllers, it's often best to run them as a daemon set. This way, you ensure a balanced spread of load without creating hotspots. However, you’ll need centralized logging to track requests easily.
Is this approach mainly for bare-metal clusters? Wouldn't having a pod on every node cause a lot of overhead?