I've taken over an EKS cluster that uses an ELB created by Istio for its LoadBalancer resource. Now, my company's security team wants me to set up WAF on the load balancer, which means I need to switch to an ALB. I managed to set it up using the AWS Load Balancer Controller and configured it to send traffic to the Istio ingress gateway service, which I changed to NodePort. However, I'm facing a frustrating problem where external requests are always returning a 502 error. I've reached out to AWS Support, and they believe the ALB is functioning correctly. Despite my efforts to increase the verbosity of Istio logging, I'm not seeing any clues about what's causing the problem. What should I try next to diagnose this issue? Thanks for any help!
2 Answers
I've actually got a similar setup running successfully in two of my clusters. Here’s my flow: ALB -> Ingress -> Istio Gateway (ClusterIP mode) -> Virtual Service -> Service. I remember switching from NodePort to ClusterIP due to issues like the one you're facing. It might be worth trying ClusterIP since that worked for me!
You should check all the health checks, especially on the Gateway service. A 502 error usually points to a routing issue between the Gateway and the upstream services. If you haven’t already, consider installing the Kiali dashboard in your cluster; it really helps visualize the network flow, which could point you to where the problem lies.
I appreciate the suggestion! The ALB health checks are passing, and I’m using the same manifest that worked before, just flipped to NodePort. Do I need to manually set up target groups for the ALB? That seems like a hassle since I’d have to adjust ALB rules each time I deploy.
Thanks for the insight! I was just trying NodePort since many examples suggest that approach, but I'll definitely give ClusterIP a go.