I'm dealing with two environments, test and prod, both set up using the same Terraform template, so they should be identical in terms of configuration. Both clusters are running Argo CD, and everything works perfectly in the test environment. However, in the prod environment, I'm hitting a 502 Bad Gateway error. It seems like the Ingress Nginx is trying to use the HTTPS port, even though my Ingress manifest specifies HTTP.
Both Argo CD instances have the insecure flag enabled and are served at a path. When I use port-forward directly to Argo CD, everything works fine in both environments. This leads me to believe that Nginx is the culprit of my issues, but I'm having a hard time pinpointing the problem.
Here's the Ingress configuration for HTTP that I'm using:
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: argo-cd
namespace: argocd
labels:
app.kubernetes.io/name: argo-cd
app.kubernetes.io/managed-by: manually-deployed
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /prod/argo-cd
pathType: Prefix
backend:
service:
name: argocd-server
port:
name: http
```
The only thing that differs between test and prod is the path. In the test environment, I get a successful 200 response, but in prod, I get a dreadful 502 error. The ingress seems to be trying to reach the Argo CD server at the HTTPS port instead of HTTP. What's going wrong here? I'd appreciate any guidance that helps me avoid total frustration!
3 Answers
It might be worth checking if there's a global ingress controller setting that's enforcing HSTS or something similar. Are you using the same ingress controller and versions across both environments? Any differences there could lead to varying behaviors. Also, check the logs of the ingress controller for insights into what's happening when your ingress resources are applied.
It looks like you've got the `nginx.ingress.kubernetes.io/force-ssl-redirect: "true"` annotation in your YAML. That could be part of the problem since it usually forces HTTPS redirection, which might not be what you want if you're trying to proxy to HTTP. Try removing or setting it to "false" and see if that helps, but I remember you said it didn't change anything before.
Maybe there's something else at play?
Check the service configuration because it’s essential that you’re pointing to the correct backend. The error suggests in prod you're targeting a service with an HTTPS suffix while in test, it's HTTP. A subtle configuration difference could be causing this.
They're both set up the same way; both clusters are running identical configurations. That's what makes this issue so perplexing.
I removed it, and it still didn't work. The setup is the same in both environments, so I'm confused why it works in test but not in prod.