I'm facing an odd issue with my Application Load Balancer (ALB) that's set up to authenticate with Okta using OIDC. It seems to be losing the OIDC client secret that I had configured in the Listener rule, leading to a *561 Authentication error*. Whenever this happens, the only solution I've found is to extract the client secret from the Okta app and re-enter it in the ALB Listener's config under "*Authenticate using OIDC*". Right now, I don't have any further information since I previously didn't have access logging enabled on the ALB, but that's fixed now. I have two other ALBs set up the same way, and one has been working perfectly fine for over six months without issues. Any ideas on how to troubleshoot this?
3 Answers
Could it be that there's something unknown modifying the listener rule? It seems odd that it would just lose the secret like that.
In our organization, we avoid keeping credentials directly in configurations. I recommend using Secrets Manager for OIDC credentials instead. You can set it up like this: ClientSecret: '{{resolve:secretsmanager:YourSecretNameOrARN:SecretString:clientSecretKey}}'. Also, for detecting any changes to your listener rule, check AWS Config and CloudTrail to see if someone is inadvertently modifying it.
I get that! This setup is temporary for now though. Plus, once you save the credentials in the listener rule via the console, they’re not accessible anymore.
You might want to check the CloudTrail logs for any potential modifications to your listener rule. I've set up numerous OIDC ALBs and haven’t encountered this issue myself. Maybe there was an incomplete update to the listener rule that's causing this?
I appreciate the suggestion! I looked through my CloudTrail logs and didn’t find anything unusual – it looks like only I’ve made updates to the ALB.
That’s definitely a possibility. However, considering that I have the other two ALBs running under the same account and VPC without issues, I feel like it’s unlikely. I'm the only one who manages these.