Why Isn't Loki Using the Correct IAM Role?
Asked by
TechExplorer92
Hey everyone! I'm working with the lgtm-distributed Helm Chart and I'm having some trouble with my configuration. Specifically, it seems like Loki isn't using the expected service account to access S3. I've set it up to point to the correct IAM role, but the logs indicate it's trying to perform actions with the EKS node's worker account instead. In the logs, I see an `AccessDenied` error when it tries to flush chunks to S3, which suggests it's not using the service account I specified. I have verified that the service account is set up correctly and seems to be in use by the Loki pod. Has anyone encountered this issue before? Any advice would be greatly appreciated! Thanks!
Answers
It sounds like you might be running into a permissions issue with your IAM roles. Try spawning an AWS CLI pod with the same service account and attempt to put a file into the S3 bucket directly. This can help isolate whether the issue lies in the IAM role settings. Switching to pod identity might simplify things as well! Good luck!
Reply to this answer
Add your answer
Answers are reviewed before publication.
DevNinja01: I tested it out using the commands you suggested to run the debug pod and it seems like the IAM role is working fine from there. I can access S3 just like I’m supposed to. So, could it be possible that Loki is misconfigured and not utilizing the service account correctly?