I'm developing a Kubernetes controller in Go, and I'm facing a challenge with how to reload tokens when the Secrets change. Right now, the tokens are read from environment variables, but they don't automatically update when the Secrets are modified, resulting in using outdated values. I've considered a few approaches: 1) Mount the Secret as files and use inotify for changes, 2) Always read from the mounted files without caching values, or 3) Use a Secret reference to read and watch the Secrets via the Kubernetes API, keeping in mind that this option would require read permissions on the Secrets. How would you recommend solving this issue, and is there a better platform for discussing these technical challenges?
3 Answers
I think option 2 could work well. If you're concerned about performance, you could even cache the file in memory for a short while instead of reading from it constantly.
You can actually give read access to specific Secrets without needing to grant access to all of them. Watching the Secret object and reloading upon changes sounds like a solid plan to me!
Option 3 would be the most real-time approach. Remember, it takes time for Secrets to propagate to a mounted file in the pod. Plus, using the Kubernetes API makes it more aligned with how a controller should function.
That's a good point! If you really need real-time updates, option 3 sounds like the most native solution.