How are you managing database credentials for Kubernetes apps at scale?

November 14, 2026 5 answers
Asked by Nerlo.62

We have shared credentials for schema migrations and per-app database users provisioned by CI, with their secrets stored in AWS Secrets Manager. It works, but we lack visibility into which service owns each user, and decommissioned services leave users and secrets behind.

We're on EKS with RDS, plus some Redshift and DocumentDB. I'm considering Vault or OpenBao and a separate repo for reviewed roles and grants, while keeping application migrations in their own repos. How much work is that to set up, how do you manage grants for the other databases, and is IAM authentication worth revisiting?

Answers

e72!wordmark_691

Separate the durable authorization policy from the credentials. A reviewed repo for migration roles, application roles, grants, and the configuration that issues credentials is a reasonable direction. Keep the schema changes with each application, but make ownership of the database roles explicit.

The missing piece is inventory and cleanup. Record which service and environment owns each role, secret, and database user. Then make decommissioning include revoking access and removing the corresponding users and secrets. Short-lived credentials can reduce the stale-password problem, but they don't automatically tell you whether a durable role is still needed.

Mavlo.8t:

Usage visibility needs its own audit trail too. Issuing a credential and knowing which service is still using it are different things.

Reply to this answer
Drelby.8k

Vault is worth a look if credential issuance and revocation are the main gaps, but budget time for operating it, not just installing it. You'll need to work through authentication from EKS, database configuration, policies, application delivery, and how credentials get renewed or replaced.

Vault Secrets Operator can sync secrets into Kubernetes Secrets. That's convenient for applications that already consume them, although it also means accepting Kubernetes Secrets as part of the storage path. Test the end-to-end flow with one service before committing to a fleet-wide rollout. Compare OpenBao against the same operational requirements rather than assuming the choice of product solves the lifecycle by itself.

Reply to this answer
8w~gearcase_413

Revisit IAM authentication for the RDS engines and configurations that support it. EKS Pod Identity can give a workload AWS permissions without distributing long-lived AWS keys, and the app can obtain a database authentication token.

That doesn't remove database roles or grants, and it may require application or connection-management changes. Check token handling and driver support with your developers first. It also isn't a blanket replacement for your Redshift and DocumentDB plans.

Nerlo.62:

Application changes could be a hurdle for us, so testing this with one willing team sounds useful.

Reply to this answer
_loosely!noted_6

For Redshift and DocumentDB, I'd start by listing the exact roles and grants you need, then check what your chosen tooling can actually manage before building a provider. A small purpose-built provisioning job, potentially a Lambda function, could be easier to maintain than a custom Terraform provider.

Whatever runs it should make ownership, review, and deletion clear. Otherwise you've just moved the existing CI library into a different runtime.

Reply to this answer
Tivby.4n

Don't overlook a simpler secrets delivery setup if dynamic credentials aren't a requirement for every service. An operator can sync credentials from a secrets manager into Kubernetes, and a UI can make stored credentials easier to find. That's useful, but it won't by itself create database users, reconcile grants, or clean up orphaned accounts.

Reply to this answer

Add your answer

Answers are reviewed before publication.

Related Discussions