I inherited a couple of in-cluster agent workloads that authenticate to internal APIs and a database using long-lived credentials stored in Kubernetes Secrets. There is also a static credential that many people on the platform team can read, and it rarely gets rotated because coordinating the change is difficult. Once these workloads reach the surrounding network, they can access much more than they actually need. I understand the case for short-lived, per-workload identities inside the cluster, but I'm looking for practical guidance on extending that model beyond the cluster. How are you limiting each workload to only the external services and operations it requires?
4 Answers
Do not expose service-account token validation as a substitute for authorization. A public JWKS endpoint can let a service validate a token, but the receiving system still needs to check issuer, audience, subject, expiry, and the workload’s assigned permissions. Use an identity provider and token exchange flow that the external service explicitly trusts, rather than accepting any valid cluster token.
Treat identity and network reachability as separate controls. Federated identity answers who the workload is, but it does not stop the workload from connecting to unrelated systems. Add egress policies, firewall rules, private endpoints, security groups, or service-mesh authorization so the workload can only reach the required hosts and ports. Then enforce authorization at the destination as well, so a compromised workload cannot turn broad network access into broad data access.
Use federated workload identity rather than distributing credentials. Bind an identity to the deployment or service account, then let the workload obtain short-lived tokens from your identity provider and exchange them for access to the API, database, or cloud service. Assign permissions to that identity with least privilege, including resource and operation-level restrictions where possible. The same pattern works for CI workloads as well as services running in the cluster.
A secrets manager such as Vault can issue dynamic, short-lived database credentials or generate secrets through an external-secrets integration. That is useful for systems that cannot consume federated identity directly, but it adds operational complexity. Prefer native IAM or workload identity whenever the target service supports it, and use a secrets manager as a compatibility layer rather than the default for every credential.
Generated secrets are still handy for legacy services or one-off keys, but make sure their lifetime, audience, and permissions are limited. They should not become another static credential copied into a cluster-wide Secret.
Most major cloud platforms provide an equivalent. For example, AWS has IAM Roles for Service Accounts, while other platforms connect Kubernetes service accounts to managed identities. The exact setup depends on the provider, but the important parts are short-lived tokens, an explicit workload binding, and narrowly scoped permissions.