I'm transitioning our authentication to EntraID, moving away from an old setup that uses Dex connected to our on-premise Active Directory via LDAP. We've successfully switched our interactive user logins to Pinniped, which is working great. However, for automated workflows, I hit a snag because it relies on a password grant type that our IDP team has ruled out for security reasons. I've been exploring options with Dex and token exchange, but I'm faced with validation issues, as EntraID doesn't seem to support client credential workflows. While I managed to get Pinniped Supervisor to function with Gitlab as an OIDC provider, this only addresses part of our needs since it's limited to Gitlab CI automation. Given our setup with about 400 clusters and hundreds of users with namespace access, I'm curious if anyone in the enterprise space has similar experiences or solutions.
5 Answers
To summarize my approach: We generate kubeconfigs in our CI/CD pipelines and store secrets in HCP Vault. As we create our clusters with Terraform on EKS, we utilize the secret outputs to make them available in Vault. For deployments, our central CI/CD component retrieves these secrets and builds a local kubeconfig for the pipeline before executing helm installs. We're also transitioning to Argo for better organization-wide management with larger clusters, which should alleviate some issues for specialized teams.
Why not just go with simple ServiceAccount tokens? It could simplify things if the issue is about handling multiple clusters. 🤔
Service account tokens can be a hassle. They either have static lifetimes, which clashes with our compliance needs (we need to rotate credentials every 6 months), or they’re too short-lived. Plus, managing them across multiple namespaces and clusters can be cumbersome.
I've had success with legacy Keycloak for similar workflows. It pairs well with Kubernetes, although it was primarily designed for user interactions and the automation part feels a bit like an add-on. I’m thinking of trying Zitadel next, yet I still want Custom Resource Definitions (CRDs).
Currently, we don't support CRDs, but you can leverage Terraform to achieve something similar.
Consider using oauth2 proxy configured with your Entra app that has the authentication and redirect URL enabled. Add app roles to the app and create a second one that assigns these roles under API permissions. This way, when obtaining an access token, you can use the first app's client ID as the scope. The oauth2 proxy can validate the token and manage authorization based on your roles, which also works well for user access.
You could use Entra ID's client credentials grant along with app registrations and service principals. This setup is quite secure and tailored for automated workflows, making it a solid choice for your needs.
I've managed to set up EntraID's client credentials successfully, but I'm struggling to integrate it with Kubernetes. Any tips?
I see what you mean, but we’re looking for an approach our application teams with limited access can use. My team has admin-level solutions, but I wouldn't want to expose those to other teams.