What’s the best way to manage layered Kubernetes configuration and secrets?

0
0
Asked By MellowPine47 On

I have extensive experience but have never designed configuration management for a fleet of applications. We deploy .NET microservices to Kubernetes, each with its own configuration. Today, a Jenkins job generates an application.json file, which is deployed as a ConfigMap and referenced by the workload. I want to replace that approach with Kubernetes ConfigMaps and Secrets, using environment variables to override values from the application's built-in configuration.

I previously used Ansible with several configuration layers. YAML files were processed in order and merged into a final configuration, but this caused duplication and made it difficult to understand where a value was defined or overridden. I'm looking for a system with a UI that lets developers inspect the effective configuration and its hierarchy, while also enforcing access controls so only trusted users can view sensitive values.

I've considered OpenBao, Infisical, and Phase. OpenBao does not appear to show the fully resolved configuration clearly in its UI. Phase seems focused on secrets, while I need both ordinary configuration and secrets with different access permissions. Infisical's self-hosted licensing is also more expensive than expected, and I would prefer not to store sensitive information in a hosted cloud service.

I already use Argo CD and Helm, and I am not looking to use Ansible. What tools or architectures are people using to manage layered configuration, secrets, overrides, and visibility of the final resolved values?

4 Answers

Answered By ClusterHarbor22 On

For Kubernetes, a GitOps workflow with Argo CD or Flux plus Helm or Kustomize is generally a better fit than using Ansible as the deployment and configuration engine. Put environment-specific overlays in Git, render them in CI, and review the generated manifests and diffs. External Secrets Operator can connect the deployments to OpenBao or another backend, while Sealed Secrets is another option if encrypted secret values are acceptable in Git. There may not be a single UI that gives perfect variable provenance across all these layers, so the rendered output and diff often become the source of truth.

Answered By SilverCactus31 On

You can also generate the Kubernetes configuration with a configuration language instead of relying on a large collection of loosely ordered YAML files. For example, KCL can produce the application manifests, while tools such as vals resolve secret references from different backends. SOPS can be used for encrypted files when you need to keep secret material in version control. This approach can make inheritance and overrides more explicit, although you may still need to add tooling or conventions to display the origin of every value.

Answered By OrbitingMango8 On

This is probably easier if you separate the concerns. Keep ordinary, non-sensitive configuration in Git and make the rendered Kubernetes manifests, ConfigMaps, and deployment diffs the thing developers review. Store secret values in OpenBao or another secret manager, exposing only references to users who should not see the values. External Secrets Operator can then sync selected secrets into Kubernetes Secrets. This gives you access control without forcing one product to manage every kind of configuration.

MellowPine47 -

That handles storage and deployment, but I’m specifically looking for a way to visualize the hierarchy and show which layer supplied each final value.

Answered By QuietLantern6 On

A UI-focused secret manager may not solve the configuration hierarchy problem by itself. Secret systems are usually designed to control access and retrieve values, not to explain every override that produced a final application setting. I would keep the hierarchy in a declarative Git-based configuration layer, enforce permissions through repository and deployment access, and use a secret manager only for sensitive values. Then expose a generated, redacted configuration view for developers so they can inspect effective settings without seeing secrets.

Related Questions

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.