We run EKS with Argo CD, Helm, and GitLab. Our GitOps repository has separate directories and values files for dev, SIT, and UAT. Secrets already live in AWS Secrets Manager; this question is specifically about ordinary, non-secret environment variables.
Developers frequently change configuration in the dev values file while testing. When the change is ready for SIT and later UAT, someone manually copies the relevant values into each environment's file. This causes missed variables, formatting mistakes, and broken deployments.
The proposed solutions are either using a branch for each environment and merging dev into SIT and then UAT, or moving the values to AWS Parameter Store. I'm concerned that environment branches would create merge conflicts and accidentally promote values that must remain environment-specific, while Parameter Store may simply move the same manual copying into a different system with less Git history and visibility.
How have you structured configuration and promotion in real-world GitOps setups? Should application configuration be promoted as a versioned artifact, handled with layered values, or managed with a tool such as Kargo? What would you choose if designing this from scratch?
4 Answers
Treat environment-specific configuration as intentional data, not something that should always be propagated from dev. Store stable defaults in a base values file and keep separate overrides for endpoints, identity resources, certificates, and sizing. The deployment should consume configuration injected by the target environment, whether that comes from Helm values, ConfigMaps, or a parameter service. Parameter Store can be a good runtime source, but it does not by itself solve promotion or provide the review and history you get from Git. For larger releases, versioned OCI artifacts or another immutable package can let each environment select exactly which configuration bundle it runs.
The cleanest pattern is to make configuration part of the application release rather than asking developers to copy values between environment folders. The source repository can define the desired configuration, and CI can render or publish the manifests and image reference to the GitOps repository. Promotion then moves the same versioned artifact from dev to SIT to UAT instead of manually recreating its contents. Environment-specific settings such as hostnames, role ARNs, and endpoints remain explicit overrides, while the application version and shared configuration stay identical across environments.
A promotion controller such as Kargo can automate the sequence through dev, SIT, and UAT. It can open or apply the promotion change, wait for Argo CD to synchronize, run health checks, and then make the next environment eligible. That gives you an auditable trail without manually copying values. The tradeoff is additional operational complexity and potentially several pull requests or approvals, so it is worth starting with a simple artifact-promotion workflow and adding automation where the current process is painful.
Kargo is useful for this workflow, but make sure its upgrade and failure-handling process is understood before making it production-critical. The important design choice is still promoting an immutable version, not copying environment files.
Avoid environment branches. They tend to become difficult to merge and obscure which changes are actually being promoted. Keep a common base, then use small, clearly named overlays for values that genuinely differ by environment. It also helps to separate application configuration from infrastructure configuration: feature flags and application defaults can be promoted, while DNS names, service endpoints, resource sizing, and identity settings remain owned by each environment. Add CI validation for YAML syntax, required keys, allowed values, and references to undefined variables so mistakes fail before deployment.
Layering only works if the boundaries are clear. Defaults can define the complete contract, with environment overlays changing only approved keys; infrastructure-specific values should be reviewed separately.
We use CI to publish rendered manifests to the GitOps repository. Promotion changes the artifact reference in the next environment, so the rendered diff is visible and there is no copy-and-paste drift.