I'm working with an ASP.NET Core app that utilizes appsettings.json for the default values, which are overridden with environment variables depending on the environment. Currently, our Terraform deployment sets over 30 environment variables at the app service level, but managing these as flat environment variables can be tricky since they lack the structure that YAML or JSON provides—making nested keys and arrays difficult to handle.
We want to avoid keeping the configuration for each environment in our source code repository. With Kubernetes, it's straightforward to use structured config maps and mount them as files, separating different configurations into their respective files. However, Azure App Service only allows overriding configuration via environment variables.
I've considered a few options:
1. Configure Terraform to read structured YAML/JSON from a config repository and transform it into a flat list of environment variables for the app service. This would definitely simplify maintaining and reviewing config changes, but it still results in dealing with a large flat list when looking at Terraform plans or the app service config directly.
2. Use Azure App Configuration Service to store JSON configurations. Although this might not significantly improve our situation, as we don't need the additional app configuration features it offers.
3. Mount appsettings.json from the config repository during the deployment pipeline to the app service.
I'm leaning toward option 1 for the short term and might consider option 3 for the long term, although it may require some testing and adjustments to our pipeline. What do you think?
4 Answers
I've had success with option 1! I use it for quite a few repeatable values in my Terraform code. Just remember, don’t store sensitive data directly; instead, you can reference Key Vault secret IDs in your configs, which work great with locals in Terraform.
Did you know that .NET supports hierarchical configuration in environment variables? You can use a colon (:) as a separator, but a double underscore (__) works on all platforms and will be converted to a colon automatically. That should help with your variable structure!
Why do you need such a large config that you want to keep it in a separate repo? That seems like quite the undertaking!
Great question! We interact with several external services, and each environment requires different settings like URLs and timeout configurations. Plus, there's also rate limiting for API calls that varies across environments.
We prefer not to bundle per environment configurations into the built artifacts since we try to follow a build-once-deploy-many approach. So we keep only the essential baseline config in the repo, overriding necessary values per environment, which seems to be increasing as the app evolves.
This is an interesting discussion. At my previous job, we had a config repository containing a large JSON file with static information, which was referenced from the Terraform repository. I believe that's similar to what you're proposing in option 1, right?
I appreciate the info! We do utilize that functionality. However, it still lacks structure and is more error-prone, especially as nested configurations get deeper. With environments having 3 or more levels, it can quickly become a mess!