Hey everyone! We're transitioning to a GitOps approach and need some advice on structuring our Kubernetes deployment repository. We currently use Terraform for our Kubernetes setup and are considering having a dedicated repo not just for Terraform, but for other services too. Inside this repo, we have subdirectories for each environment: development, staging, and production. However, we're facing a challenge with duplicating a lot of code across environments, since I typically test in dev and then copy the same setup to staging. Although we've tried to mitigate some of this by creating modules for each service, I feel like there might be a better solution out there.
We also have a separate repository for our Helm charts. Right now, our application deployments are managed from a single repository, which includes all the app-related manifest files. This setup makes it tough for developers to see what's being deployed when. Ideally, we want to place the app-related manifests within the app repository itself. But again, we encounter the issue of duplicating a lot of Helm chart code across different apps.
So, how do you recommend structuring the Terraform, Helm, and app CI/CD setups to minimize duplication while keeping the relevant code in their respective repositories?
3 Answers
I'm interested in seeing what everyone else thinks too! Personally, I keep Terraform in its own repo. For Helm and CI/CD configs, I place those within each project under a resources folder by service. Since we’re a small team without dedicated DevOps, this setup works well for us.
If your projects are spread across multiple repos, I suggest looking into publishing your Helm charts to an artifact repository—this way, you can manage them all in one place.
Check out the Flux documentation for some great options on repository structure and find a setup that fits your team or organization. It might help steer you in the right direction!
There's really no one-size-fits-all solution here, but I can share some patterns that have worked for me:
- For your Terraform setup, keep a single infrastructure repo, but create reusable modules. Instead of duplicating files across environments, use an 'envs/' folder where each environment has just a minimal main.tf that references those modules with different variable settings.
- Regarding Helm, avoid duplicating entire charts. You can either use upstream charts and just customize values.yaml in each app repo or create a common 'platform' chart for shared components, while letting individual apps override the values as needed.
- Finally, it's a good idea to keep manifests or Helm values with the app repo itself. This way, developers have ownership over their deployments, while your GitOps tool like Argo or Flux manages the syncing process. In short: your infrastructure repo contains Terraform modules and cluster setups, a platform repo for shared Helm charts, and the app repo has the app code alongside its Kubernetes manifests.
Absolutely! Publishing Helm charts to an artifact repo makes sharing and reuse so much easier. And I like how you’re keeping Terraform separate—putting app-specific resources in the app repo under 'resources/' helps avoid the complexity of a huge monorepo while giving developers control over their own deployments. Out of curiosity, how do you manage shared settings like logging or monitoring across services?