I'm a software developer with a day job, but I'm not very experienced in managing infrastructure. I'm working on a side project using AWS and Terraform, and while I've been gradually learning and putting together my Infrastructure as Code (IaC) repository with GitHub CI/CD, I've run into a few roadblocks. My setup involves three AWS accounts structured by directories in my Terraform repo, where I manage environments and separate resources. I have a bootstrap module for GitHub Actions, terraform state, and roles management. However, every time I need to update my bootstrap, I have to run Terraform locally, which just adds to the hassle, especially since I'm concerned about giving extensive IAM permissions to my GitHub Actions roles. Additionally, the way I'm doing deployments now, with manual steps for staging and production, isn't satisfying. I wonder if tag-based deployments might be a better fit. Lastly, I'm not a fan of my directory structure; it makes it hard to validate changes for subsequent environments. How can I make my Terraform setup more efficient and manageable for someone working solo?
5 Answers
Honestly, you should avoid running Terraform locally as soon as possible. Relying on CI/CD for everything—from planning to applying—is the way to go. Local use can create a whole bunch of issues like state-locking problems. It's best to keep the local CLI usage to a minimum, ideally only for emergencies like state repairs or debugging. Also, while you might dislike directory-based management now, in the long run, it helps to control resource configuration without messy duplication. You could set up a common module that your different environments can reference, making shared updates easier while still allowing for environment-specific changes. Just remember, don’t let anyone apply changes except on the main branch to avoid conflicts between feature branches!
How do you handle plans that fail during apply? I find that with APIs often not being well-documented, it leads to unexpected issues.
You might want to check out Terragrunt. It's a wrapper for Terraform that can manage multiple environments and reduce repetition in your configuration, which sounds like it could fit your needs pretty well.
For a GitOps approach without all the overhead, consider self-hosting Atlantis. It integrates well with Terraform and allows you to manage IAM permissions more effectively by tying the permissions used by Atlantis to a role with necessary access. Just make sure you secure it properly since exposing it to the public isn’t ideal.
Have you looked into Terraform Cloud? It offers a free edition which is great for managing your infrastructure without the hassle of running Terraform locally. Plus, its git-driven approach might align well with what you need for your CI/CD setup.
Have you seen TerraTeam? It could be exactly what you're looking for as it helps manage team collaboration on Terraform projects efficiently.
Do you think allowing Terraform to create IAM policies will simplify things? That could definitely streamline your process.