I'm looking for recommendations on resources that can help me design a CI/CD pipeline for Infrastructure as Code (IaC). I'm currently using CDK and CloudFormation but don't have a proper pipeline set up yet. Right now, whenever I push changes to Git, I create a separate branch, do some manual testing, and merge it with the master branch. After that, I go to Jenkins, specify parameters, and create an artifact that's then pushed through CodePipeline for different environments. We're single-tenant at the moment, so I'm also unsure how to manage this. I think it's important to keep my application and IaC processes separate, but I'd love to hear how others are doing it!
4 Answers
When we work on a feature, we branch from main and create a pull request. On that pull request, we run a "terraform plan" (or "cdk diff" like you), which helps us see what changes will happen upon merging. Once it's approved and merged, our main build pipeline kicks in: it builds artifacts, tests everything, and deploys the infrastructure. We usually deploy to production about 8-10 times a day, and we do quick smoke tests in pre-prod to ensure everything's good before going to prod!
That sounds like a dream process! How did you manage to scale up your deployment rhythm?
CDK Pipelines might be a solid fit for you! You can set up a stack for your application and another stack for the build pipeline, both triggered by Git commits/merges. This setup automates the whole workflow! Here's a handy link to the documentation: https://docs.aws.amazon.com/cdk/v2/guide/cdk_pipeline.html.
Just a heads up - CDK pipelines might not be the safest long-term choice. It's good for now, but keep an eye out since some services around it are being deprecated.
For our backend, I set up GitHub actions along with GitHub environments. It was a bit tricky because of stack sets and multi-regional deployments, but it’s been working out well!
In our multi-account AWS setup, we use a pull request workflow where every PR triggers an isolated acceptance pipeline in its own AWS account. It deploys the full stack and checks against the production environment. After approval, it rolls out to production. It’s quite effective for both Terraform and CDK!
How do you actually run a diff against an AWS environment?
Consider using Atlantis; it can streamline your process.