Hey everyone! I'm currently working on setting up a CI/CD pipeline and I'm faced with some architectural choices around VPC deployment patterns that I could really use senior engineer insights on. Here's what I've got so far: I'm using a monorepo for my infrastructure (using CDK) and applications (Lambda + EC2) with multiple environments: localdev, staging, and production. I'm utilizing CodePipeline along with CodeBuild for my deployments and building custom Docker images for the build environments.
I'm torn between two main approaches for deploying my VPC and RDS instances:
**Approach A:** Keep the Infrastructure Stack separate from the applications by deploying the VPC and RDS stacks independently, and reference this infrastructure in app deployments by exporting and importing necessary values.
**Approach B:** Integrate everything into a single pipeline where both the infrastructure and applications are deployed together, eliminating the need for complex exports/imports by using direct object references.
I have a few specific questions:
1. Should core infrastructure like the VPC and RDS be deployed separately or together with the application pipeline? I'm encountering a circular dependency issue where the RDS deployment requires access to the VPC it creates.
2. What's the best way to handle stack dependencies? Should I stick to CloudFormation exports vs direct CDK references?
3. For pipeline architecture, should I package applications independently or combine it with infrastructure deployment?
4. How do you manage development and production environments within the same pipeline while ensuring proper isolation?
Given that I'm currently using direct object references to avoid export/import problems, I'm starting to worry about the tight coupling this might create, especially with how apps depend on existing infrastructure.
Any patterns shared from others who have scaled these setups successfully would be greatly appreciated! If you had a chance to start from scratch, what would you do differently? Thanks!
7 Answers
It's crucial to set up your VPC with all necessary configs before devs touch anything. I also make a point never to bundle my RDS/Aurora clusters with app stacks to avoid complications during major updates. Keeping database configurations separate can prevent downtime during maintenance periods, especially with DNS routing adjustments.
I've also settled on a multi-stack deployment pattern. I establish the foundational resources like S3 buckets and permissions first. My network infrastructure is built out separately which allows tighter coupling for app stacks later. I prefer using the Parameter Store to pass data between stacks instead of exports due to upgrades often creating issues. For pipelines, I encourage a trunk-based deployment approach where changes are pushed directly to a shared dev environment before propagating to production.
Totally agree with the advice against deploying foundational elements like VPC with your app infrastructure. One wrong change can lead to disaster. Keeping concerns separate helps prevent potential conflicts in your deployments, and utilizing parameterization to pass values between stacks is key for successful management.
How does that align with the goals of IaC? Aren't we aiming for a streamlined deployment process where values can be managed efficiently?
In terms of the circular dependency issue, if you're referring to necessary migrations, consider running your migrations through a bastion. This allows you to keep your database and VPC deployments aligned effectively. In relation to your other questions, if your application and its dependencies have a clear relationship, deploying them together in a single stack could simplify the process greatly.
I suggest keeping your VPC setup separate from your application deployments. In my experience, having the VPC and RDS defined distinctly avoids a lot of circular dependency headaches. We use context references in our CDK to pull in VPC IDs without issues. For stack dependencies, I’ve shifted to using SSM parameters over CloudFormation exports due to less complexity. Finally, combining the built stage for both infrastructure and app packaging works great for us, but we prioritize getting the data tier live before rolling out apps.
That makes sense! How do you manage deployments across different branches though? I'm aiming for simplicity with a single main branch model.
For your VPC strategy, consider keeping things simple—if your pipeline for RDS relies on VPC access, ensure these are either handled in the same stack or establish a clear dependency to manage it effectively. Direct references can work well if stacks are set up correctly, but monitor for any potential issues.
For environment isolation, using branches for different environments is a good strategy, as it ensures each rollout is version controlled and isolated without overlap.
Think about your pipeline strategy carefully. It's generally wise to deploy your VPC in harmony with your corporate network to ensure usability. As for your deployment architecture, try to keep your stacks as decoupled as possible—working towards a cleaner architecture makes everything much easier down the line. When it comes to CI/CD, I’d recommend sticking to Parameter Store and moving away from nested stacks to avoid that convoluted setup you can run into with exports.
How do you handle integrating your baseline deployment to ensure it fits into the VPC per environment?