I'm working with an AWS CDK application managed through CodePipeline, which includes resources like DynamoDB tables, Lambda functions, S3 buckets, and SageMaker endpoints. In the past, we've had to delete and recreate our CloudFormation stack due to various deployment challenges, but we chose to retain our DynamoDB tables and S3 buckets to avoid losing production data.
The issue I'm facing now is that, when I attempt to redeploy, CloudFormation tries to recreate the existing DynamoDB tables, which results in errors since they are already present. To tackle this, I've been using the context flag `--context import-existing-tables=true` to switch to using `dynamodb.Table.from_table_name(...)` for existing tables. This works well but presents a challenge when I add a new table: the pipeline keeps passing that flag, leading to a reference to a non-existent table without any error messages, which is not ideal.
Currently, I've implemented a workaround where I manually create the new table regardless of the import flag, leaving existing tables to be imported. However, this feels fragile, as I have to remember to do this every time a new table is added. I'm curious if there's a more systematic way to follow the "create if not exists, import if exists" pattern in CDK deployments. Any suggestions?
4 Answers
One approach would be to keep the code as is and let CDK handle the creation of resources on its own. As for importing existing resources, you might consider not automating this process too much since it doesn’t occur very often. It’s better to just manage that manually when necessary.
Honestly, this is one reason I’m not a fan of CDK. It heavily relies on CloudFormation, which can be frustrating when issues arise. Consider renaming your stack resource, creating a new CloudFormation stack, and migrating data from existing resources. It could save you a lot of hassle if there isn’t too much to transfer, but just ensure your stack design allows for flexible naming.
These quirks with CDK and CloudFormation can really be a pain. It makes me appreciate Terraform more since it manages state better and can keep track of what's been deleted and what still exists. You might want to explore resources that explain how Terraform’s state management works, as it could provide some clarity on best practices for resource management.
You could consider segmenting your stacks based on how stable or volatile the resources are. Think about separating stateful and stateless resources. If you can't have multiple accounts, maybe use an environment suffix in your stack names to clearly differentiate between development and production. Also, the automatic context flag sounds like a bad idea; it might be leading to the issues you're experiencing. I’d recommend sticking with standard practices that don't require such hacks. By the way, if you're open to alternatives, I suggest exploring GitHub Actions with a self-hosted runner in your AWS environment; it might simplify things for you.
Comparing tools can be helpful, but sometimes sticking with what you know well is best. Your solution might need some tweaking but don't discount the robustness of CDK just yet!