Our team is considering migrating from DynamoDB to RDS because several workloads require relational joins and more flexible pagination than we originally anticipated. The migration is complicated because our application relies heavily on DynamoDB Streams, particularly the ability to receive both the old and new images for each update.
We initially considered using the outbox pattern: update the RDS table and write an event that would be published through SNS to various Lambda consumers. However, we have many consumers, each interested in different attributes across different tables, so maintaining all of those events manually could become cumbersome.
Another option is change data capture with Debezium, streaming row-level changes into Kinesis. That would avoid adding event-publishing logic to every write, but we have separate development, staging, QA, and production environments. Running a dedicated ECS service for Debezium in each environment may add more operational overhead and cost than we want.
What are the practical, production-ready approaches for getting DynamoDB-Streams-like change events from RDS, including old and new row values, without creating unnecessary complexity or cost?
4 Answers
The database engine matters. With PostgreSQL, logical decoding can expose changes directly from the write-ahead log, so Debezium is not the only possible CDC implementation. Aurora PostgreSQL can also integrate with Lambda for lower-volume workloads, although that approach may be less suitable when write volume is high or delivery guarantees are important.
AWS DMS feeding Kinesis is worth considering. It can capture row-level changes without requiring every application write path to publish its own event, and consumers can process the stream independently. You could use a small DMS instance for development and QA, then size production separately, rather than running a full Debezium deployment in every environment. Verify that the selected source engine and DMS settings provide the before-image and after-image data your consumers require.
The outbox pattern is more work up front, but it is often the cleanest choice when events represent business actions rather than raw database mutations. Write the business row and its event in one transaction, then publish the outbox asynchronously. For a large number of tables and consumers, CDC can reduce application changes, while an outbox gives you more intentional event contracts and avoids coupling consumers to the RDS schema.
Before choosing a replication mechanism, revisit whether every existing stream consumer must survive unchanged. RDS and DynamoDB have different data models and consistency, indexing, transaction, and scaling characteristics. Some consumers may be better served by database views, scheduled projections, or purpose-built events instead of exposing every row update as a generic stream.
For substantial traffic, I would avoid invoking downstream work synchronously from database operations. A durable stream or outbox gives you buffering, retries, replay, and better isolation between the database and consumers.