We're planning to move from DynamoDB to RDS because several workloads require joins and more flexible pagination. However, our application heavily depends on DynamoDB Streams, particularly the ability to receive both the old and new images for every update.
I'm considering an outbox pattern that updates the RDS table and publishes an event to SNS, which then fans out to consumer Lambdas. The concern is that we have many consumers interested in different attributes across different tables, so maintaining events manually could become tedious.
Another option is change data capture with Debezium, streaming row-level changes into Kinesis. I'm concerned about operating separate Debezium infrastructure for our development, staging, QA, and production environments, since the added cost could be significant.
What are the most practical, production-ready ways to capture RDS changes—including before and after values—and deliver them asynchronously to multiple consumers?
4 Answers
The outbox pattern is more work up front, but it’s usually the safest choice when the events are part of the application’s business contract. Write the row and its corresponding outbox record in one transaction, then have a relay publish the event with retries and idempotency. This gives you control over exactly which attributes are exposed and avoids losing an event when the database update succeeds but a separate notification call fails.
AWS Database Migration Service with Kinesis is worth considering. It can read database changes and publish them without requiring application code to emit an event on every write. You can use a small replication instance for lower-volume non-production environments and scale the production setup independently. Check the supported before-image and change-record options for your specific RDS engine, since the exact payload depends on the source database and configuration.
For Aurora PostgreSQL with modest write volume, an Aurora-to-Lambda integration could be a relatively inexpensive option, especially in development and test environments. It’s convenient for simple workflows, but it may not be the best fit for high throughput or many independent consumers. In those cases, publish a durable change event to a stream or queue first, then let consumers process it independently.
If you’re using PostgreSQL, logical decoding may let you capture WAL changes directly without running Debezium. A CDC connector can consume those changes and publish them to Kinesis or another event system. This avoids modifying every write path, but you still need to operate the connector, manage replication slots, monitor lag, and decide how to represent deletes and before-images.
The main reason we’re considering RDS is that the existing workload needs joins and more flexible pagination, which weren’t accounted for when DynamoDB was first chosen.