Is It Possible to Achieve Content-Based Deduplication with EventBridge and SQS?
Asked by
CuriousCat42
Hey folks! I've got a setup where EventBridge sends events to my SQS FIFO queue. The issue is that the event body includes the event ID and timestamps, which messes up the content-based deduplication since the hash changes each time due to those fields. I'm wondering if there's a simple workaround for this. Could I use EventBridge input transformations to modify the body to include only the **detail** field, or maybe move the **id** and **time** fields outside the body? Here's what an example event body looks like:
'body': '{"version":"0","id":"c76d719f-bb24-4945-bda3-3f162a57c8ce","detail-type":"Mapping","source":"<source>","account":"<account_num>","time":"2025-05-16T19:32:13Z","region":"us-east-1","resources":[],"detail":{<event_detail>}}}'
Answers
You can provide your own deduplication string if needed. Just remember, if you don’t have control over the source sending events to the bus, that might limit your options.
Reply to this answer
Yes, you can definitely use input transformations. They allow you to customize what gets pushed to SQS. Are you using an EventBridge pipe for this?
CuriousCat42: We're not using a pipe at the moment—just the basic EventBridge bus to the event target. But it seems input transformations will do the trick, thanks!
Reply to this answer
Add your answer
Answers are reviewed before publication.
CuriousCat42: Good point! We don’t have control over the source, so that complicates things a bit.