I'm building a notification system for missed calls, alerts, and other notifications, but I'm hitting the limitations of DynamoDB with the 1MB response size. My users can accumulate thousands of notifications, and I need a way to mark all of them as read quickly. Currently, my query for unread notifications is returning too much data, making it expensive to update every single row in the database. I tried using a timestamp in Redis to streamline this process, but I'm still being charged for "Read" units in DynamoDB for notifications that have already been marked as read, which feels counterproductive. If you've created a notification feed before, I'd love to hear how you managed the "Mark All as Read" feature. Did you use a watermark timestamp or a clever method for batch updates? Any insights or experiences would be greatly appreciated!
5 Answers
You could also explore creating a secondary index that stores just the user ID along with a 'hasBeenRead' flag. This would allow you to make more efficient queries when marking notifications as read without having to dive into the full notification set each time.
If you’re okay with some data duplication, think about tracking unread message IDs separately for each user. When they hit 'Mark All as Read', you can easily delete that user-specific record. Keeping it in S3 could help manage scalability and cost, especially if notifications aren't needing to persist indefinitely.
I've faced something similar! For my app, I mark messages as read by setting a boolean on them, which is indexed by timestamp. This lets you create a 'master' record that represents the latest read time. When fetching notifications, you just compare timestamps to determine if they've been read or not. This keeps everything normalized and efficient!
It sounds like you're facing quite a challenge! One common approach is to ditch the read markers from individual notifications and instead keep a watermark of when the user last marked notifications as read directly in the user record. This way, when you fetch unread notifications, you simply look for any notifications added after that timestamp, which minimizes unnecessary reads.
That makes sense; I hadn't thought of tracking it in the user record like that!
Consider adding a version field to your user config. Each time notifications are marked as read, increment the version number. When fetching notifications, just check against the latest version; this simplifies queries immensely!
Good idea! That should definitely help with the performance issues.