I'm working on a system that requires polling an API every 2 minutes. Whenever the API indicates a "new event," I need to log it and immediately notify customers via email and text messages. Reliability is crucial since missing an event could cost the customer around $2000 or more. Currently, I'm considering using a Lambda function for polling, along with three additional Lambdas for sending emails, sending texts via Twilio, and logging to a database for later UI display. I also plan to use an SQS queue for managing notifications to users, as each event could require notifying anywhere from 10 to 50 users. It's worth noting that I want to batch the messages effectively to avoid overloading. However, I'm wondering if having four Lambdas might be excessive. I've also thought about using a small EC2 instance with multiple processes instead, which may be easier to test and develop. But I'm a bit concerned about the reliability of EC2 compared to Lambda. What do you all think?
3 Answers
Using Lambda sounds like a solid choice! You might want to consider using EventBridge for scheduling instead of regular triggers, as it could streamline your setup. SQS is great for triggering Lambdas, but EventBridge or SNS might save you some hassle by avoiding multiple publishes to SQS.
Four Lambdas might seem a bit much, especially if you can handle it within a single Lambda that's triggered by a timer. It really simplifies things.
Totally agree. Less overhead with fewer Lambdas means easier management!
If reliability is your top priority, maybe go with Fargate. It’s more robust than EC2 for polling tasks. But if you're budget-conscious, Lambda is your best bet for this use case since you could leverage their pay-per-use model without spinning up EC2 instances.
Yeah, EventBridge could also give you more options for event routing in the future!