I'm working on an API proxy that will receive requests from a source system. It needs to query DynamoDB and then forward a request to a target API. During peak times, the source could generate over 100 requests per second, but the target API has a maximum limit of, say, 3 requests per second—any excess requests will just get dropped. There could also be long periods of inactivity, sometimes up to an hour without any requests.
The target API doesn't need to receive requests immediately, but ideally, they should be processed within a couple of minutes. I need help designing a system that can throttle outgoing requests to this preset limit while efficiently handling high volumes of incoming requests. I'm looking for reliable, cost-effective architecture in AWS, and would appreciate any tips on potential pitfalls to avoid. Thanks in advance!
1 Answer
One effective approach is to send initial requests from the source system to an SQS queue. You can then use a Lambda function to poll that queue and forward requests to the target API at the rate that's allowed.
Make sure to implement thorough error handling, and if this is an asynchronous call, allow the source system to retrieve any necessary responses later. Also, consider setting the lambda concurrency to one to keep things simple and controlled.
I was wondering about the costs involved with polling the queue. If polling once a second is pricey for empty queues. Long polling is affordable, right? I'm looking forward to testing the ideal frequency based on how busy it gets.
Totally agree! Don't forget to incorporate long polling and a batching mechanism to prevent delays in your Lambda without increasing costs unnecessarily.
Also, if you encounter any throttling errors, it's a good idea to send those requests back to the queue for reprocessing.