How Can I Prevent Concurrent Executions for My Lambda Function?
Asked by
CuriousDev42
I'm relatively new to AWS, working as an intern, and I'm developing an internal app for my company. I need to ensure that my Lambda function does not run concurrently since it's modifying a file in S3, which could lead to overwrites. I'm currently using SQS as an intermediary between the trigger and the Lambda function. I'm considering setting reserved concurrency to 1 to limit this, but I'm open to suggestions for better approaches. Any advice would be greatly appreciated!
Answers
Definitely set your Reserved Concurrency to 1. This setting defines both the max and min number of executions for your Lambda function, effectively controlling concurrency. It's pretty much the only way to guarantee that only one instance runs at a time.
Reply to this answer
Using an SQS FIFO queue could be your best bet. With a FIFO queue, if you put all your messages into a single message group, Lambda will ensure only one execution runs at a time, which should prevent those overwrites. Plus, FIFO queues handle deduplication, which might be useful in your case.
Reply to this answer
Quick related question: If I have an SQS trigger set to max concurrency of 2 and my Lambda's reserved concurrency is 1, can the SQS poller face issues due to this limit?
Reply to this answer
You could configure the trigger to allow a minimum of 2, while keeping the function's reserved concurrency at 1. This way, the trigger won’t keep trying to invoke more Lambdas. Also, consider using ifMatch in your S3 requests to prevent updates if the object has changed. However, I’d suggest reevaluating your architecture; there might be better ways to manage this.
Reply to this answer
Add your answer
Answers are reviewed before publication.
LambdaGuy87: Yes, in that scenario, the poller will try invoking two Lambdas, but it will get throttled because of your concurrency limit. It's not a disaster, just something to keep in mind.