The timeout limit for Lambda event functions has gone up to 30 minutes. Is that enough to make longer batch jobs a good fit, or are there still reasons to split them up or use containers?
Lambda event functions can run for 30 minutes now. What changes in practice?
Answers
It gives event-driven work more breathing room, especially processing larger files that used to hit the timeout. I would treat 30 minutes as headroom rather than automatically setting every function to the maximum.
Check the cost of a long invocation at the configured memory size. Also plan for a failure near the end: retries can repeat work, so make the handler idempotent and consider a dead-letter queue where appropriate.
Reply to this answer
If the function reads from SQS, revisit the queue visibility timeout before changing the function timeout. AWS recommends at least six times the function timeout, which would be three hours for a 30-minute function.
That has a practical downside. A message associated with a hung invocation could wait hours before becoming available for another attempt. An alarm on message age would be useful.
Reply to this answer
For a batch job that can fail halfway through, containers on Fargate may still be the better choice. A Lambda retry can rerun the work and incur the cost again. The longer limit doesn't solve that by itself.
Reply to this answer
The timeout is only part of the changelog. It also mentions EventBridge webhook event sources and STS GetScopedCredentials. The latter may be useful for narrowing credentials to what a particular request needs, such as a tenant-specific DynamoDB key or an S3 prefix.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Veldravi-7s:
The extra time can still be handy when the alternative is splitting a job up just to fit the old limit.