Hey folks,
I need your thoughts on a design pattern I'm using for my AWS Lambda function. Here's the situation:
I have a Lambda function that's called directly by a client application. The task it performs takes about 15 seconds to complete. The catch is, the client has a hard timeout of 10 seconds, which means they lose patience and abandon the request before my function can finish its job.
To tackle this issue, I've come up with a self-invocation pattern:
The function operates in two modes based on a flag in the event payload:
1. **Trigger Mode:** When the function is first called, it sees that the flag is missing, so it re-invokes itself asynchronously and adds the flag to the new invocation. This lets it quickly respond with a `202 Accepted` status, fitting within the client's 10-second limit.
2. **Worker Mode:** The subsequent invocation, which now includes the flag, knows it's time to carry out the 15-second process.
I have a couple of questions about this:
1. Is this a smart approach for the problem?
2. Would it be better to separate this into two different Lambdas (like `TriggerLambda` and `WorkerLambda`)? Given that my task is just slightly longer than the timeout, creating another Lambda feels a bit like overkill. What do you all think?
Thanks for any insights!
3 Answers
Instead of self-invoking, how about implementing an architecture where the initial invocation creates a signed S3 URL for file handling? The client could receive a `202 Accepted` status along with the file URL, then the worker will process and store results there. Client-side polling can keep them updated while you handle Lambda processing in the background. Just be sure to set up a dead letter queue in case something goes wrong!
I feel like having a separate async call right off the bat would make things cleaner. Returning a `202` on an extra synchronous invocation doesn't seem beneficial. It's better to let clients trigger the function asynchronously instead of having a workaround with extra layers.
You might want to think about adding the client request to an SQS queue instead. This way, you wouldn't have to stress about timeouts even if processing takes a while. It allows you to decouple the client request from the processing, and you can have a worker Lambda process those requests without holding up the client.
That's a solid suggestion, but I'm currently limited by how the client operates. Trying to work within their constraints, unfortunately.