Hey everyone! I'm working on a backend app that primarily interacts with the OpenAI API. I've traditionally used API Gateway with Lambda because it's cost-effective and handles small projects well. However, I've hit a snag because the OpenAI API can be slow, leading to exceeding the API Gateway integration timeout of 29 seconds. I'm considering a few alternatives, but I'm not sure what's best:
1. I could create a separate Lambda function to manage background tasks, but that would change my app's architecture quite a bit.
2. Running it on a cheap EC2 instance seems like an option, but I'd prefer not to incur costs for a side project that doesn't get much traffic.
3. There's also ECS/Fargate, which I'm unfamiliar with, and I'm concerned about potential cold starts compared to Lambda.
Any advice would be greatly appreciated!
4 Answers
If you have control over the client-side, here’s a cheap approach: 1. Consider using a function URL. 2. If there's a database in play, implement polling on the client: submit a request, get a GUID, and check the database for status. 3. Alternatively, think about using WebSockets for a real-time update. Services like Pusher can simplify that for small-scale projects.
Actually, AWS has increased the Lambda integration timeout for these types of cases. Go to the service quotas and adjust the REST API integration timeout option. This might be the simplest fix without disrupting your architecture too much.
You might want to consider returning an immediate response from Lambda or API Gateway to acknowledge the request. Then, you can relay the OpenAI API response through another channel, like WebSockets or GraphQL, for real-time updates. This setup would allow you to stream results back to the client and avoid timeout issues.
I'd suggest using a job ID system with a POST and GET process. Since Lambda is stateless, you could use DynamoDB for job tracking. If performance is crucial, consider using ElastiCache, though it can be pricier than DynamoDB for this purpose.
Thanks for the tip! I actually did this, but I had to switch from HTTP API to REST API. It worked out nicely!