I'm trying to handle around 15,000 HTTP requests in under 10 seconds, where each request does a quick calculation taking about 10 to 20 milliseconds. My current setup involves a web app that triggers about 40 HTTP requests to a Python function app. This initial function app looks at the request and determines that it needs to perform between 1,000 to 15,000 calculations, which are sent to a second function app. While smaller batches of requests finish in under 10 seconds, the full 15k requests take over 5 minutes to complete. I suspect there may be a limitation with Azure Functions that I need to address. Should I consider switching to an App Service or a VM instead? Previously, this worked smoothly on AWS Lambda. I've also noticed that when I do a test without the calculation step, the 15k requests execute in under 10 seconds, which makes me think the delays are happening during the calculation phase. What are my options here?
1 Answer
Have you considered scaling out your function apps? More endpoints could help you handle the load better. It’s also worth checking the concurrency settings because if too many requests are coming in, it might be causing a backlog and you could hit some limits. Overall, I think you're on the right track looking for ways to optimize your functions.
I tried increasing the instances but it didn’t seem to help much. Interested to hear what others have experienced!