Hey everyone! I'm trying to wrap my head around how AWS Lambda handles scaling, specifically when it comes to provisioned and on-demand concurrency. The official AWS documentation outlines limits on request per second (RPS) and concurrency scaling behavior, but I find some of it a bit unclear. For instance, the docs say that if a function has a duration of less than 100ms, there's an account-wide RPS limit that's ten times the account's concurrency limit. So, for a standard concurrency of 1,000, that would translate to 10,000 RPS. But how exactly do provisioned concurrency and reserved concurrency fit into this? I'm particularly confused about how spillover thresholds are determined—do they adhere strictly to stated limits, or does request duration impact them as well? I'd love to hear your thoughts, experiences, or any clarifications you might have on these rules and how they play out in real-world scenarios. Thanks!
3 Answers
An experiment might be the best way to really understand this in practice. If you're under the free tier, testing different durations and seeing how they impact scaling can shed light on the real behavior. It might help you fine-tune your architecture as well!
In my experience, the duration does matter but mostly in relation to the RPS limit. If your function runs under 100ms, you could actually hit that RPS limit before reaching your concurrency cap. So with a 10ms function, yes, you could theoretically breach the TPS limit very quickly without really hitting the concurrency limits as such. I guess that's where the confusion comes from with the documentation.
Right? It's super confusing. The way they phrase it makes it sound like you’re limited to those numbers when in fact it's really about the duration of your function's execution. It’s like they want to keep it simple but ended up complicating things more, especially for rapid-fire functions.
Exactly! 100 RPS isn’t a hard limit if you have shorter execution times, but the docs don’t really clarify that. It's like they want to keep the math simple for beginners, but then you have all these edge cases that get ignored.
I see your point, but I still think it's misleading. The docs tend to imply a strict spillover at 100 TPS which makes sense only if you're operating under a certain request duration. Plus, those requests handled under provisioned concurrency don’t contribute to the 10,000 TPS cap, which can be a huge factor.