Rate limits still have to be enforced somewhere, so the first layer can become a bottleneck itself. How do you handle that, especially when requests may include large uploads? Can you reject them before receiving the whole body?
How do you stop the first network entry point from getting overloaded?
Answers
Yes, sometimes you can reject before the body arrives. If a request declares a Content-Length above your limit, a proxy can return 413 based on the headers. For a streamed or chunked body, the proxy can stop accepting it once the limit is exceeded, but it has already received data up to that point.
Clients using Expect: 100-continue can wait for permission before sending the body. Set header and body timeouts too, so slow connections cannot hold resources indefinitely. Check the behavior of your particular proxy and clients rather than assuming every client will wait.
Reply to this answer
Push filtering as far upstream as practical. A CDN or managed protection service can distribute traffic and absorb more of it than a single gateway, while caching and edge rate limits may keep some requests from reaching your origin at all.
That doesn't make capacity infinite or eliminate cost. Check what traffic the service can filter, what it charges for, and whether your origin can be reached directly and bypass the protection.
Reply to this answer
Give large uploads their own hostname or listener and their own limits. That way a surge of upload connections is less likely to starve small API requests. Connection caps can help as well, since open connections consume resources even when request rates don't look that high.
Reply to this answer
Some scaling is unavoidable if you want to serve more legitimate traffic. Reject unwanted requests early, then make sure the remaining traffic has enough capacity or gets a clear rejection when capacity runs out.
DNS routing can distribute clients across entry points, potentially using geography and health checks. Layer 4 load balancing is another option. Neither replaces admission control at the receiving endpoints, and DNS changes won't reach every client instantly.
Reply to this answer
Add your answer
Answers are reviewed before publication.
_c47!zordexa.3:
Per IP caps need some care because many legitimate clients can share an address. Separate capacity and limits help, but neither makes the upload entry point immune to overload.