We had a webhook endpoint that could jump to five times its normal traffic within a minute, overwhelming the origin and causing events to be dropped. We put a small edge function in front of it that immediately acknowledges the request, forwards the payload to the origin asynchronously, and stores the event in durable edge storage if forwarding fails. That setup now handles roughly 195 million requests per month for about $92, while the origin only receives successfully forwarded traffic. We have not seen any dropped events, including tests with payloads up to 10 MB. One tricky implementation detail cost us a day: after sending the response, the runtime does not let you safely read the request body. Small payloads appeared to work because they fit in the first chunk, but larger real-world payloads failed in production. Has anyone built a similar pattern on a major cloud provider, and how do you decide when edge buffering is enough versus when to put a proper queue behind the endpoint?
4 Answers
This sounds close to the point where the edge function should simply enqueue the payload and let pollers or workers consume it. A managed queue gives you durable delivery, backpressure, retries, dead-letter handling, and straightforward monitoring. Edge buffering can still make sense for absorbing sudden bursts or protecting the queue, but I would avoid making the edge layer responsible for too much delivery logic.
Be careful with the status code. A 200 usually implies the request was handled successfully, while 202 means it was accepted for later processing. If the upstream system treats anything other than 200 as a failure and retries, though, returning 202 may not be practical. Whatever code you choose, document the delivery semantics and make the downstream operation idempotent.
The request-body behavior is an important warning. Once the response is sent, the runtime may stop exposing the streaming body, so the complete payload needs to be captured or passed into the asynchronous operation before returning. For larger payloads, test chunking, memory limits, timeouts, retries, duplicate delivery, and payload size limits explicitly instead of relying on tiny test requests.
If the client needs to know whether processing actually succeeded, an immediate success response is the wrong contract. A more conventional design is to accept the request, write it durably to a queue, and return only after that write succeeds. Workers can then process the queue, while queue depth, retry counts, processing latency, and dead-letter messages provide much better operational visibility.
That is the direction I would take when the sender supports asynchronous acknowledgment. In this case, some senders retry whenever they do not receive a 200, so the immediate acknowledgment was chosen to prevent retry storms.
That is the tradeoff here. Some webhook providers interpret anything other than 200 as a failed delivery and immediately retry, so 200 is being used as the protocol-level acknowledgment rather than as a claim that downstream processing has finished.