I'm making an API request with Axios, and the server returns nearly 850 MB of JSON. The request appears to succeed, but the response data is an empty string. The same approach worked with roughly 300 MB, so I suspect I'm hitting a runtime, string-size, memory, or response-processing limit. For now, I need to parse the large JSON document rather than redesign the API, although I understand that streaming, pagination, or downloading a file would normally be better options. What is the practical way to handle a JSON response this large in JavaScript?
3 Answers
The API design is probably the main issue. A response that large should generally be paginated, exposed as a downloadable file with resume support, or returned in a streamable format such as newline-delimited JSON. Sending everything as one traditional JSON response makes failures, retries, parsing, and memory usage much harder to manage.
In Node.js, very large strings can hit V8 limits. JSON.parse and JSON.stringify also require substantial temporary memory, so an 850 MB document may need several gigabytes while it is being received and parsed. A streaming JSON parser can process the response incrementally, or you can split the payload into smaller pieces before parsing it. Check the actual error and memory usage rather than relying only on the empty response value.
First determine whether the failure happens during the download or during JSON parsing. Add error logging, inspect the HTTP status and headers, and check whether Axios is buffering the entire response. If you control the server, pagination or an export endpoint is the safest fix. If you do not, use a Node.js response stream and a streaming parser instead of waiting for Axios to build one enormous string.
That makes sense. I’ll verify whether the response is being received completely before changing the parsing approach.