Hey everyone, I'm a junior developer looking for some guidance. I have a task to implement client-side CSV exports for data stored in MUI DataGridPro. The requirement is to stream the export directly to the user's file system, avoiding browser in-memory storage. This has been straightforward in Chrome and Chrome-based browsers using the native File System API.
However, I'm struggling with Safari and Firefox. I've found a workaround using the Blob API, but I'm worried it's not the most efficient solution, especially with customers that have tables containing over 1,000,000 rows, and an average of 170k rows per table.
When the user clicks 'Export', I create a WritableStream to handle the data. After writing the headers, I convert the data from each page into Blobs without keeping the data in the browser memory. The raw data is collected and garbage collected while holding only references to the Blobs. This prepares a final Blob that triggers the download via a link.
I'm seeking any suggestions for more efficient workarounds or approaches, as I've spent a lot of time figuring this out. Thanks in advance!
4 Answers
It sounds like you're caught in an XY problem. Can you clarify the specific requirements? Understanding the context might help find a better solution.
Honestly, this seems like a pretty wild task to give to a junior dev! But let’s focus on the solution. It’s useful to explore why this requirement exists in the first place.
I think it’s important to reconsider the goal here. Your data will eventually reside on disk in some form, whether it’s in the browser cache or swapped memory. The challenge might not be avoid disk writes but controlling how memory is allocated. Your Blob solution could work, just be cautious about how much data you're holding in memory at once.
When you say you can’t access the file system, what exactly do you mean? Have you checked the documentation? Remember, some features are only available with secured connections. It’s also worth considering if this could be handled server-side with a job/queue system that allows users to download once processing is complete.
That’s a fair point! My goal isn’t to avoid writes, just to keep the JS heap manageable so the tab doesn’t crash. I wanted blob handling to let the browser handle the caching without overwhelming the memory.