How to Stream CSV Exports in Safari and Firefox Efficiently?
Asked by
CoffeeLover92
I'm a junior developer tackling a project that requires me to implement client-side CSV exports for data stored in MUI DataGridPro. The catch is it has to stream directly to the user's file system without storing anything in-memory in the browser. I've managed to get it working fine in Chrome with the native File System API, but I'm running into a lot of trouble with Safari and Firefox.
I'm currently using the Blob API to do the export and handling it as follows:
1. User clicks 'Export'.
2. A WritableStream is created in-memory.
3. The header is written and immediately converted to a Blob, while the raw data is garbage collected to save memory.
4. Each page of rows is fetched, encoded, converted to a Blob, and the raw data is also garbage collected.
5. Finally, all Blob references are combined and a download is triggered.
However, with customers having over a million rows in some cases, I'm worried this isn't the most efficient solution. I would really appreciate any insights or better workarounds!
Answers
You might want to rethink the rationale behind not wanting to write to disk. It’s nearly impossible to avoid disk interactions, considering data is often stored in memory pages and browser cache anyway. Trying to keep everything off-heap in JavaScript will likely lead to more headaches than it’s worth. Just my two cents!
Reply to this answer
Honestly, this sounds like a tough task for a junior dev. It could even be a bit comical depending on the data involved. No offense to you, but your managers might not have picked the best project for someone new. Hang in there!
Reply to this answer
Have you looked at the specific error messages? Some browsers only provide file system access over secure connections, which could be playing a role. Also, consider whether using APIs that are compatible across browsers might help. It’s worth investigating if a server-side solution could simplify this, processing the job and allowing users to download once it’s done. Just a thought!
Reply to this answer
It sounds like you might be dealing with an XY problem here. Can you clarify what the exact requirements are? Sometimes diving deeper can uncover alternate solutions that align with your goals.
Reply to this answer
Add your answer
Answers are reviewed before publication.
TechSkeptic88: Good point! The main worry is keeping the JavaScript heap stable to prevent performance issues with large datasets. I thought the Blob solution would mitigate that, but I’m concerned if it’s the right approach.