I'm a junior developer tackling a project that requires me to implement client-side CSV exports for data stored in MUI DataGridPro. The catch is it has to stream directly to the user's file system without storing anything in-memory in the browser. I've managed to get it working fine in Chrome with the native File System API, but I'm running into a lot of trouble with Safari and Firefox.
I'm currently using the Blob API to do the export and handling it as follows:
1. User clicks 'Export'.
2. A WritableStream is created in-memory.
3. The header is written and immediately converted to a Blob, while the raw data is garbage collected to save memory.
4. Each page of rows is fetched, encoded, converted to a Blob, and the raw data is also garbage collected.
5. Finally, all Blob references are combined and a download is triggered.
However, with customers having over a million rows in some cases, I'm worried this isn't the most efficient solution. I would really appreciate any insights or better workarounds!
3 Answers
Have you looked at the specific error messages? Some browsers only provide file system access over secure connections, which could be playing a role. Also, consider whether using APIs that are compatible across browsers might help. It’s worth investigating if a server-side solution could simplify this, processing the job and allowing users to download once it’s done. Just a thought!
Honestly, this sounds like a tough task for a junior dev. It could even be a bit comical depending on the data involved. No offense to you, but your managers might not have picked the best project for someone new. Hang in there!
You might want to rethink the rationale behind not wanting to write to disk. It’s nearly impossible to avoid disk interactions, considering data is often stored in memory pages and browser cache anyway. Trying to keep everything off-heap in JavaScript will likely lead to more headaches than it’s worth. Just my two cents!
Good point! The main worry is keeping the JavaScript heap stable to prevent performance issues with large datasets. I thought the Blob solution would mitigate that, but I’m concerned if it’s the right approach.