I'm developing a screen recording video editor and I've hit a major snag with the export process. It currently takes forever because the process of setting video.currentTime to frameTime lags between 5 to 163ms for each frame. To give you context, I'm trying to export a 9-second 1080p video at 60fps, and 90% of the export time is wasted on seeking frames, which is dragging the total processing time to anywhere between 39 seconds to over a minute!
Here's what I think is happening: each seek requires the browser to find the nearest keyframe, decode all frames from that keyframe to the target frame, and then discard the intermediate ones, which I have to repeat 558 times.
I've tried a couple of optimizations, such as preloading the video and reducing the timeout for seeks, but nothing seems to help significantly. I'm looking for tips on: 1) faster ways to extract frames at specific timestamps, perhaps using the WebCodecs VideoDecoder, 2) pre-processing the video to make it more seek-friendly, 3) any WebAssembly solutions that could work outside traditional browser video APIs, and 4) any other tips that might make this process more efficient. I'm open to recommendations on APIs or libraries that could assist in this.
3 Answers
Just curious, why are you exporting at 60fps instead of 30? It could help with performance.
Have you checked out mediabunny.dev? It could be a good alternative to ffmpeg that might simplify things for video processing.
Sounds like you're really facing the frustration with browser-performance limits for video processing! A common workaround is to push the exporting task to a backend server since the client side can be really unpredictable among different devices and setups. A server with a solid CPU and enough RAM can handle the heavy lifting much better. Think about it—users will have a much smoother experience instead of dealing with varying speeds in the browser. Good luck!
If you prefer keeping it client-side, consider using WebAssembly for processing like with ffmpeg. Just be cautious, as I'm not sure if ffmpeg in WASM can leverage GPU for encoding.
I have both options available but thought including 60fps might give users more choices compared to the competition!