I'm working on a project where I have two programs: one acts as a sender (S) and the other as a receiver (R). My setup involves R sending a starting signal to S, which then transmits image data. Before sending the actual image data, S sends a struct containing a verification code, the image width, height, and total byte size to R so that R can allocate the necessary memory. This process repeats for each frame, with a 20ms delay to prevent overwhelming R. However, I've been facing issues where the struct sent from S is sometimes out of sync, causing a binary misalignment that leads to errors in receiving the image. I'm looking for advice on how to design this process better to ensure synchronization.
5 Answers
Since this is a live video feed, sending all the data at once isn’t feasible. You might want to focus more on ensuring synchronization rather than just sending the video data. A strategy similar to TCP's sliding window could help keep S and R in sync without overwhelming either side. Just a thought!
Make sure you're sending the entire structure at once instead of in parts (like headers and frames separately). It could help with maintaining sync. Also, consider UDP; if possible, it’s typically easier to work with for video streams since it doesn’t wait for lost packets. Just ensure to implement your protocol to manage that loss!
Have you thought about using an established protocol like RTP for your transmissions? It’s been battle-tested for live video, and it might save you some headaches in syncing. Plus, it’s worth checking if your ESP32 Cam supports it. That way, you wouldn't have to reinvent the wheel!
Think about this: if you're able to buffer all image data in memory on S, you could send the header along with the correct size followed by the frame data, keeping things in sync that way. And what about frame size variability? Are you sending compressed data, or is there a max frame size you can ensure? R could preallocate larger buffers based on that. But definitely consider UDP for the video stream if you can.
You might also want to explore using Motion JPEG. It’s quite straightforward and should play nicely with your resource-constrained setup. Just something to look into!
I'm in a similar boat, and I've been told RTP is great for handling live video stream needs. Definitely something to consider!