I'm using a music-player library to receive album artwork, but the image arrives in multiple chunks and I don't know its total size beforehand. What's the recommended way to store the complete image? Should I grow a dynamically allocated buffer, keep the chunks separately, or write each chunk directly to disk? I'd like to avoid excessive copying and unnecessary allocation overhead, especially when loading many album covers for a scrolling interface.
4 Answers
A common approach is to start with a reasonable capacity, track how many bytes are currently used, and call realloc() when the next chunk won’t fit. Grow the capacity geometrically—usually by doubling it—rather than adding a fixed amount each time. That keeps the total copying to roughly O(n) instead of copying the entire image on every small append. Use a temporary pointer for realloc() so you don’t lose the original allocation if it fails.
You don’t necessarily need one contiguous buffer. Keep each received chunk in a small list or array and only combine them at the end if the image decoder requires contiguous memory. This avoids repeatedly copying data when the incoming chunks are very small. If you only need to save the image or pass it through another stream, you can write each chunk directly to a file and reuse the same receive buffer.
For a scrolling album-art view, loading everything permanently into memory may be the bigger issue. Load artwork for items near the visible area, decode it to the display size, and discard or cache older images with a fixed memory limit. If the library lets you process chunks incrementally, stream them into a file or decoder instead of assembling the entire compressed image in memory.
If the protocol or library can provide the total image size first, allocate the exact amount once and fill it as chunks arrive. Otherwise, a reusable buffer that doubles in size is usually simple and fast. Allocating a separate block for every chunk is valid, but it adds bookkeeping and may require one final copy if a contiguous image is needed.
That makes sense. I was worried that realloc() would always copy everything, but letting it extend the allocation in place when possible—and doubling the capacity when it cannot—sounds much better than growing by a small fixed amount.