If lossless compression cannot guarantee that every image becomes smaller—or even that different images will have similar compressed sizes—why can't an easily compressed video frame be followed by a much larger one that arrives late and creates a visible gap? For example, a frame with a large area of one color should compress much more efficiently than a detailed or noisy frame. How do streaming video systems maintain smooth playback despite these differences?
4 Answers
The main reason is buffering. A video player usually downloads several seconds of content before playback begins, then keeps downloading more while showing the already-buffered frames. If one frame or segment is larger than usual, the player can absorb that temporary slowdown from the buffer. Playback only stalls when the network delivers data more slowly than the video consumes it for long enough to empty the buffer. Live video uses a smaller buffer, so it has less protection against interruptions.
Video codecs generally do not encode every frame as an independent image. They use key frames and predicted or delta frames that describe changes from nearby frames, along with motion estimation. A mostly unchanged scene can therefore require very little data, while a scene full of new detail requires more. The decoder also has timing information telling it when each frame should be displayed, so frames are not simply shown whenever their packets happen to arrive.
The encoder usually targets a bitrate or maximum data rate rather than preserving the same quality for every frame. Simple frames may be encoded with excellent quality, while complicated frames—such as snow, glitter, smoke, or fast motion—get more aggressive, visibly lossy compression so the stream stays within its bandwidth budget. The changing quality is often visible as smearing or blockiness, but it normally does not change the playback timing.
The compression theorem applies to lossless compression. Streaming video is normally lossy, so the encoder can always reduce the quality further if a scene is difficult to compress. It cannot preserve every detail at a fixed bitrate, but it can keep the stream within a rate limit. If the available bandwidth is insufficient, the result is usually reduced quality, dropped frames, adaptive resolution, or eventually buffering—not necessarily a gap after every unusually large frame.
This is why visually complex scenes often look much worse than simple scenes at the same stream quality. The player is still receiving frames on schedule, but the encoder has thrown away more detail to fit the bitrate.