Why does my LZ77 compressor produce larger text files?

0
0
Asked By MellowPine42 On

I've been learning the basics of LZ77, LZ78, and LZW and implemented a simple LZ77 compressor. Decompression reproduces the original text exactly, but the compressed output is often larger than the input. For example, a text file around 55–62 KB became roughly 85 KB. Is this expected, or does it suggest a problem with my implementation or output format?

4 Answers

Answered By SilverKite28 On

Make sure the output is being written as a binary format and that you are measuring the actual compressed data rather than an intermediate buffer or text rendering of it. It can also help to include a stored, uncompressed mode: if compression does not reduce the payload after headers and metadata are counted, write the original bytes instead.

Answered By QuietMarble19 On

Lossless compression cannot make every possible input smaller. If it did, you could repeatedly compress the result until every file became zero bits, which is impossible because there are more possible inputs than shorter outputs. Random or already-compressed data has little predictable structure, so it may stay the same size or get larger. Ordinary text usually compresses well, but the file needs to be large enough for the savings to outweigh the format overhead.

Answered By BinaryCedar5 On

Exact decompression only proves that your process is reversible; it does not prove that the representation is efficient. For a text file that is tens of kilobytes and still grows substantially, check whether you are finding the longest useful matches and whether your sliding window is large enough. Also avoid writing every position and length as oversized fixed-width integers—using 32-bit fields for each token can easily waste more space than the match saves.

Answered By CleverOtter7 On

This can be completely normal, especially for small files. A compressed format needs headers and metadata, and each encoded match usually has to store things like a position and length. If those fields take more space than the repeated text saves, the result grows. Try a much larger, highly repetitive file; you should see the compression become more effective. Your later test shrinking about 5 MB to 850 KB is a good sign.

MellowPine42 -

That makes sense. The smaller file growing was caused by the overhead, while the much larger repetitive file compressed well.

Related Questions

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.