I'm curious about neural compression algorithms. I'm not too concerned about whether they're lossless (though that would be preferred), but I'm looking for something that can identify patterns in files and effectively regenerate them when needed. I've seen this done with images using diffusion models, and I'm hoping to find something similar that minimizes randomness in the output. If you have any research papers, code examples, or basic videos on this topic, I'd love to see them!
2 Answers
One interesting approach is to start by analyzing the binary data and removing all the zeros, which usually don't contribute much. This can potentially cut the file size in half. Then, you can count the number of 1s in the file for even better compression! Of course, decompression depends on interpreting that data correctly, so keep that in mind.
You should definitely check out Pied Piper's lossless "middle out" compression platform—it's the best one I've tried! It really nails the neural compression concept you're looking for.
I totally thought of Silicon Valley when I read that, haha! Glad to see others appreciate it.
That was a clever reference!