
Ask HN: Compression with common binary database - tarikozket
Today I realized something. Humans compress their communication by knowing things mutually. For example, if you know a joke and if your friend knows the same joke too, you can remind him&#x2F;her that joke any second by just telling a word of it.<p>Today our compression algorithms compress stuff by only thinking that nobody knows the joke we are gonna tell and tries to choose shorter words instead of just outputting a word of the joke.<p>What if, we would analyze many compressed files and find the most common binary patterns and create a new compression algorithm which uses these patterns to tell the joke with only a word? And the person who wants to decompress the file would need to have the database as well.
======
PaulHoule
See
[https://en.wikipedia.org/wiki/Brotli](https://en.wikipedia.org/wiki/Brotli)

~~~
tarikozket
This is exactly what I have thought :) Thinking this as a software and making
the database bigger and cloud connected makes more sense.

------
throwaway_374
Great in concept until you run into the pigeonhole principle. Such compression
schemes are fundamentally flawed and you can spend years wasting time on them
until some corner edge case will throw you. Nonetheless, I'd encourage you to
satisfy yourself by trying to devise such an elaborate scheme... Pied Piper
may have a job opening for you.

