
The Shannon Limit (2010) - jonbaer
https://news.mit.edu/2010/explained-shannon-0115
======
dang
An excellent discussion from 2012:
[https://news.ycombinator.com/item?id=4342991](https://news.ycombinator.com/item?id=4342991)

------
RaoulP
If I have understood this correctly - can I equate the underlying principle of
this "possibility of more efficient error correction" to spotting a typo in a
text? That as words grow to sentences, to paragraphs on specific topics and so
on, it becomes easier to guess from context which word should have been
written, when a letter is written incorrectly?

~~~
willis936
Human languages have a lot of error correction built into them. Lempel Ziv
gets pretty close to perfect compression for most types of data. If you put
the ASCII plaintext of almost any book into a zip file the compression ratio
would be >20.

Almost all data sent electronically is first compressed until it is at entropy
and then put into error correcting and line codes to handle channel
imperfections.

~~~
personjerry
I have no knowledge of this Lempel Ziv, but shouldn't we make this Lempel Ziv
our language of choice? If all the data is so compressed then surely it means
we would be able to consume it faster.

~~~
mturmon
The Lempel-Ziv source encoding (LZ for short) is able to asymptotically
approach maximum efficiency because it adapts to the source. It does this by
keeping a dictionary of past phrases, and referring back to that dictionary as
it encodes new stuff. (There are several ways to do this, all offering
guarantees, so there are several flavors of LZ.)

The key is that the dictionary changes per-source.

This adaptive encoding would not really work for direct human interpretation,
because you'd have to maintain that dictionary somehow in your head,
separately for each source.

You'll note that our selection of acronyms and terms of art ("ROC curve",
"tach", "JS") has something of this flavor, though -- and the lexicon is
adaptive within each universe of discourse.

~~~
antisemiotic
Even more extreme version would be abbreviations like "k8s" or "a11y". T4h s7s
I w2h p4e w1o a3e t3e w3d j2t f2k o1f a1d d1e.

~~~
mturmon
Perfect examples! Like i18n and all the rest. Y2r sentence illustrates the ...
limitations of that method.

------
Peter_Franusic
The Shannon limit is simply a number: ln(2) or in decibels approximately -1.59
dB.

~~~
rrss
That's a different, but related, Shannon limit than the channel capacity
discussed in the article.

