

Journey into Information Theory - gshrikant
https://www.khanacademy.org/computing/computer-science/informationtheory

======
beagle3
Shannon's original paper, titled "A Mathematical Theory of Communication"[0]
which basically founded Information Theory as its own coherent and well
defined field is also very readable. Some of the concepts were well known
before he published it, some were original -- but the coherent formulation,
notation and concentration in that paper is what established the field now
known as "Information Theory".

[0]
[http://worrydream.com/refs/Shannon%20-%20A%20Mathematical%20...](http://worrydream.com/refs/Shannon%20-%20A%20Mathematical%20Theory%20of%20Communication.pdf)

~~~
gshrikant
Thanks for pointing me to
[http://worrydream.com/refs](http://worrydream.com/refs). It is a treasure
trove of wonderful writings!

------
gshrikant
I found Hartley's paper [1] (also discussed in the video sequence) on
quantifying information surprisingly accessible and a good read in general.

[1]
[http://www.dotrose.com/etext/90_Miscellaneous/transmission_o...](http://www.dotrose.com/etext/90_Miscellaneous/transmission_of_information_1928b.pdf)

------
anirudt
Elements of Information Theory, by Thomas Cover is also a very good read for
fundamentals.

~~~
stared
By J. Thomas AND Thomas Cover. Even worse combination than Stephan-Boltzmann
law or Albert-Barabasi networks.

The book itself is great, IMHO the best intro to information theory and
entropy.

------
murbard2
"History of the alphebet" [sic]

Not mentioned is the curious fact that there seems to have been only one
independent discovery of the alphabet. All alphabets are descended from the
same root, proto-sinaitic or have been made up by people who knew about
alphabets (Korean). Irish runes and Khmer script thus share a common ancestor.

------
mrcactu5
I think it's insane that Khan Academy should attempt to cover information
theory giving the Coursera course has a 1% passing rate.

I am two videos in, biting my lip to not make technical comments that don't
matter at this level. They do a great job of simplifying a very complex
subject.

------
yen223
Information Theory is one of my most favourite courses that I took at uni. The
idea that you can take something as abstract as "information", and quantify it
in a useful manner was eye-opening.

~~~
bandrami
Same here. Particularly the idea that "information" and "entropy" are the same
quantity. That was an eye-opener.

