
The Bandwagon – Claude Shannon (1956) [pdf] - twtw
http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1056774
======
Isamu
You can substitute current buzzwords (AI, ML) for Information Theory and it
reads as fresh as ever:

> INFORMATION theory has, in the last few years, become something of a
> scientific bandwagon.

> It will be all too easy for our somewhat artificial prosperity to collapse
> overnight when it is realized that the use of a few exciting words like
> information, entropy, redundancy, do not solve all our problems.

> What can be done to inject a note of moderation in this situation? In the
> first place, workers in other fields should realize that the basic results
> of the subject are aimed in a very specific direction, a direction that is
> not necessarily relevant to such fields as psychology, economics, and other
> social sciences. Indeed, the hard core of information theory is,
> essentially, a branch of mathematics, a strictly deductive system. A
> thorough understanding of the mathematical foundation and its communication
> application is surely a prerequisite to other applications.

~~~
jm_l
To the extent that we can do this simple word substitution, it should probably
make us less impressed, not more.

Basically only the most generic points that you can make about any new
technology (e.g. this will not solve every single problem, this is commonly
misunderstood and misapplied) will still make sense. We should focus our
attention on the specific claims made about the specific new technology being
critiqued, rather than the things someone can always say.

(This observation is actually an application of information theory - if the
points being made are 100% predictable then they contribute no negentropy :)

~~~
Isamu
Thing is, reasonably smart people fall for this same mistake over and over.

I think Claude hits on the issue here - it is the general unwillingness to go
back to the actual math and work through the narrowness of its implications.

Instead we start with a popular prose explanation and then argue our points
from our understanding of that.

We barely notice when we are using the ambiguity and wiggle room in the prose
explanation to argue something that is unsupported by the math.

------
mturmon
Relatedly, this editorial by Peter Elias, around the same time period -
another classic of the genre:

[https://oikosjournal.files.wordpress.com/2011/09/elias1958ir...](https://oikosjournal.files.wordpress.com/2011/09/elias1958ire-
editorial.pdf)

"Information Theory, Photosynthesis and Religion”

~~~
hprotagonist
oh, my god. That's a thing of beauty.

------
iamjohnsears
This is awesome. "Seldom do more than a few of nature’s secrets give way at
one time."

Resonates with the application of (to practitioners) blackbox ML techniques to
other domains. I saw an economist stuff an XGBoost model into a talk about
demand elasticity a couple years ago. I think there can be insight generated
by casting a wide net in applying new methods, but the really fundamental
breakthroughs seem to be generated by careful and informed application of the
scientific method.

------
memming
How one feels when what they created becomes an over-inflated bubble...

