
Tench: When data is messy - tosh
https://aiweirdness.com/post/622648824384602112/when-data-is-messy
======
cs702
Imagine for a moment that a mad scientist is able to grow a human brain in the
lab.

Once grown, this brain knows nothing about objects in the physical world. It
has had no physical experiences. It has had no interactions with anything or
anyone else.

The mad scientist then wires this _know-nothing_ brain to a computer. The
computer feeds RGB images from the ImageNet ILSVRC2012 dataset, one image at a
time, again and again, to the brain's visual system. Upon feeding each image,
the mad scientist measures which of 1000 previously chosen neurons in the
brain's cortex fire, each neuron corresponding to exactly one label in the
dataset. When the right neuron fires, the scientist activates the brain's
pleasure centers; if the wrong neurons fire, the scientist activates the
brains's pain centers.

After seeing each image multiple times, one can imagine, the know-nothing
brain will learn to activate the correct neuron associated with each image
label well enough to get high accuracy on a validation dataset consisting of
previously unseen images.

But that poor brain will still know nothing about objects in the physical
world.

Substitute "human brain" for "convolutional neural network," and you obtain a
remarkably accurate description of how neural nets learn to recognize images.

~~~
jes5199
we often discuss AI in terms of “smartness” which is thought to be inherent to
the architecture. But maybe we should talk in terms of “experience”. When will
AI surpass human beings in quantity/quality of lived experiences?

~~~
cs702
That is a very succinct way of expressing the same idea. Many AI researchers
understand this. Note that, unlike human beings, the experience that AIs are
accumulating via interactions with the real world will be digitally saved and
gradually accumulated, and built upon, bit by bit, such that over time, AIs
will be able to draw on the accumulated experience of many prior AIs. We can
already see this happening today, for example, with the wide availability of
pretrained vision and language models that incorporate the experience of AIs
developed and trained by others in the past.

------
skybrian
I'm reminded of the Tank legend [1]. It's nice to have a clear example of the
really happening.

[1] [https://www.gwern.net/Tanks](https://www.gwern.net/Tanks)

------
gus_massa
What about a longer title like "Tench and image recognition algorithms"?

~~~
dang
Submitted title was just "Tench". That's a cool word so I consed it onto the
article title when changing it.

~~~
cs702
Completely irrelevant, but HN is one of the few places online in which you can
use the verb _to cons_ , and most people will understand what you mean, even
though this verb isn't part of any spoken language.

~~~
Icathian
I was actually just getting stuck there. Where does the term come from, I'm
curious?

~~~
cs702
[https://en.wikipedia.org/wiki/Cons](https://en.wikipedia.org/wiki/Cons)

------
dang
Url changed from
[https://twitter.com/janellecshane/status/1279151512879423488](https://twitter.com/janellecshane/status/1279151512879423488),
which points to this.

