
Jeff Hawkins: Thousand Brains Theory of Intelligence [video] - jonbaer
https://lexfridman.com/jeff-hawkins/
======
gnode
I think the cortex is attractive to AI research because its highly uniform
micro-anatomy suggests there is a simple algorithm behind general intelligence
to be found within. While intelligent behaviours appear to reside in the
cortex, the limbic system is heavily involved in learning and cognition
generally. Additionally, animals lacking a cortex, like birds, are still
capable of many intelligent behaviours typically regarded as cortical. I
wonder whether the cortex may be a kind of FPGA wired up by the limbic system,
not naturally containing the machinery for general learning, and a red herring
to the pursuit of general intelligence.

~~~
criddell
I know the brain isn't a computer, but why do we think examining the structure
is going to reveal an algorithm for intelligence? Isn't that like taking an
iPhone, cutting it into slices and studying it with the hopes of finding how
GarageBand works?

~~~
cowteriyaki
Funny you'd mention that, there was a study from Jonas and Kording [0] that
considered a microprocessor as an organism and applied analytic methods used
in neuroscience to see if they can figure out how it processes information.

[0] Jonas, E. and Kording, K.P., 2017. Could a neuroscientist understand a
microprocessor?. PLoS computational biology, 13(1), p.e1005268.

[https://doi.org/10.1371/journal.pcbi.1005268](https://doi.org/10.1371/journal.pcbi.1005268)

~~~
JPLeRouzic
I think it is named after a 2002 study "Could a biologist fix a radio" which
advocated that the current methods in biology were inadequate to understand a
living body. It was a plea to do more System Biology, but even if it is widely
known it does not changed much the way biology is done.

[https://www.math.arizona.edu/~jwatkins/canabiologistfixaradi...](https://www.math.arizona.edu/~jwatkins/canabiologistfixaradio.pdf)

------
TaupeRanger
Jeff and Subutai have been on this hobby horse for a very long time. They
don't have much to show for it, and many of their ideas are controversial even
within neuroscience (for example, some think that their venerated "cortical
columns" are structures without meaningful functions). It is very easy to bet
against them given the fact that nothing much has happened, and "theories"
about the brain are a dime-a-dozen. I'm glad that they have the gumption to go
all-in on these ideas, but I'm not expecting anything earth shattering anytime
soon.

~~~
nabla9
That's the blessing/curse with completely self funded research group. No
strict timelines, yearly review, benchmarks or external pressure. There is no
hurry to deliver, you can dream on and keep developing your pet idea
indefinitely. Nobody can tell you that you were wrong and you are now done.

~~~
p1esk
_No strict timelines, yearly review, benchmarks or external pressure._

Maybe not imposed on Jeff himself, at least not in this narrow sense, but the
people who work for him most likely have all that.

------
jcims
Just a side note, Lex has been killing it with this podcast. He’s been able to
land some remarkable guests for a relatively new show with a niche market. I
personally think he does a great job of mixing up the conversation with both
technical and philosophical angles.

Highly recommended.

~~~
nabla9
My favorite podcasts are Lex Friedman and Sean Carroll.

Anyone, please feel free to suggest other similar ones.

~~~
espeed
Eric Weinstein: Revolutionary Ideas in Science, Math, and Society
[https://www.youtube.com/watch?v=2wq9x2QcZN0](https://www.youtube.com/watch?v=2wq9x2QcZN0)

[https://lexfridman.com/eric-weinstein/](https://lexfridman.com/eric-
weinstein/)

Discussion:
[https://news.ycombinator.com/item?id=19458503](https://news.ycombinator.com/item?id=19458503)

~~~
nabla9
Eric Weinstein one was great.

What I meant is other podcast channels.

------
neural_thing
I have a metric ton of respect for Jeff and his team.

However, I do think that they did not go far enough in figuring out the
computational capabilities of individual pyramidal neurons, which IMO is the
foundation to figuring out how the neocortex works. I wrote a short book about
that once - if anyone's interested, check it out, it's free and meant to be
readable for AI/neuroscience enthusiasts:

[http://www.corticalcircuitry.com/](http://www.corticalcircuitry.com/)

~~~
teabee89
Thanks for sharing! Curious to learn more about this. Have you posted it in
the HTM discourse forum? I would be interested in what the community thinks of
it!

~~~
neural_thing
Yes, there is a discussion thread here:

[https://discourse.numenta.org/t/a-book-about-dendritic-
compu...](https://discourse.numenta.org/t/a-book-about-dendritic-computation-
and-cortical-layers/2810)

------
monkeydust
Been following hawkins and Numenta for a while. They have never really taken
off in commercial applications as far as I know despite some sound reasoning
behind their theories. Hopefully they will make an impactful breakthrough
soon.

~~~
KingFelix
same, I have been following before they changed the name to Numenta.
Interesting stuff, will listen to this podcast soon

------
KirinDave
Does someone have links to written variants of this material rather than a
podcast?

No offense if you like podcasts but I just don't have time for them even at 2x
speed. they're just too slow and disorganized.

~~~
patpending
This looks relevant: [https://numenta.com/blog/2019/01/16/the-thousand-brains-
theo...](https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-
intelligence/)

------
thrwaway3742
Is there a transcript somewhere? (It's 2 hours 9 minutes)

~~~
tim333
There's a youtube generated one if you go to
[https://www.youtube.com/watch?v=-EVqrDlAqYo](https://www.youtube.com/watch?v=-EVqrDlAqYo)
and click the three dots under the vid.

I don't know if there's a way to get those things into an easy to read format?
Clicking select all and pasting to the text editor sort of works but could be
better.

------
bhouston
Jeff has been at this for a really long time now. If it is this hard to make
progress, it is likely basic research that just requires a hard slog and also
may not turn out in the end as all basic research is risky. My default bet
then is against him because most basic research fails, with a small chance of
success.

~~~
tolstoy77
Got introduced to Jeff after watching him keynote a conference about a decade
ago. He's a great speaker. Also thought they had some interesting tech when it
came to computer vision at the time. Sadly it never took off and then ConvNets
blew whatever Numenta was doing out of the water. Surprised even with someone
like Jeff that Numenta hasn't had any commercial success.

------
melling
Learning a lot about the neocortex. Any recommendations on how to learn more?

@7:45 It’s the size of a dinner napkin, 2.5mm think, uniform, and similar in
other animals.

@9:50 if you took the optic nerve and attached it to another part of the
neocortex, that part would become the visual region.

@34:30 Thousand brains theory of intelligence.

~~~
Balgair
> if you took the optic nerve and attached it to another part of the
> neocortex, that part would become the visual region.

FYI, that is _super_ hand-wavey and covers over a lot about how the path of
the information from the cones/rods gets into V1. The chain of neurons that
pass infomation from your eyes to V1 is well studied [0]. Interruptions in
that path cause a lot of sightedness issues and are not fun diseases to have.
The musician, Stevie Wonder, among others, aledgedly has a form of blindness
known as blindsight [1] where relfexes to motion are perserved, but
information is not passed into the conscious mind.

In the end, though neuroscience is a facinating subject, we're just in the
beginning of our understanding of the brain. More research is needed.

[0]
[https://en.wikipedia.org/wiki/Optic_chiasm](https://en.wikipedia.org/wiki/Optic_chiasm)
a good place to start learning about the chain of information transfer.

[1]
[https://en.wikipedia.org/wiki/Blindsight](https://en.wikipedia.org/wiki/Blindsight)

EDIT: Additionally, if you want to learn more about neuroscience, the best
place to look is at Kandel's _Principals of Neural Science_ [2]. It is a
_tome_ of a book, but is the best place to get a deep dive into the brain and
our understanding of it. I've not yet seen anything else that is somewhat
accessible to the general public but also gets into all the issues with any
particular experiment. Most pop-sci book brush over a lot of the very
important and thorny issues that each experiment has. I'd also love to know of
a good book that is more accessible than Kandel.

[2] [https://www.amazon.com/Principles-Neural-Science-Fifth-
Kande...](https://www.amazon.com/Principles-Neural-Science-Fifth-
Kandel/dp/0071390111)

~~~
outworlder
Incidentally, Blindsight is the name of a science fiction novel which I really
like. The name is inspired by that phenomenom, but it covers a lot of ground,
from neurological processes to how aliens would look like (and how they would
think), chinese room, etc.

[https://rifters.com/real/Blindsight.htm](https://rifters.com/real/Blindsight.htm)

------
fjfaase
What about bird intelligence? The modern view is that the parts of the bird
brain use to be called the pallium actually are organized in a similar way as
the neocortex. Although a bird brain is usually smaller than the mammal brain,
it has in many case the same amount of neurons. Actually, a high percentage of
neurons are located in the pallium than in the neocortex. Please note that our
cerebellum contains more than half of the number of neurons in the brain.

------
canistel
Dileep George, one of the original co-founders of Numenta have since moved on
to co-found Vicarious.They were in the news after breaking captcha, that too
with commonplace hardware. They are far too secretive, but there is reason to
believe that they too are part of the HTM legacy...

~~~
p1esk
They publish occasional DL papers, but results are not impressive, so
mainstream DL people ignore them.

~~~
raindeer3
The don't do deep learning. Their papers are based on graphical models. Their
RCN model can be said to be inspired by Numentas earlier HTM model that dileep
developed there, but is very different in fundamental ways.

~~~
p1esk
Yeah they do, here are 3 papers on convnets:

[https://arxiv.org/abs/1611.02252](https://arxiv.org/abs/1611.02252)

[https://arxiv.org/abs/1611.02767](https://arxiv.org/abs/1611.02767)

[https://arxiv.org/abs/1706.04313](https://arxiv.org/abs/1706.04313)

But it seems like lately they focus more on RL for robotics.

------
yters
No fundamental difference between centralized and decentralized computation,
even if you throw in stochasticity. It is all reducible to a deterministic
Turing machine.

------
sysbin
Can anyone clarify Jeff Hawkins usage of the word time? I'm wondering why he
isn't using rate or frequency instead of time.

~~~
p1esk
“Time” as in “predicting the future”. Same as in RNNs.

------
RosanaAnaDana
I'm like 50/50 on whether or not Lex is himself an artificial intelligence.

------
devoply
Once intelligence is "solved", most humans are completely and utterly screwed.

~~~
UnpossibleJim
There's the "Wally vs. Star Trek" theory of Artificial Intelligence (I know it
sounds goofy, but it makes a difference when looking towards the direction of
AI). If you look at the direction AI will take, science fiction is really all
we have to go on and we have three platforms:

Terminator - To be avoided at all costs (everyone agrees)

Wally - AI does its best to "take care" of us, turning us into infantile,
unthinking, pleasure seeking dweebs. The ultimate basal brain utopia. But
empty.

Star Trek (TNG) - AI as an assistant to help us fulfill our higher purpose and
make us better at being a higher order being. There is no doubt that the
ship's computer has advanced AI if you look at the questions posed to it by
the engineers and the crew, but it does not impose its "will" upon them. It is
an observer until directly asked, even when death is at hand (seemingly).

There is a very good video on YouTube about it, but I can't find it (right
now) that explains it in a very accessible way (that's pretty entertaining)
with no real knowledge of the underlying functionality necessary. I suggest
you watch it. It's great!

EDIT:
[https://www.youtube.com/watch?v=48mf2QUtUmg&t=949s](https://www.youtube.com/watch?v=48mf2QUtUmg&t=949s)

~~~
whatever_dude
When thinking on where we're headed, I tend to subscribe to the Culture
theory, which I suppose is a bit in the middle between Wally and Star Trek: AI
is certainly superior and running the show, but they're happy letting humanity
live as if they're the bosses.

In it, humans are basically treated like pets or toddlers, in that they "do
stuff" but they're kind of oblivious to the fact that AI is moving the world
forward.

~~~
jacobush
Are you all talking about Wall-E?

~~~
p1esk
Yes :)

