
Computational Power Found in the Arms of Neurons - theafh
https://www.quantamagazine.org/neural-dendrites-reveal-their-computational-power-20200114/
======
cr0sh
I don't think any serious deep-learning researchers believe that their
artificial models represent the reality of how an actual brain (or even the
individual neurons) actually work.

That's an idea, I believe, that has been perpetuated in the general public's
mind by popular science articles and descriptions about deep learning and
neural networks, in a (largely successful) attempt to attract and gain
attention for the subject and those articles.

One thing that did capture my imagination from this new research, though, as
applied to deep learning, is whether this would necessarily cause any change
in actual deep learning models?

That is, even if the dendrites are performing more complex computations, does
that change anything about the "standard" layered model of deep learning
systems, or does one simply add more artificial neurons to the model to
compensate (and thus, nothing is really gained)?

Finally - and I know this is not the focus of this research - this new
understanding doesn't seemingly do anything to advance our knowledge of how
actual brains and neurons learn; as mentioned elsewhere, our current
artificial models implement "back propagation" for learning purposes. While
that has been a radically successful method, even its creator (Hinton)
believes it to be wrong (and is working toward a different model of so-called
"capsule networks" in response). As far as I am aware, no form of back
propagation has been found to be performed in natural brains or neural
networks, so we really don't know how this aspect works. Plenty of competing
theories and ideas have sprung up regarding this.

Hopefully one day we'll figure it out; I'm of the opinion that, like so many
other things that have been discovered in the past, when we do figure it out,
it will probably be something extremely simple and curious as to how it
remained overlooked for so long.

I kinda have a thought or idea that its done via "societal feedback loops" \-
that is, brains talking/interacting with other brains - though how you
integrate this with a feedforward system isn't apparent to me. I'm also
certain, though, that this idea isn't something new or unique (and it very
likely is wrong, too).

------
neural_thing
I have written a short, clear, and dense book about this exact subject. It's
free! Read it here:

[http://www.corticalcircuitry.com/](http://www.corticalcircuitry.com/)

~~~
neural_thing
A preprint that came out after I released the book that I would have
mentioned:

[https://www.biorxiv.org/content/10.1101/613141v1.full](https://www.biorxiv.org/content/10.1101/613141v1.full)

They model a single cortical neuron as a deep neural network with 7 hidden
layers consisting of 128 hidden units each.

Biological neurons are A LOT more powerful than "neurons" in neural networks.
If you hear a claim about computer-brain parity being close - the people
making it almost certainly don't understand the power of cortical neurons.

------
GolDDranks
What I find interesting about this finding: I wonder if intra-dentrite vs.
inter-dentrite vs. inter-neuronal learning has different learning rates,
activation functions etc. If that's the case, maybe artifical networks could
benefit such variety.

~~~
raverbashing
More than the computational aspects of neurons (which might be abstracted
computationally), the learning process is something that seems it's still more
of a mystery.

The brain doesn't backpropagate. So how do you do it?

~~~
im3w1l
Well part of it is local unsupervised hebbian learning, fire-together-wire-
together, as it's succinctly put. In particular, neurons will try to fire more
like upstream neurons and less like downstream neurons as determined by time-
delay in correlation.

Then there is also the brains reward center, the one that famously goes wrong
when you get addicted to drugs. It releases chemicals which reinforce
behavior.

And finally some parts are "hard-coded" to be a certain way our DNA.

------
pflanze
See earlier discussion at
[https://news.ycombinator.com/item?id=22061718](https://news.ycombinator.com/item?id=22061718)

------
erikerikson
Actually, single layer/later free networks can learn XOR under naturally
occurring modulated learning conditions[1]. I am the author and was able to
demonstrate 6-dimensional XOR learned in a Hopfield network using Hebbian
learning.

[1]
[https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT...](https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT3DiZz/view?usp=drivesdk)
(PDF)

------
axilmar
Has anyone ever tried to develop (or more appropriately, grow) an actual
artificial biological brain, instead of a computer program?

Perhaps trying to grow something like a brain may give us an insight on the
actual algorithm used.

~~~
Enginerrrd
Yes, and they've trained neurons to do tasks successfully. But the way they
trained them is very different to how they naturally work. I don't think we
know how to train a collection of neurons the "natural" way.

[https://www.newscientist.com/article/dn6573-brain-cells-
in-a...](https://www.newscientist.com/article/dn6573-brain-cells-in-a-dish-
fly-fighter-plane/)

------
zie1ony
Hopefully this research will boost deep learning.

