
Research on the cerebellum yields rewards - laurex
https://www.nature.com/articles/d41586-020-00636-x
======
blennon
I did my Ph.D. work on the cerebellum and proposed a theoretical model of
reinforcement learning and can add some context for why this is a "big deal".

First, it represents a fundamental shift in how the cerebellum learns. For a
long time, it was thought that the cerebellum learned liked a perceptron
neural network, i.e. that an error signal was computed and used to change the
strength of synapses within the cerebellum to result in the correct output
signal to guide motor control (and, we know now, cognitive control). In other
words, the cerebellum was a _supervised learning_ machine [1]. But how these
error signals were actually computed using the neuronal circuitry was never
made clear; most arguments centered around the microcircuitry of the inferior
olive. In a perceptron, an error signal is the difference between the correct
output and the predicted output. But how is the "correct output" supplied?

As far as I know the first to propose the cerebellum learned by reinforcement
learning was the famed cerebellum researcher Richard Thompson [2].
Unfortunately, the idea was only vaguely sketched out the field didn't take
this very seriously and continued on with the general belief that the
cerebellum learned by supervised learning.

To me and my collaborator, Tadashi Yamazaki, it seemed a more natural signal
that the nervous system could capably supply would be a graded reward signal.
Moreover, this meant we could interpret the structure of the cerebellum within
the theoretical frameworks of reinforcement learning such as the actor-critic
framework [3,4]. This is the second reason these finding are a big deal: the
paradigm shift to the cerebellum being a reinforcement learning machine, if
correct, will be a boon for building better models of it. In the last few
years there has been some impressive work done with reinforcement learning in
artificial neural networks that could be applied to models of the cerebellum,
especially within context of the brain at large.

[1] Doya, Kenji "What are the computations of the cerebellum, the basal
ganglia and the cerebral cortex" (1999) [2] Thompson, Richard "The nature of
reinforcement in cerebellar learning" (1998). [3] Lennon, William "Towards
more biologically plausible computational models of the cerebellum with
emphasis on the molecular layer interneurons" (2015) [4] Yamazaki and Lennon
"Revisiting a theory of cerebellar cortex" (2019)

~~~
vosper
> most arguments centered around the microcircuitry of the inferior olive

If, like me, you wondered how a poor fruit has microcircuitry, well, I think
they're actually talking about the Inferior Olivary Nucleus. Per Wikipedia:

> The inferior olivary nucleus (ION), is a structure found in the medulla
> oblongata underneath the superior olivary nucleus. In vertebrates, the ION
> is known to coordinate signals from the spinal cord to the cerebellum to
> regulate motor coordination and learning. These connections have been shown
> to be tightly associated, as degeneration of either the cerebellum or the
> ION results in degeneration of the other.

[https://en.wikipedia.org/wiki/Inferior_olivary_nucleus](https://en.wikipedia.org/wiki/Inferior_olivary_nucleus)

