
Do Bayesian statistics rule the brain? - dood
http://www.mindhacks.com/blog/2008/05/do_bayesian_statisti.html
======
kurtosis
Here's a contrary view: There is a very famous tradition in biology of
deriving the form and function of organisms by a mathematical optimization
(see D'arcy Thompson). It shouldn't be surprising that the nervous system
would converge on Bayesian stats. The dutch book argument shows that any other
method of updating beliefs about the world will lose money in a gambling
strategy. If "gambling" is replaced by "foraging" or "mating" then Bayes is
the optimal way to play. (whether we're playing for the interests of the
organism or its genes)

But saying that the brain is Bayesian is not that profound. It's like saying
that the brain is ruled by electricity. The key is what priors are being
modeled, how is inference implemented with neurons, and what constraints, or
"hyperparameters" are built into these priors.

~~~
LPTS
You obviously know more about the science and math here than I do. So I'm
offering these thoughts deferentially. I'm coming at this as a jack of all
trades type who did a ton of philosophy of mind and cognitive science stuff in
college but somewhat deficient with the hard science.

"What priors are being modeled"

I think this is the most interesting thing. There is so much information about
how memories are recreated as you remember them, and about memory being
unreliable. Even if you had an answer to the question "what priors are being
modeled?", it would need to be indexed to a time. Indexing your question to
times is where it gets interesting.

This would create a question that would be a lot harder and necessary to
understand to get at the truth of this. Something like "How can the constantly
changing sea of fragmented memories we base our idea of self on serve as
reliable priors at all." Or, less poetically and more formally:

Given a person, P, at 2 times, T1 and T2, and two sets of priors, M1 and M2,
such that M1 and M2 belong to P, at T1 and T2, respectively, and between T1
and T2, P obtained exactly one datum, D, what is the relationship between M1
at T1 in P, and (M2-D) at T2 in P?

And, generally, what calculus explains the relationship between Mx at Tx in P,
and (My-Dn) at Ty in P, where Dn is the sum of the data P acquired between Tx
and Ty?

I don't think M1 is equivalent to M2-D in the first, and I don't think that
My-Dn is equivalent to Mx in the second. I think all the evidence about
memories being inconsistent, recreated, count towards my intuition. I also
think the way things like a sudden smell can cause priming of memories, or an
emotional state can alter memory recall suggest this.

"how is inference implemented with neurons"

This seems like the most trivial, least profound, and most uninteresting part.
We know that the brain does these inferences, and we know that even very
simple systems based on cellular automation can be turing complete and make
inferences. The particular details will not shed light on the fundamental
problems understanding mind.

"what constraints, or "hyperparameters" are built into these priors."

It sounds to me like their idea that hallucinations and delusions as
breakdowns of Bayesian statistics functionality would bear fruit that could
answer this question, although I don't have an answer.

I also wonder, wasn't "the brain is ruled by electricity" a profound insight
for it's time?

~~~
kurtosis
I've always thought of Noam Chomsky's "language acquisition device" as a kind
of prior on the space of languages that's hardwired into the brain. The
process of learning a language consists of fitting the parameters of this
model to the sense data that each child is immersed in. Of course Bayes' rule
provides one way to update our guess about the parameters of this model given
the current data but who knows if that's what's really going on. I think the
evidence is pretty strong that this flexibility in the space of priors over
language only lasts a for small precious window until we become adults, and
then we're all stuck with lame accents. So it does indeed have some time
dependence.

When I was talking about the implementation of inference I was referring to
the people doing experiments trying to figure out the "neural code" -
Questions like: how much information is conveyed by a single spike? Is there
any meaning in the rate of the spikes? How much information is there in a
coincidence when two or more neurons fire simultaneously? What rules govern
the plasticity of the activation or inhibition relationships among neurons?
Does any of this have anything to do with probability? A great book on this is
"Spikes" by de Ruyter and Bialek et. al.

I didn't mean to sound dismissive about the electricity thing - I think
hodgkin and huxley were both given medicine nobels for their work on the
nerves. I guess the electricity thing really goes back the 1700's when galvani
made a dead frog move with electric current.

~~~
LPTS
"I think the evidence is pretty strong that this flexibility in the space of
priors over language only lasts a for small precious window until we become
adults, and then we're all stuck with lame accents."

I think this is true for most people, not everyone.

If this is true for most people, we can locate these differences by comparing
autistic people who quickly acquire languages as adults with normal people.
It's interesting that autistic people skilled with language often don't have
noticeable accents in that language, even with a language they learn in
adulthood.

This flexibility in the space of languages involves a honing of perception. As
you learn your first language, you become better at perceiving the sounds of
that language, but your ability to perceive the sounds of another language go
down. For example, a person who speaks exclusively english speaker cannot
discern between sounds that would have different meanings that a chinese
speaker would easily hear.

(Deferentially), I think you are wrong about where the hardwiring occurs. I
think the hardwiring that limits most peoples ability to learn language arises
is perception, not the space of priors. I believe if you could change the
hardwiring that occurs in making sensory discriminations relevant to the
language, the "space of priors" would still show the ability to learn
language. It's true that there is a precious window for most people to learn
languages, but that's because the perceptual paths get hardwired, not the
memory and processing that make up what we seem to be calling the "space of
priors"

But, that's not the scale of time dependance I was thinking. Even if the scale
of time dependence was 5 seconds, and the new data was just seeing it was
raining out, the way your brain understands the concept of rain involves
accessing memories it destroys and recreates as part of recognizing "rain", as
well as the particulars of your emotional state and what's going into your
sense of smell, so much that the set of priors 5 seconds before would be
different then the ones now, even if you took out the information about it
raining.

------
schtog
if bayesian probability is how the brain works, why is it so hard for most
people to understand this: <http://en.wikipedia.org/wiki/Monty_Hall_problem>

seriously, even the most clever people even have huge trouble grasping it.

~~~
gnaritas
Because people don't necessarily have a conscience understanding of how the
lower layers of their minds work. Being a Bayesian neural network doesn't
imply you understand Bayesian neural networks in general.

~~~
LPTS
Even if your brain is ruled by Bayesian statistics, that doesn't make YOU a
bayesian neural network, or even tell us much about identity at all (that
would be to confuse the hard and easy problems of consciousness). But your
point about a bayesian network not having to understand other bayesian
networks stands, and is the right answer to the guy you responded too.

~~~
gnaritas
You're correct, however, we still haven't defined what _you_ actually is yet.
Conscienceless could merely be an illusion or as Jeff Hawkins says, it could
merely be what it feels like to have a neocortex.

I do reject the idea that conscienceless is anything more than the byproduct
of the physical process of the brain, of course, I don't think that's what
you're implying, I hope.

~~~
LPTS
I didn't mean to imply anything about my beliefs. I have a more general
atheoretic attitude towards it. My belief could be formalized "For any person
with a set of beliefs about consciousness, their beliefs are wrong, and they
are not justified in holding them."

You said: "I do reject the idea that conscienceless is anything more than the
byproduct of the physical process of the brain, of course, I don't think
that's what you're implying, I hope."

This is exactly like saying: "I reject the idea that final digits of pi is
anything more than the byproduct of the physical processes in my computer's
calculator program"

I don't think actually I implied not-physicalism, but I would be happy to
endorse it.

The idea that consciousness is illusory relates to high functioning autism in
people who develop the theories in a complicated way. Ludwig Wittgenstein, for
example, had a very autistic picture theory of language that gave way to a
non-autistic theory as he got older. For very intelligent people who are blind
to other minds, studying consciousness is very important, because, to them, it
seems like the most logical way to understand people. I don't mean this as
criticism of intelligent people with autism. But, relying on these people to
understand consciousness is like relying on a colorblind person who has
mastered scientific and mathematical models of analyzing wavelengths of light
to explain color as an artist experiences it.

~~~
cperciva
_This is exactly like saying: "I reject the idea that final digits of pi is
anything more than the byproduct of the physical processes in my computer's
calculator program"_

I reject the idea that pi _has_ any final digits. :-)

~~~
LPTS
Well, that's exactly the point. I cribbed that line from a poem I wrote:

Trying to find / the explanation of why we are conscious / in patterns in the
brain / is like / trying to find / the last digit of pi / in the circuitry of
a scientific calculator

------
robg
No, but it's as good a description as any for neuronal interactions. The
problem though is not with a finite set of connections, but rather the
infinite number of possibilities.

~~~
eru
Or rather - very large number of possibilities.

------
michael_dorfman
Anyone have a clue as to why (or, indeed, whether) Bayesian mathematics work
any better than the more old-school neural network algorithms?

~~~
robg
Honestly, I think it just comes down to simplicity. Bayesian algorithms tend
to make fewer assumptions a priori. By contrast, look at something like
backprop.

~~~
kurtosis
I respectfully disagree:

Bayesian methods do not necessarily make fewer assumptions. They just allow
you to make your assumptions explicit and to evaluate how sensitive your
conclusions were to those assumptions. Conventional statistical methods all
have ways of doing this but they are ad hoc. In this sense you are totally
correct - bayes is simpler.

Also because bayesian inference in any interesting problem domain is often
computationally impossible, you must make approximations - which adds even
more complexity.

Backpropagation on the other hand, is remarkably simple -- it's just the chain
rule from calculus.

~~~
LPTS
If the brain uses bayesian inference, could emotions and intuitions perhaps be
functionally equivalent to the approximations needed to do the impossible
computations you are talking about?

~~~
nazgulnarsil
people who froze up when confronted with incomplete information about
probabilities of outcomes were out competed by those who did something, even
if they chose wrong a lot of the time.

------
andreyf
No, or we wouldn't suck at Bayesian statistics so much:

<http://en.wikipedia.org/wiki/Base_rate_fallacy>

