
Foundations of probability theory - websec
https://terrytao.wordpress.com/2015/09/29/275a-notes-0-foundations-of-probability-theory/
======
GregBuchholz
I don't know what mathematicians think of it, but I enjoyed "Probability
Theory: The Logic of Science".

[https://www.google.com/#q=probability+the+logic+of+science](https://www.google.com/#q=probability+the+logic+of+science)

~~~
joseraul
Its take on probability as a way of modelling our brain is refreshing, for
instance, the beginning of chapter 1:

Suppose some dark night a policeman walks down a street, apparently deserted;
but suddenly he hears a burglar alarm, looks across the street, and sees a
jewelry store with a broken window. Then a gentleman wearing a mask comes
crawling out through the broken window, carrying a bag which turns out to be
full of expensive jewelry. The policeman doesn't hesitate at all in deciding
that this gentleman is dishonest. But by what reasoning process does he arrive
at this conclusion? Let us first take a leisurely look at the general nature
of such problems.

A moment's thought makes it clear that our policeman's conclusion was not a
logical deduction from the evidence; for there may have been a perfectly
innocent explanation for everything. It might be, for example, that this
gentleman was the owner of the jewelry store and he was coming home from a
masquerade party, and didn't have the key with him. But just as he walked by
his store a passing truck threw a stone through the window; and he was only
protecting his own property. Now while the policeman's reasoning process was
not logical deduction, we will grant that it had a certain degree of validity.
The evidence did not make the gentleman's dishonesty certain, but it did make
it extremely plausible. This is an example of a kind of reasoning in which we
haveall become more or less proficient, necessarily, long before studying
mathematical theories. We are hardly able to get through one waking hour
without facing some situation (e.g. will it rain or won't it?) where we do not
have enough information to permit deductive reasoning; but still we must
decide immediately what to do.

~~~
joe_the_user
Ah, but policeman's actual thought process might have actually been purely
deductive or purely reflexive, it might only be that the probabilities justify
a series of a discreet calculations.

------
tyc2021
The fact that Shannon entropy is still relevant in numerous modern mathematics
research is amazing.

Apart from minor typos that make him a "sloppy mathematician", he is a good
educator based on my personal experience taking grad. courses from him. He's
not the passionate high school STEM teacher type, but he offers great
insights. I think which textbook he used is somewhat secondary. For courses he
had taught before, he usually pick a standard text and teach based on the
material he wrote in his blog post, including exercises.

------
amelius
I have a question that might be related to the topic. I have found that often,
when solving problems related to probability theory, it is more convenient to
think in terms of combinations than to think in terms of probabilities
(perhaps because I'm a programmer). Would it be possible to devise a
programming language that allows me to program a filter that selects the
desired outcomes out of the complete set of combinations, and to have the
compiler automatically deduce a closed-form formula for the probability
(without enumeration)? If this is not possible in general, what would the
restrictions on this programming language be, to make it work in practice?

So for example, given the question what the probability is that, when throwing
4 coins, 2 of which will be heads; I could write a function that generates all
possible outcomes "TTTT", "TTTH", etc. And I could write a filter function
that returns true for "TTHH", "THTH", "THHT", etc.

~~~
dragandj
This is possible, and in fact probably implemented in some probabilistic
programming languages, but I think you are looking at the wrong direction.

The point is that even for fairly simple real use cases, the computation
complexity is so huge, that all computers in the world couldn't compute it in
your lifetime if you don't employ some approximation or optimization and stick
to naive algorithms.

So, that is what the whole field of machine learning is about: finding some
clever ways to deal with random variables in a computationally feasible way...

~~~
foobar2020
All those probabilistic programming languages will become exponentially faster
once we have feasible quantum computers, since BPP \in BQP. We currently use a
weaker inclusion, BPP \in PSPACE, as the core execution model.

~~~
j2kun
Check your facts, because this is simply not true.

------
graycat
Looked at the text to be used, Durrett. There's more and with higher quality
in any of M. Loeve, J. Neveu, K. Chung, and L. Breiman.

For what Tao wrote on his page about _determinism_ or whatever on that page --
just f'get about that.

And what he wrote, confusing a sample space and a probability space, just say
that he had a bad day that day.

~~~
nerd_stuff
Out of curiosity, what's the background that lets you be so dismissive of a
Fields Medal winner?

At this point I see a blog post written by a well respected mathematician whom
I feel comfortable trusting and it's being brushed aside by I don't know who.

~~~
d9fb698e010974b
Well you can just read the reviews to see that the Durrett text isn't well
regarded while others like Chung's are. And the criticism about a probability
space not being a sample space is correct, but I think it's clear what Tao
meant there, namely that the sample space would be a part of a probability
space.

~~~
keithpeter
Thanks

The Amazon UK reviews of Chung's book lead me to _A Probability Path_ by
Sidney Resnick which appears to be aimed at non-mathematicians. I have
invested (speaking as a renegade physicist lacking a systematic exploration of
measure theory).

