
Information, Physics and Computation (2009) - KKKKkkkk1
https://web.stanford.edu/~montanar/RESEARCH/BOOK/book.html
======
munificent
Can I rant on how bad much mathematical notation is in terms of usability? On
the first page, we have:

> A discrete random variable X is completely defined by the set of values it
> can take, X , which we assume to be a finite set, and its probability
> distribution {pX(x)}x∈X . The value pX(x) is the probability that the random
> variable X takes the value x.

Actually, we don't have that, because copy/pasting it here loses essential
information. This paragraph uses "X" to refer to three entirely different
concepts, distinguished only by _the font used_.

Not only that, but it relies on subscripting and case, yet uses letters like
"p" and "x" which difference in case is mainly in their vertical size and
position.

Alright, rant over.

~~~
adrianratnapala
They are not distinguished by font: there is an an upper-case "X" (to mean a
random variable) and a lower-case "x" to mean a sample from that radom
variable. This is a very reasonable and useful case convention.

Finally there is the Greek letter Chi to mean the set over which that self-
same RV varies. Chi seems deliberately chosen because it resembles "X". That
might not be a good idea, but there is a defensible argument for it:

The domain Chi is a piece of mathematical machinery that is logically needed,
but is not very important to the main disucssion. He wants to note it down,
but also let it fade into the background. It's precesly because you want to
pay too much attention to it that you are getting upset.

(I'm more worried that he seems to elide the distinction between the domain of
an RV and the underlying sample-space. But then, sample-spaces are also
mathematical machinery that aren't needed once the rubber hits the road.)

~~~
alimw
That's not a chi, it's a calligraphic X. $\mathcal{X}$ in LaTeX.

------
dramaqueen
I am interested in studying Computer Science Theory for the Information Age by
Hopcrot and Kanaan[0]. How do these books compare? Is there much overlap
between the two? Are they totally different?

[0] [https://www.cs.cmu.edu/~venkatg/teaching/CStheory-
infoage/ho...](https://www.cs.cmu.edu/~venkatg/teaching/CStheory-
infoage/hopcroft-kannan-feb2012.pdf) (PDF)

~~~
LolWolf
Very different. Their text focuses on the connections between stochastic
systems, information theory, and machine learning/discrete models in
mathematics, whereas the one you linked is purely about statistics and machine
learning from a classical perspective.

------
eli_gottlieb
It looks deep but unmotivated. That doesn't mean there's no motivation, but
I'd like it communicated to me more clearly: why should I value the material
of this book being put together all in one place?

~~~
Retra
I'm personally struggling to find a way of answering this that doesn't sound
ridiculous on account of the fact that I've been obsessing over this stuff for
several decades. These subjects address the heart of what it means to be able
to make decisions and use language effectively. This subject matter is
essentially a continuation of a broad attempt at a modern conceptualization of
_meaning_ , and as such it's value is immediately obvious to me... You should
value this material because it is this kind of material that describes what
'value' is, why it is important, and what limitations it has.

But this is all absurd when I try to apply it to a specific book. It's a low
rung on a ladder so tall nobody has seen the top.

~~~
eli_gottlieb
>I'm personally struggling to find a way of answering this that doesn't sound
ridiculous on account of the fact that I've been obsessing over this stuff for
several decades.

Then go ahead and sound ridiculous! You've marinated your mind in the
material, so I can only benefit from knowing what connections you see.

>These subjects address the heart of what it means to be able to make
decisions and use language effectively. This subject matter is essentially a
continuation of a broad attempt at a modern conceptualization of meaning, and
as such it's value is immediately obvious to me... You should value this
material because it is this kind of material that describes what 'value' is,
why it is important, and what limitations it has.

Really? I might have used similar descriptions for certain applications of
probabilistic modelling and information theory to neuroscience and cognitive
science, but why for this particular conjunction of graphical models,
randomized forms of NP-complete computational problems, and statistical
physics?

>But this is all absurd when I try to apply it to a specific book.

Oh. Well, can you just spout off in general for me, then?

>It's a low rung on a ladder so tall nobody has seen the top.

Well, nobody would call it research if they knew exactly what they were doing.

------
tkvtkvtkvtkv
nice to see factor graphs included

