
All of Statistics, by Larry Wassserman (2013) [pdf] - rfreytag
http://www.stat.cmu.edu/~larry/all-of-statistics/index.html
======
ilzmastr
Happy to see a book like this trending on hn, especially with a sentence like:
"Using fancy tools like neural nets, boosting, and support vector machines
without understanding basic statistics is like doing brain surgery before
knowing how to use a band-aid." in it's preface. I definitely agree, since I
wasted a lot of time doing fruitless surgeries before I went and learned about
band aids in depth.

From my look at Part 1, it has some great coverage of the basics, all of which
are important. Some of the fundamentals that I see left out are rightly left
out since they require experience in real analysis to appreciate, and maybe
aren't very actionable. There's few proofs, but, since the goal is a quick
understanding, I can also appreciate this.

It looks to me like a great intro of statistics for CS people, as the author
says.

~~~
clishem
> "Using fancy tools like neural nets, boosting, and support vector machines
> without understanding basic statistics is like doing brain surgery before
> knowing how to use a band-aid."

Having studied both statistics and neural networks, I'm not sure if I
completely agree with that quote. There are lots of neural network
applications that have little to do with statistics (image recognition with
convolutional neural networks for example).

I am pretty sure that the author means neural networks for statistical
applications though.

~~~
DavidSJ
Image classification has everything to do with statistics: you're guessing the
probability distribution over the classes conditional on the input image
vector; the model is trained through a process of statistical inference (using
gradient descent).

~~~
argonaut
You have stated something that someone with a high school-level (e.g.
superficial) understanding of probability/statistics would understand. Most
research in neural networks requires only a very superficial level of
statistics.

There are certainly many areas of neural networks where statistics is
important (more theoretical areas), but those don't form the core of the
research field.

Also, calling (stochastic) gradient descent a form of statistical inference,
while _technically correct_ , is a ridiculous stretching of the term. No
researcher considers SGD to be a statistical inference algorithm.

------
Zuider
Sadly, no link to free eBook, which is not surprising because it seems that
the book is still in print, having been released as recently as 2004, and
updated in 2005 and 2013.

This post links to the website supporting the book and provides links to
errata, code and data. The links on the page to Springer and Amazon are
broken: Here are valid links:

[http://www.springer.com/de/book/9780387402727](http://www.springer.com/de/book/9780387402727)

[http://www.amazon.com/All-Statistics-Statistical-
Inference-S...](http://www.amazon.com/All-Statistics-Statistical-Inference-
Springer/dp/1441923225)

Here is the Google Books link:

[https://books.google.ie/books?id=th3fbFI1DaMC&printsec=front...](https://books.google.ie/books?id=th3fbFI1DaMC&printsec=frontcover#v=onepage&q&f=false)

~~~
rjeli
Not sure about HN's policy on posting links to pirated material, but as a
Freedom of Information supporter, I will note that the book can be found at
[http://gen.lib.rus.ec](http://gen.lib.rus.ec).

~~~
dmix
Direct link [pdf] [9mb]
[http://libgen.io/get/B9E6052395BA047BD154EA45130026FC/Larry%...](http://libgen.io/get/B9E6052395BA047BD154EA45130026FC/Larry%20Wasserman-
All%20of%20Statistics%20-%20A%20Concise%20Course%20in%20Statistical%20Inference-
Springer%20%282004%29.pdf)

------
stdbrouw
While All of Statistics is wonderful in its genre, it really isn't a good
place to start to learn statistics. Firstly because it focuses very heavily on
the theory and contains very little on practical modeling. Secondly because
the theory isn't even necessarily going to be very enlightening: frequentist
statistics is a mathematical tour de force, using every possible hack you can
think of to be able to draw statistical conclusions from nothing more than a
few pen and paper calculations, but as a result frequentist theory won't
actually give you any sort of deeper insight into the core theoretical
foundations of probability and statistics.

If you're new to statistics, try Allen Downey's
[http://greenteapress.com/thinkstats2/index.html](http://greenteapress.com/thinkstats2/index.html)
or Brian Blais' [http://web.bryant.edu/~bblais/statistical-inference-for-
ever...](http://web.bryant.edu/~bblais/statistical-inference-for-everyone-
sie.html). Both are free.

Then, go in depth on regression. Not just feeding in the numbers and getting
back a fitted model, but actually knowing how everything works, what the
common issues are, how to interpret the estimates and so on. Once you've got
that down, read Regression Modeling Strategies by Harrell to go really in
depth.

Or if you're really just interested in prediction, Hastie and Tibshirani is
wonderful of course.

~~~
mathheaven
Brian Blais's free book doesn't contain any reference to the Poisson
Distribution.

For ML Hastie and Tibshirani ISLR is very good but is more for applications of
machine learning: classification, regression and prediction.

~~~
stdbrouw
The Poisson distribution is just a limiting case of the binomial distribution
and not needed to explain any of the concepts that drive statistical
inference, so its absence from an introductory text is hardly something to
fuss about.

------
gaur
Who is this book supposed to be for? Given the heavy emphasis on formalism
(theorem, proof, theorem, proof, theorem, proof), and the lack of a single
example that actually computes a number, I hazard a guess that this book is
not for people who actually want to apply statistics to real problems.

A while back I had to teach myself Fisher matrices and the Cramér–Rao bound to
solve a problem I was working on. I quickly found that 90% of statistics
textbooks and lecture notes on this subject are completely useless for people
like me who want to arrive at a _number_ , not some abstract expression
involving angle brackets or measures or E[...] or whatever.

The Wikipedia article on Fisher information [0] is one such example of a
resource that is full of useless formal crap that crowds out an explanation
for real people about how to use this statistical tool. This book appears to
be of the same ilk. (Also, this book apparently does not discuss the
Cramér–Rao bound. Ironic given the book's title.)

If anyone is curious, the single best explanation of the Fisher matrix and the
Cramér–Rao bound that I have found is tucked away in an appendix of the Report
of the Dark Energy Task Force [1]. In one page they manage to concisely and
clearly explain where the Fisher matrix comes from, how to compute it, and how
to apply the Cramér–Rao bound.

[0]
[https://en.wikipedia.org/wiki/Fisher_information](https://en.wikipedia.org/wiki/Fisher_information)

[1] [http://arxiv.org/abs/astro-ph/0609591](http://arxiv.org/abs/astro-
ph/0609591)

~~~
haberman
I found this book to be a godsend. I never took statistics and always wanted
to better understand the deep conceptual ideas in the field. I had so many
frustrating experiences with books that came highly recommend to me, and
turned out to be not what I wanted at all. They spend chapters and chapters
beating around the bush, conversationally talking about general ideas around
data management and measurement bias and research design and different ways of
charting data sets.

I cannot tell you how frustrating this was for me. I wanted just the meat: the
core mathematical concepts on which statistical models and inferences are
built. Don't tell me a folksy story about gathering soil samples, show me the
tools and what they can do, both their power and their limitations. I can
think for myself about how to apply those concepts.

I loved this book for being exceptionally clear and terse. I was hooked from
the first sentence: "Probability is a mathematical language for quantifying
uncertainty." That one sentence makes the concept clear in a way that the
entire chapter on probability from "Statistics in a Nutshell"
([http://www.amazon.com/Statistics-Nutshell-Sarah-
Boslaugh/dp/...](http://www.amazon.com/Statistics-Nutshell-Sarah-
Boslaugh/dp/1449316824)) did not.

I'm not someone who thrives on theorems and proofs, I thrive on _concepts_.
And I found this book dense with clear explanations of the key concepts.

------
MichailP
Can someone say in few sentences what Statistics is all about? I can't shake
off the feeling that it is just glorified curve fitting.

Edit: Please stop the down votes, just an electrical engineer here, with one
basic course in Probability and Stat. :)

~~~
DavidSJ
It is just glorified curve fitting. Glorified curve fitting is a very rich
field.

Another way of thinking about it (described in Wasserman's book) is that
statistics is the inverse problem of probability.

Probability theory asks: given a process, what does its data look like?
Statistics asks: given data, what process might have generated it?

~~~
glial
> Probability theory asks: given a process, what does its data look like?
> Statistics asks: given data, what process might have generated it?

Excellent summary, thank you.

------
marmaduke
This is one of the few hard cover books I have found worth its price. The
statistical principles are succinctly explained such that they can be quickly
implemented.

------
krosaen
This is a major piece of my machine learning self study curriculum:

[http://karlrosaen.com/ml/](http://karlrosaen.com/ml/)

Some links to problem set solutions there

------
zacharyfmarion
One of my favorite books! Shows you how connected statistical inference and
machine learning really are.

------
sandGorgon
Can someone compare this with Hastie and Tibshirani
([https://lagunita.stanford.edu/courses/HumanitiesSciences/Sta...](https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about))
? I wonder which one is more practical

~~~
huac
The Hastie/Tibshirani book ([http://www-bcf.usc.edu/~gareth/ISL/](http://www-
bcf.usc.edu/~gareth/ISL/)) is very practical.

------
pddpro
How does this compare to, say "Introduction to Statistical Learning" and
"Elements of Statistical Learning" by Trevor et al? As I understand, the
former is also supposed to be a concise introduction to statistical concepts
while the latter offers a more rigorous treatment. Where does this book fall
in between?

~~~
kgwgk
This book is not between ISL and ESL. All of Statistics is an introductory
course (and has a much wider scope, including advanced topics) while even the
watered-down ISL assumes that the reader knows already a bit of statistics.

------
ndr
There's a typo in the title, it's Wasserman, 2 's' only. Can someone fix it?

~~~
gaur
I think it's just part of the neue Rechtschreibung.

------
throwaway6497
Price of this book is steep :( $90+ for new and $70+ for used.

