

Ask HN: best way to become a "Numerati"? - Tichy

Not sure if "Numerati" is the best word. I already have a maths degree, but did little with statistics so far. I am capable of reading a book on statistics, but it is not the same as getting some practical experience.<p>I guess I could just start with randomly selected data sets, but somehow that doesn't seem to be enough motivation (like stock market data: seems unlikely to discover something new, with all the Quants already working on it). So I guess what I mean is, among other things: how to find worthwhile statistical problems for starting out? I am not into sports, so betting is kind of out. Also missed the elections, since I am not from the US. I look forward to the book by the 53something website guy, though.
======
mechanical_fish
You need to ask a biologist, a sociologist, or an epidemiologist.

Or you could take up statistical process control. Useful skill, and applicable
to many things besides semiconductor manufacturing. Here's a hilarious quote
that I've remembered for years, from
[http://www.pbs.org/cringely/pulpit/2003/pulpit_20030925_0004...](http://www.pbs.org/cringely/pulpit/2003/pulpit_20030925_000448.html)

 _"I taught over 300 courses for industry where we designed cars and
electronic devices, but it wasn't until one day I took over my wife's kitchen
and used Taguchi to perfect my recipe for vanilla wafer cookies that I
realized how broadly it could be applied," Kowalick recalls. "It took 16
batches, but by the end of the afternoon I had those wafers dialed in."_

Go forth and bake some cookies.

------
jbert
Read some good sources:

[http://www.badscience.net/2008/11/you-are-80-less-likely-
to-...](http://www.badscience.net/2008/11/you-are-80-less-likely-to-die-from-
a-meteor-landing-on-your-head-if-you-wear-a-bicycle-helmet-all-day/)

[http://www.google.co.uk/search?hl=en&q=the+tiger+that+is...](http://www.google.co.uk/search?hl=en&q=the+tiger+that+isn't&btnG=Google+Search&meta=)

Perhaps take some public health scares and look up the papers behind them (if
they exist). Try and work out if the scares are justified or not, based on the
evidence.

Perhaps do some meta-analyses of studies to see if you can determine evidence
of publication bias: <http://en.wikipedia.org/wiki/Publication_bias>

Find something that interests you and Do Science To It.

------
ntoshev
<http://netflixprize.com>

Statistical language processing is also cool:

<http://nltk.sourceforge.net/index.php/Main_Page>

<http://norvig.com/spell-correct.html>

------
hugh
If numerati were a word, it'd be a plural. I'm reasonably sure the correct
singular forms would be literatus (male) and literata (female).

Oh, as for your actual question? I can't think of anything.

~~~
davatk
Nitpicking here, but wouldn't it be numeratus and numerata?

~~~
hugh
Ummm, yes.

Drat.

------
helveticaman
I would still use stocks, like you said, for starting out. I don't know if
you're unlikely to find new things, but it'll probably be more interesting
than sports.

------
dronethebone
Look at some fun stuff like the Monty Hall problem, and Benford's law. These
are delightful examples of how counterintuitive probabilities and statistics
can be. If either of these things surprise you, you are in the great majority,
and unlikely to perform practical statistical work without first doing a great
deal of studying. Nothing to be ashamed of, not even Gauss was born knowing
this stuff.

~~~
Tichy
Well I am familiar with these. I just never got my hands on a real world data
analysis problem.

------
fbbwsa
i think there are still plenty of fun things to find in stock market data.
most quants are looking at the same problem. i play with market data a lot and
you'd be amazed how uneducated the general public seems to be.

also, i second ntoshev's recommendations for netflix prize -ish applications.
collaborative filtering/clustering seems to be an inadequate science at best
as far as i can tell.

------
sammyo
If you're a hacker you might look for a niche in the R project and do some
development.

------
scott_s
Look at graduate programs in statistics.

~~~
Tichy
That occured to me, too, but is it really necessary? Also, will a graduate
program lead to practical applications?

~~~
scott_s
It's never necessary, but it might be the easiest place to find interesting
problems to work on, and the place that is most conducive to study. It's also
probably the best lead-in to finding a job in the area.

You don't have to do a PhD. Look up some Master's programs, which typically
take two years.

Disclosure: I'm a Computer Science PhD student.

