

The Earth is Round (p &lt; 0.05) (1994) - nathell
http://ist-socrates.berkeley.edu/~maccoun/PP279_Cohen1.pdf

======
nathell
Side note: while trying to submit this, I've noticed that the submission form
doesn't escape less-than signs automatically. I had to manually escape the
entity.

------
theophrastus
Why 0.05? because 1 in 20~22: two standard deviations. so if your distribution
isn't normal Gaussian you don't even have tradition on your side, (or perhaps
1 in 20 doesn't fit your risk model for _say_ , testing pharmaceuticals?)

[http://www.jerrydallal.com/lhsp/p05.htm](http://www.jerrydallal.com/lhsp/p05.htm)

"Still, why should the value 0.05 be adopted as the universally accepted value
for statistical significance? ... It was Fisher who suggested giving 0.05 its
special status... The value for which P=0.05, or 1 in 20, is 1.96 or nearly 2
...Deviations exceeding twice the standard deviation are thus formally
regarded as significant. Using this criterion we should be led to follow up a
false indication only once in 22 trials"

------
tedunangst
It's a good article, but I was really hoping they'd prove the earth round (or
not). Sadly the only mention is the title.

------
Selfcommit
Relevant XKCD [http://xkcd.com/1478/](http://xkcd.com/1478/)

~~~
ikeboy
Hypothesis: the submitter either came across the link while researching this
xkcd, or remembered about the link after seeing the xkcd.

Unfortunately I just realized I have no idea how to do statistics anymore, so
we'll have to wait for the submitter to confirm.

~~~
nathell
Submitter here. The latter is correct.

------
murbard2
There's a very easy way to do null hypothesis testing.

Take the sha256 sum of all your experimental data. If the first fives bits of
the hash are 0, then you can reject the null hypothesis with p < 3%

~~~
martinko
What is a sha256 SUM?

~~~
murbard2
a cryptogtaphic digest,
[http://linux.die.net/man/1/sha256sum](http://linux.die.net/man/1/sha256sum)

The message is a joke intended to make people think about limitations of null
hypothesis techniques. On the one hand, the technique I describe is
statistically rigorous, on the other hand, it is stupid because it has no
power and is always random.

However, in the Popperian framework of null hypothesis testing, there is no
way to even express the fact that this test is worse than any other.

------
api
From 1994... that's depressing.

~~~
jtth
The discussions over the problems of frequentist statistics are old, old
problems in which people have grown and died without settling. It seems like
we're getting close, finally, now that Bayesian methods are becoming easier
and easier.

~~~
Chinjut
Mind you, there are also a number of old, old discussions of the problems of
Bayesian statistics, which I fully expect to grow and die without the world
having resolved (or, even worse: considered resolved too glibly!).

