
LHC May Have Found Crack in Modern Physics - evo_9
http://www.wired.com/wiredscience/2011/11/lhc-crack-physics/
======
pak
"...a certainty of three sigma, so there’s about one chance in 100 the result
is a fluke."

This is a very incorrect way of summarizing the result.

When will popsci writers ever get this right? Hypothesis testing _cannot_ tell
you the probability that a hypothesis is true. It also cannot tell you if the
result is a "fluke." What it _can_ tell you is the probability of seeing the
data assuming the null hypothesis is true. Without knowledge of the prior
probability of the hypothesis, you have no right to say anything about the
probability of the result being a "fluke" or not.

This is commonly misunderstood. Perhaps I can illuminate it with an example.

Suppose I take a coin out of your pocket--a good old American quarter. I then
proceed to flip it six times and egads, I get six heads. The chance that a
fair coin flips the same way six times in a row is 2/(2^6), or 1/32--close to
3%. Whoo hoo, that's more than a two sigma result.

Does that mean there's only a 3% chance the coin is actually fair? NO! If you
_assume_ the coin is fair, there is a 3% chance the coin flips the same way
six times in a row. There's a huge distinction in flipping the conditional.
Going by the infinitesimal number of quarters in circulation that wouldn't
flip fairly, we'd say before any experiment that there's an overwhelming
chance the coin is truly fair. So there isn't a 3% chance the result "is a
fluke," if by "is a fluke" we mean the coin really is fair AND the coin
flipped in a surprising way. We KNOW the coin flipped in a surprising way, and
if we know that at least 99% of quarters are fair, there's probably a
near-100% chance that the data can be called a "fluke."

This is all to say, if the prior probability of your null hypothesis is high,
one unlikely result should not impress. You are merely demonstrating that
unlikely things happen--and with so many coins and people flipping them they
_should_ be, from time to time.

<http://en.wikipedia.org/wiki/P-value#Misunderstandings>

[http://invasiber.org/EGarcia/papers/MatthewsFinancialTimes.h...](http://invasiber.org/EGarcia/papers/MatthewsFinancialTimes.htm)

tldr; what the article says --> P(Fluke|Data) != P(Data|Fluke) <\-- what the
statistic means

~~~
cousin_it
> _What it can tell you is the probability of seeing the data assuming the
> null hypothesis is true._

Sigh. When will internet explainers ever get this right? :-) Your explanation
cannot be literally correct because, if the null hypothesis is true (the coin
is fair), the "probability of seeing the data" is exactly the same for all
possible sequences of six coinflips. What hypothesis testing actually does is
arbitrarily pick a class of data that's "at least as extreme" as what was
actually seen, and report its total probability.

To quote Eliezer Yudkowsky rephrasing Steven Goodman's example:

> _So lo and behold, I flip the coin six times, and I get the result TTTTTH.
> Is this result statistically significant, and if so, what is the p-value -
> that is, the probability of obtaining a result at least this extreme? Well,
> that depends. Was I planning to flip the coin six times, and count the
> number of tails? Or was I planning to flip the coin until it came up heads,
> and count the number of trials? In the first case, the probability of
> getting "five tails or more" from a fair coin is 11%, while in the second
> case, the probability of a fair coin requiring "at least five tails before
> seeing one heads" is 3%._

Also see the "voltmeter story":
[http://en.wikipedia.org/wiki/Likelihood_principle#The_voltme...](http://en.wikipedia.org/wiki/Likelihood_principle#The_voltmeter_story)
.

~~~
pak
> arbitrarily pick a class of data that's "at least as extreme" as what was
> actually seen

You are absolutely correct, in my haste to explain one misinterpretation of
p-values, I stumbled into another gross oversimplification. The correct way of
saying it must always involve some language about the deviation or extremity
of the data being at least as great as observed.

It goes to show that proper reporting of p-value statistics takes a lawyerly
craft with language; while writing the post I had to consider what the
mathematical interpretation of "is a fluke" should be. I think the definition
I chose is what most people will interpret it as. However, the word fluke just
means "unlikely chance occurrence" on its own, so saying "there is a 1% chance
the result is an unlikely chance occurrence" is uninformative if taken
literally. It has to imply one of two things:

1\. You are referring to the null hypothesis as the unlikely occurrence, i.e.
"There is a 1% chance the null hypothesis is true [therefore making this data
surprising]."

2\. You are temporarily _assuming_ it is true to make such a statement, i.e.
"There is a 1% chance of seeing data at least this surprising [if the null
hypothesis were true]."

Someone that does know what a p-value is might be generous and think you meant
the latter, which is a correct statement. However, I think that most people
hear it the first way.

The fact that p-value reporting (when properly done) involves thinking through
double negatives and conditions that are easily ignored probably indicates
that it's time for other measures of significance to become better accepted.

------
JonnieCache
For a writeup of this that includes some _gasp!_ physics, see the excellent
Résonaances blog [http://resonaances.blogspot.com/2011/11/lhcb-has-evidence-
of...](http://resonaances.blogspot.com/2011/11/lhcb-has-evidence-of-new-
physics-maybe.html)

For something slightly more comprehensible than that, but a lot more useful
than bloody _wired,_ try Sean Carroll:
[http://blogs.discovermagazine.com/cosmicvariance/2011/11/14/...](http://blogs.discovermagazine.com/cosmicvariance/2011/11/14/new-
physics-at-lhc-an-anomaly-in-cp-violation/)

For more LHC/higgs gossip see this recent article from Not Even Wrong
<http://www.math.columbia.edu/~woit/wordpress/?p=4136>

------
hardtke
Most important I even learned in physics graduate school was "physics research
has many more 3 sigma effects than it should." Part of it is that if you do
100 searches you are going to see one three sigma effect. Couple that with
reporting bias (only the interesting stuff is reported).

~~~
JoshTriplett
As far as I know, 3 sigma implies 1 in 370, not 1 in 100. (Still
insufficiently unlikely for a conclusion.)

<https://en.wikipedia.org/wiki/68-95-99.7_rule>

~~~
waqf
And it certainly does not imply "one chance in 100 the result is a fluke".
Will science journalists never learn how to interpret a _p_ -value into
English?

~~~
hessenwolf
I imagine about 200 years after scientists finally stop getting it wrong, so
almost surely never.

------
nagrom
It's maybe worth noting that the Nobel prize in 2008 was given to three
Japanese (two actual-Japanese and one Tokyo-born American) scientists for the
prediction and inclusion of CP-violating broken symmetries in QCD:
[http://www.nobelprize.org/nobel_prizes/physics/laureates/200...](http://www.nobelprize.org/nobel_prizes/physics/laureates/2008/)

------
cr4zy
This would actually solve one of Physics' most important unsolved mysteries
about the beginning of our Universe, Baryogenesis:
<http://en.wikipedia.org/wiki/Baryogenesis>. Less of a crack and more bridging
a huge gap in our understanding, I would say.

~~~
JonnieCache
[http://blogs.discovermagazine.com/cosmicvariance/2010/06/04/...](http://blogs.discovermagazine.com/cosmicvariance/2010/06/04/marketing-
cp-violation/)

------
andrewfelix
The article acknowledges _"...so there’s about one chance in 100 the result is
a fluke."_ and then this from a particle physicist... _"I’m not betting my
pension on this result standing the test of further data..."_

Why is this article getting so many up-votes?

~~~
sp332
This result points the direction for future research.

~~~
andrewfelix
> This result points the direction for future research. Thank you.

------
bad_user
Can somebody explain why this 0.8% difference between the decay rates matter?
I understand that this will give us a better understanding of what happened at
the big bang, but will this also translate into practical applications?

Not trying to criticize or diminish this finding btw, I'm just wondering if
these findings we get from LHC will translate into something to get excited
about in the near future - like, I don't know, alternative energy sources
maybe.

~~~
80hours
You mean something like the World Wide Web? ;-)

If you know the results and applications beforehand, it is more engineering
then research.

------
zalambar
"LHC does exactly what it was built to do".

~~~
ars
Except it didn't - yet. This is only a 3 sigma result which is so low this
should never have even been reported.

~~~
iand
3 sigma is significant enough to warrant external investigation. 5 sigma is
the realm of the interesting...

------
Karellen
What's that you say? A new study overturning old physics?

I bet $200 that the new result won't pan out.

------
andrewflnr
If they take the actual data from the Fermilab results, could they get a
quantitatively higher accuracy?

~~~
juiceandjuice
Yes, but not at the six sigma level which is needed (error propagation)

------
suivix
I fully expect them to destroy the Earth by the end of 2012.

------
Create
That bunch openly discriminates people according to origin.

"The cost [...] has been evaluated, taking into account realistic labor prices
in different countries. The total cost is X (with a western equivalent value
of Y)" [where Y>X] -- LHCb calorimeters : Technical Design Report

ISBN: 9290831693 <http://cdsweb.cern.ch/record/494264>

