
The scientific study has become a flawed manual for living - tokenadult
http://www.theguardian.com/commentisfree/2014/mar/12/scientific-study-flawed-manual-women-heart-disease
======
lutusp
A quote: "So while it's interesting to hear that women are less likely to die
of heart disease if they have a partner, no one's going to go out to find one
in order to have a marginally better chance of making it to old age."

The above sentence shows that the author -- and most readers -- miss the
single most important thing about cohort studies -- which is that they can't
distinguish causes from effects. Contrary to everyday understanding, the
statement "women are less likely to die of heart disease if they have a
partner" _does not mean_ that choosing to have a partner _results in a
reduction_ of a woman's chance to die of heart disease, and it _does not mean_
that seeking out a partner _will reduce_ one's changes to die of heart
disease. It only means that having a partner, and reduced chance of dying of
heart disease, are _correlated_.

The above very common confusion is the source of any number of meaningless
"breakthroughs" in science publishing, and fuels any number of public
controversies with little or no scientific substance.

~~~
dllthomas
It _does_ mean that the particular causal link in question is marginally more
likely than if the correlation had not been observed. Not enough to base
action on, though.

It certainly implies (how strongly depends on the actual data) that there is
_some_ causal link (A -> B, B -> A, or A <\- C -> B), because something is
reducing the entropy between the two systems. That again, though, doesn't
really tell us enough to be useful.

~~~
Retric
It can be a lot more complex than that.

Suppose healthy people live longer and are more attractive and Attractive
people are more likely to be married. So A <\- D <\- C -> B.

Sometimes A between .5 and .8 -> B but A less than .5 or more than .8 -> !B.
So now you can have study's that show more A -> B and others that less A -> B.
(Think nutrition)

~~~
dllthomas
My arrows weren't meant to be _direct_ causal links.

It's still true that there are potential situations I didn't touch (A <\- B
_and_ B -> A, &c), but those can be reduced to multiple instances of the
above.

Edited to add:

 _" Sometimes A between .5 and .8 -> B but A less than .5 or more than .8 ->
!B. So now you can have study's that show more A -> B and others that less A
-> B. (Think nutrition)"_

My arrows also weren't meant to be implication, just a causal relationship. A
-> B and !A -> B would be the same thing.

I should probably have been more explicit about all these points.

~~~
lutusp
> My arrows also weren't meant to be implication, just a causal relationship.

But in the typical case of a measured correlation, no causal relationship can
be implied based on the data alone, regardless of the direction. This is
especially true in examples of data mining, where many correlations are
located that arise purely from chance rather than design.

~~~
dllthomas
No _particular_ causal relationship, right. Existence of _a_ causal
relationship can be.

~~~
Retric
He is referring to the issue where data-mining creates a lot of meaningless
correlations. [http://xkcd.com/882/](http://xkcd.com/882/) aka _many
correlations are located that arise purely from chance rather than design_

A _single_ study is generally close to worthless.

~~~
dllthomas
Yes, the correlation has to be sufficiently unlikely to have arisen by chance.
Even in that case, though, our estimate of green jellybeans having some causal
relationship with acne _should_ increase.

------
nazgulnarsil
We need services that curate how much you should actually pay attention to a
given study. Science journalists have proven themselves inadequate to the task
because they are after eyeballs. Large, well designed studies, with applicable
samples and large effect sizes? Yes. Small studies with poor design on rats,
or elderly Finnish twins with diabetes? No.

Why can't I find a list of some of the largest and most robust systematic
reviews and meta anlyses ever done?

~~~
biofox
"Why can't I find a list of some of the largest and most robust systematic
reviews and meta anlyses ever done?"

The Cochrane Collaboration is the closest thing I know of:

[http://www.cochrane.org/](http://www.cochrane.org/)
[http://en.wikipedia.org/wiki/Cochrane_Collaboration](http://en.wikipedia.org/wiki/Cochrane_Collaboration)

~~~
nazgulnarsil
Yes, and I use them extensively. But they do not publish a guide to the
studies they think are most worth paying attention to due to their quality and
size.

------
jcromartie
It's rational to be irrational, because being irrational feels good, which is
what makes life worth living. At least this is true for the average person.
There are individuals who get a lot out of a more methodical evidence-driven
life.

~~~
wwweston
I wonder how many such individuals there are -- maybe we could find out what
the correlation is between methodical evidence-driven people and people who
feel good!

------
squozzer
Maybe worse, studies have become a bludgeon to justify government policy.

------
0xdeadbeefbabe
The _misunderstood_ scientific study _is still_ a flawed manual for living.

------
hkon
... And discussion on the internet.

