

Psychologists Welcome Analysis Casting Doubt on Studies - evolve2k
http://www.nytimes.com/2015/08/29/science/psychologists-welcome-analysis-casting-doubt-on-their-work.html

======
bobcostas55
How much of this is the result of outright fraud? I know Hanlon's razor and
all, but I find it extremely difficult to believe that virtually everybody in
this field is incompetent.

It seems like the majority of these highly intelligent people, despite
spending a decade in training, were essentially statistically illiterate, even
though statistics are absolutely central to their field. That's not a
catastrophe in itself, you can always hit up the "real" statistician two
floors down...but it seems like they didn't want to bother with that. Did they
overestimate their ability or did they know they were serving up shitty
research?

The papers that do replicate are tightly clustered a small number of
researchers (Kahneman & Tversky)...I'm not sure how to interpret this. Did
they get lucky and stumble upon the one area of psychological research that
had a "high yield"? Did they implement a superior research methodology that
nobody could/wanted to imitate?

~~~
DanBC
Have a look at this what this Harvard neuroscientist says about failed
replication. It's pretty scary.

[http://web.archive.org/web/20150115080542/http://wjh.harvard...](http://web.archive.org/web/20150115080542/http://wjh.harvard.edu/~jmitchel/writing/failed_science.htm)

~~~
bobcostas55
Well that makes me think the fraud angle is fairly likely...

~~~
DanBC
It made me think of incompetence, but I guess fraud fits too.

The rejection of scientific method was what scared me.

------
evolve2k
"One piece of evidence that preregistration can act as a strong corrective
comes from clinicaltrials.gov, a registry of publicly and privately funded
clinical studies involving human subjects. Before 2000, when the site was
established, 57 percent of large clinical trials funded by the National Heart,
Lung and Blood Institute showed a significant benefit of drugs or other
intervention, according to a recent analysis published in the journal PLOS
One. After the registry was put in place, only 2 percent of such trials found
a clear benefit."

~~~
jMyles
I think it's healthy to suspect that the entire pharmacopoeia-industrial-
complex has a similar rate of veracity.

~~~
ArekDymalski
I'm absolutely sure about that. As a student I was working (as a technician)
on many medical science conferences, so I had an opportunity to see the
"bleeding edge" research presentations. I was shocked by the level of
confidence demonstrated by "scientists" who promoted results of their N=5
experiments (without a comparison group) as "convincing". If all therapies are
based on such research, we have to be happy that our bodies are impressively
self-repairing machines.

------
dominotw
At the risk of sounding really stupid, What is the crowning achievement of
psychology ? What gives this field the legitimacy that it enjoys ?

~~~
SolaceQuantum
Well, we we now understand a lot more about how people think than before.
We're finally past that id-ego-superego and black box theory bullshit
(although some people still ascribe to freudian ideas). There is real
treatment for people with mental illnesses other than hospitalization and
drugs (e.g. people with personality disorders, dissociation disorders, etc.
can now receive structured talk therapies)

The crowning achievement would probably be the ability for people who are
mentally ill to receive help and understanding from trained professionals
rather than be considered possessed, needing drugs, shoved into insane
asylums, or eugenics'd.

