

Journal rejects studies contradicting precognition - wallflower
http://www.newscientist.com/article/dn20447-journal-rejects-studies-contradicting-precognition.html

======
tokenadult
Richard Wiseman, interviewed for the submitted article, is a psychologist who
studies human cognitive biases. His concern regarding Bem's purported studies
on precognition is that if journals reject studies showing no effect, while
accepting flawed studies appearing to show precognition, this results in a
file drawer problem

[http://en.wikipedia.org/wiki/Publication_bias#The_file_drawe...](http://en.wikipedia.org/wiki/Publication_bias#The_file_drawer_effect)

in which publication bias among the published studies will then skew any
subsequent attempt at meta-analysis of the published studies. The file drawer
effect is of particular concern in meta-analysis of purported extrasensory
perception.

<http://www.skepdic.com/filedrawer.html>

By the way, Wiseman is perhaps best known online as the author of the Colour-
changing Card Trick website, featuring videos of an experiment that
demonstrates selective attention in human perception.

[http://www.quirkology.com/USA/Video_ColourChangingTrick.shtm...](http://www.quirkology.com/USA/Video_ColourChangingTrick.shtml)

If you haven't seen the video before, it is well worth watching.

I hope the more responsible journals of psychology will step up their efforts
to publish commentaries on the study design

<http://norvig.com/experiment-design.html>

of ESP studies and publish in general more studies showing failures of
replication of earlier study findings.

------
sigzero
Did they reject the studies BEFORE they were submitted or AFTER?

------
scott_s
As a response to the original study, please refer to "Why Psychologists Must
Change the Way They Analyze Their Data: The Case of Psi":
[http://commonsenseatheism.com/wp-
content/uploads/2010/11/Wag...](http://commonsenseatheism.com/wp-
content/uploads/2010/11/Wagenmakers-Why-Psychologists-Must-Change-the-Way-
They-Analyze-Their-Data.pdf)

The main point of it is that the parapsychologists have done nothing wrong:
they're following the same methods and using the same techniques as the actual
psychologists. The real problem is that those methods and techniques are not
good enough.

Also related, "Parapsychology: The Control Group for Science":
[http://lesswrong.com/lw/1ib/parapsychology_the_control_group...](http://lesswrong.com/lw/1ib/parapsychology_the_control_group_for_science/)

~~~
naveensundar
I don't know how the lesswrong people came to that conclusion about
parapsychology a-priori. This is a problem in Science in general. People use
parametric-value-based reasoning (e.g. significance values) to get binary
conclusions (existence or not of physical processes etc).

<http://dilbert.com/fast/1998-01-19/>

------
schrototo
The TL;DR version: "This journal does not publish replication studies, whether
successful or unsuccessful"

~~~
faboo
Which, to me, seems to be an admission that the Journal of Personality and
Social Psychology isn't actually a scientific journal. An initial
experiment/study is only a part of the process. If you get extraordinary
results out of an experiment, but no one else sees the same results with the
same experiment, well, something was up with your run of it. It is vital to
the body of scientific knowledge that re-runs of published experiments are
performed, recorded, and published. Otherwise, there's no way to know whether
the original result was real or a fluke (or fraudulent).

~~~
mcburton
When dealing with failed replicability it is important not to conflate a
poorly designed experiment with a poorly written methods section. Often there
is tacit or socially negotiated knowledge required to properly replicate an
experiment. This doesn't mean the science is bad, it means science is more
social than some would like to admit. For some interesting reading on this
check out Harry Collins's paper "The Seven Sexes: A Study in the Sociology of
a Phenomenon, or the Replication of Experiments in Physics"[1]

[1] <http://soc.sagepub.com/content/9/2/205.short>

------
Semiapies
I'm curious how this will play out in the scientific world - if there's a slew
of failed attempts to reproduce the findings, this will be the new cold
fusion, and such an embarrassment may actually affect the way papers get
published.

In the layman world, of course, the original paper will be used as
justification for whole shelves of pseudoscience books, and the public won't
actually hear about any failed attempts to reproduce the claims.

------
nickff
This is exactly what Richard Feynman described in his 1974 commencement
address at Caltech.

<http://www.lhup.edu/~DSIMANEK/cargocul.htm>

