
Surprising Studies Are Usually Wrong - williamkuszmaul
https://algorithmsoup.wordpress.com/2018/09/20/surprising-studies-are-usually-wrong/
======
pknight
The first example is about "power posing", it's suggested that this has been
"debunked". This has become a controversial topic for some reason (the idea
that posture influences mood and mindset is not particularly surprising, but
ok) but the episode has less to do about a surprising study being wrong and
more to do with people getting carried away with findings.

It's interesting that we treat the debunking treatment with the same appetite
for salaciousness though. Very little actual skepticism is reserved for those
doing the debunking, because just as people yearn for positive results, they
also like to see people being taken down. In Amy Cuddy's case it is
particularly strange how personal people got who have attacked her and her
line of research became and how eager people are to use the pseudoscience
label despite the fact that this is serious research striving to understand a
topic that isn't even that controversial to start with. Where does all that
passion come from?

In any case, when follow-up studies fail to reproduce earlier findings or when
other authors spot issues with the experimental methods used, this is just
science working properly. Unsuccessful studies in the wake of a surprising
study don't immediately indicate that the earlier study was a failure and to
suggest so is to do the scientific method is disservice. A good line of
research is going to produce successful and unsuccessful studies, calls for
moderate interpretation of findings should go both ways and with respect to
how we develop an understanding in the first place.

And for what it is worth, a further analysis of 55 studies from 2017 does show
that the line of research has merit. See:

P-Curving a More Comprehensive Body of Research on Postural Feedback Reveals
Clear Evidential Value for Power-Posing Effects: Reply to Simmons and
Simonsohn (2017)

