
Debunking the Stanford Prison Experiment [pdf] - ascertain
https://psyarxiv.com/mjhnp/
======
zawerf
Can someone explain to me what researchers considers "scientific validity" in
psychology?

I feel like human behavior will change as culture evolves. For example how
people react to a plane hijacking drastically shifted after 9/11\. In a
similar way the Stanford Prison Experiment probably permanently changed a part
of human culture so that it will no longer be replicable (regardless of
whether the original setup and results were valid or not).

How do you conduct science when whatever important truth about human behavior
you find invalidates the same findings once that knowledge enters popular
culture?

~~~
abathur
I think this is exacerbated by a dual blind-spot in both our thinking and
languages regarding the difference between "how something was the last time we
checked" and "how something intrinsically/durably/permanently is and will be
no matter how many times we check". It probably isn't something we've needed
in large quantities for long.

Your question focuses on psychology (and I agree it's important here), but
this probably happens to greater and lesser degrees when studying _any_
complex adaptive system.

Say you go study the carbon-uptake potential of a few species of tree, model
how planting more trees could impact the climate, and click the publish
button.

Your research may already be out of date. Your ability to completely control
variables in a complex system has limits. Most of the environmental
measurements that went into your model are moving targets. Even if your model
is predictive at publication, there's a good chance there are unmodeled
dependent variables lurking.

What if your research captures the zeitgeist? What if you inspire the planting
of a hundred-billion trees? Do your measurements consider lone trees? Does the
density and surface area forested (and eventually the weather-shifts a large
forest triggers) impact how it grows and functions as a carbon sink? What if
your research inspires someone to plant a massive monoculture that ultimately
incubates and spreads a parasite that ultimately destroys the species?

~~~
abathur
To unpack a smidge, I think the first N steps all involve developing the
perception and language to spot, tag, and catalog instances of this problem.

There's a fundamental shortfall in the quality of our thinking relative to the
task (and the ultimate answer may just be that we're too time-bound to think
this way). It's in the study designs, papers, abstracts, and unavoidable in
the popular press.

------
dang
Related from last year:

[https://news.ycombinator.com/item?id=17796953](https://news.ycombinator.com/item?id=17796953)

[https://news.ycombinator.com/item?id=17387601](https://news.ycombinator.com/item?id=17387601)

[https://news.ycombinator.com/item?id=17287319](https://news.ycombinator.com/item?id=17287319)

------
gwern
Published version:
[https://www.gwern.net/docs/psychology/2019-letexier.pdf](https://www.gwern.net/docs/psychology/2019-letexier.pdf)

