
New paper: “Why most of psychology is statistically unfalsifiable” - yk
https://medium.com/@richarddmorey/new-paper-why-most-of-psychology-is-statistically-unfalsifiable-4c3b6126365a
======
StefanKarpinski
Short, good read. The money quote is near the end:

> We also suggest a way of powering future experiments: power your experiment
> such that you, or someone else, can conduct a similarly-sized experiment and
> have high power for detecting an interesting difference from your study. We
> need to stop thinking about studies as if they are one-offs, only to be
> interpreted once in light of the hypotheses of the original authors.

~~~
tpeo
Isn't this simply maximizing the probability of a successful reproduction?
Seems almost obvious given the whole problem was low reproducibility in the
first place.

~~~
SilasX
Sufficiently advanced reproducibility is indistinguishable from correctness.

~~~
TheOtherHobbes
Or systemic (as opposed to systematic) bias.

 _Independent, adversarial_ reproducibility implies some possibility of
correctness.

But psychology doesn't seem to do what science usually does - build models and
make independent predictions from them.

Finding patterns in data is not the same as testing a null hypothesis which is
the result of a model, and not just a random query. Or an accidental
discovery.

------
pmoriarty
Is whether some psychological are falsifiable matter? Why is a paper like this
one concerning itself with falsifiability?

These are questions the present paper does not concern itself with. It seems
to take for granted that falsifiability of a study is something important, but
never bothers to say why.

Falsifiability was popularized in the mid-20th century by the philosopher Karl
Popper, who proposed it as a criteria of measuring whether something is
"scientific" or not.

Unfortunately for his project, and perhaps for the authors of this paper, the
falsifiability criteria was largely rejected by later philosophers:

 _" Sir Karl Popper is not really a participant in the contemporary
professional philosophical dialogue; quite the contrary, he has ruined that
dialogue. If he is on the right track, then the majority of professional
philosophers the world over have wasted or are wasting their intellectual
careers. The gulf between Popper's way of doing philosophy and that of the
bulk of contemporary professional philosophers is as great as that between
astronomy and astrology."_[1]

More critiques here: [1]

I'm not sure if such critiques or philosophers of science other than Popper
are looked to by scientists, if scientists have some special reason for still
caring about falsifiability, or if they just latched on to the concept and now
either don't know or don't care what philosophers of science think about the
demarcation between science and non-science.

One other thing that goes unremarked on by this paper is why whether a study
or branch of psychology is considered scientific or not matters. There is in
fact sometimes a lot of money involved, and certainly careers to make or
break. Scientists and psychologists both have a vested interest in making
their way of doing science or practising psychology the most highly respected,
desirable, and valuable way.

In the current money-conscious climate, insurance companies are often keen to
pick the shortest possible (ie. cheapest) and most "evidence-based" (or most
"scientific") treatment, and reject the rest. This way, they can reject as
much as possible, and pay as little as possible for treatment. Consumers,
usually knowing little about psychology or the battles that have happened
between branches of psychology often pick the most "scientific" sounding
treatment (if they have, or one that their insurance company covers, if they
don't -- and they often don't). So that's where the money goes.

[1] -
[https://en.wikipedia.org/wiki/Falsifiability#Criticisms](https://en.wikipedia.org/wiki/Falsifiability#Criticisms)

~~~
WalterSear
It's not necessarily that falsifiability is important, but that these studies
claim it to be so, and yet aren't.

~~~
kem
It seems to me the authors of the original paper are confused about, or are
confusing the meaning of "falsifiability" as it's usually meant in philosophy
of science.

It was an interesting paper, if not entirely original, but I wouldn't say it's
about falsifiability. It's about power. There's some overlapping issues, sure,
but they're not the same concepts.

