
Manipulating the Alpha Level Cannot Cure Significance Testing - gattilorenz
https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00699/full
======
bcoughlan
We rely on science as presented by the media to inform us about everything
from what we should eat to how our surroundings influence our behaviour. It is
a pervasive cargo cult - no article is considered credible without vague
cherry picked references. The whole pop-science/self-help book industry must
be worth billions.

Yet we hear about p-hacking and replication crises and seem to adopt an
attitude of sweeping inconveniences under the rug.

I don't know enough stats to fully understand the article. Can anyone explain
why we should not consider the claims of a huge body of scientific work that
have relied on these statistical methods for over 50 years to be invalid?

~~~
aoki
NHST was always intended to provide evidence within a corpus of replicated
research. institutionalized careerism creates disincentives for science to
work that way in practice.

it's actually consistent with the NHST paradigm to treat any given result with
skepticism; the degree of skepticism depends on many things, some of which are
statistical and some of which are domain-specific. but skepticism is not the
same as assuming invalidity. imperfect experiments are still (expensive)
evidence for something.

imo, what you should reject entirely are media reports (usually based on
somebody's PR team) that try to tell you to do anything based on one study or
the work of a single lab.

