
Why, if p=.04, we can't say the study has a 4% chance of being false positive - Tomte
https://twitter.com/methodsmanmd/status/997482408973922305
======
hprotagonist
[https://threadreaderapp.com/thread/997482408973922305.html](https://threadreaderapp.com/thread/997482408973922305.html)

for increased readability.

------
zawerf
I don't know any stats. I want to understand how to interpret p-values and it
seems like according to this post I also need to know how to estimate the
"Proportion of True Hypotheses"?

He starts off with "Imagine a world of scientific hypotheses" but isn't that
world infinite? I could ask whether 1 is a number, 2 is a number, 3.14 is a
chicken, etc. Aside: I would argue that the cardinality of those infinites are
actually the same since you can form a bijection between true and false
hypotheses just by negating the statement (1 is a giraffe, 1 is not a
giraffe).

So I assume he means the finite world of all hypothesis that scientists(which
are finite since the universe will eventually end) would consider interesting
enough to test? Even then it doesn't seem narrow enough. The pool of
hypotheses has a shifting distribution based on time period (e.g. 1800s vs
modern science). And you can further slice them down field, journal,
affiliation, exact subject, etc.

If that's being pedantic, ignore defining what the world of hypotheses looks
like for now. Even given a fixed world, how can you estimate the true number
of "True Hypotheses"? You only really see the end result of your study
(positive or negative) and you probably don't even know about the existence of
most of the negative studies.

In short, still super confused! Links to intro to p-values/stats appreciated.

~~~
tscs37
From what I understood in my recent statistics course, the p-value is the
probability of receiving a specific result considering the null hypothesis is
true. It's usually compared by using some distribution model (binomial, etc.)
over the actual study results.

Short version; the p-value is only sufficient to disprove a hypothesis, not to
prove it. If your p-value is 1% than that means you make experiments until the
probability that you are wrong is less than 1%. (for example, you throw coins
100 times, 90 times is head, the probability of that result is below 0.04 so
the coin is not fair)

That still doesn't mean you are right. Just that at the moment you have not
been prove wrong or the data is not sufficient to prove you wrong.

