

Statistics: How Many Would You Check? - cawel
http://blog.jpalardy.com/posts/statistics-how-many-would-you-check/

======
cozzyd
I've always used a Clopper-Pearson interval for binomial confidence intervals,
but never questioned why (it's the default in ROOT). I found
[http://en.wikipedia.org/wiki/Binomial_proportion_confidence_...](http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval)
quite useful. Sounds like using Clopper-Pearson is safer than using a Wilson
interval .

~~~
louden
It's a trade off. Clopper-Pearson can be overly conservative in many
instances. I tend to use Jeffreys interval personally, which is a Bayesian
method.

Brown et al. give a formal treatment of the subject with simulations showing
actual coverage probabilities for the various methods. It is worth a read if
you want to really dive into the subject.

ref: Brown, Lawrence D.; Cai, T. Tony; DasGupta, Anirban (2001). "Interval
Estimation for a Binomial Proportion". Statistical Science 16 (2): 101–133

~~~
cozzyd
Thanks. To save others the trouble, a (apparently accessible to all?) link to
that article is at
[http://projecteuclid.org/download/pdf_1/euclid.ss/1009213286](http://projecteuclid.org/download/pdf_1/euclid.ss/1009213286)

It looks like Clopper-Pearson is the only method guaranteed to not
underestimate the size of the confidence interval (which makes sense, given
its derivation), but it almost always overestimate its (unless your confidence
interval exactly matches the discrete p-values allowed by the binomial
distribution).

------
hervature
To be a pedant and assuming your confidence interval is at the 95% confidence
level. Taking 73 checks will produce the 95% lower bound 95% of the time if
you take any random 73 samples.

~~~
jpalardy
Right, in this specific case it's awkward to mention both 95% (confidence) and
95% (lower bound) in the same sentence.

It could have been clearer if I used different values.

