
Would you prefer three N=300 studies or one N=900 study? - sndean
http://andrewgelman.com/2017/04/23/prefer-three-n300-studies-one-n900-study/
======
MrTonyD
I was lucky to have a University class where we studied studies - constructing
them, running them, and evaluating them. We went through many historically
important studies to understand them better. Our conclusion? Lots of studies
are very flawed and lead society in the wrong direction for decades - even if
double-blind and statistically sound.

When we tried to reformulate the studies so they would have been correct, the
most common problem was that it wasn't possible to reformulate them at the
time they were designed - what to measure wasn't known, or there was no way to
measure it, or inputs or outputs were not known to exist or known to be
measurable.

Really, I wish everybody had taken that class. I hear all the "fact-based"
exhortations by people who don't really understand that they are trading one
religion for another (but, hey, I believe in taking the best judgment at the
time, based on the best data - just understand it's limitations and take those
into account.)

