Hacker News new | past | comments | ask | show | jobs | submit login

Statistical significance is almost meaningless in this case (or necessesary, but not sufficient). One can calculate the necessary sample size to guarantee stasticial significance before the study even starts.

I just calculated it for this study: With an estimated small effect size (0.1), 95% confidence (α=0,05) und 1 degree of freedom you have guaranteed significance at n=197. Phase 2 of the study at hand had n=167, so almost guaranteed significance from the sample size alone.

The problem is that the confidence intervals are huge and their range covers the negative space, too. In other words: It is quite possible that a reproduction of the study would show negative results.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact