

Statistical Advice for A/B Testing - sl8r
http://sl8r000.github.io/ab_testing_statistics/

======
Machow
Sorry if this sounds condescending. I'm asking out of total naivety. What is
the background of people who typically run A/B tests? Are they often
unfamiliar with things like multiple comparisons, stopping problems, and the
like?

~~~
DougN7
I can answer from personal experience. I'm a dev at a tiny startup. I took
statistics at college 20 years ago but haven't used it since the final exam,
and remember little other than sample size is critical. I've figured out
through recent experience that you need to let a test run a while - results
seem to flip flop for the first few thousand visitors. I use tools like VWO
because it makes it simple to run a simple one variable test, and I'm happy
with the results (though ignorance is bliss, so maybe others would not be so
content).

