Hacker News new | comments | show | ask | jobs | submit login

A/B testing is linear regression against a single binary variable. There are many better techniques. Multi-armed bandit is just one of them. A more direct comparison would be simply using a continuous independent variable, or at least one with more than two values. You can have more variables pretty easily, too.

Of course you get statistical significance with fewer observations if you have fewer variables to model. Although you'd get even significance with even fewer observations if you had a continuous independent variable. But more variables produces a higher r-squared. As in, you can explain more of the variation in your conversion rate.

The prevalence of A/B testing demonstrates a sad lack of statistics training. It's a problem in many science fields as well, especially biology.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: