Hacker News new | comments | show | ask | jobs | submit login

"A/B testing is meant for strict experiments where focus is on statistical significance, whereas multi-armed bandit algorithms are meant for continuous optimization where focus is on maintaining higher average conversion rate."

Who cares about conversion rates, right? I'd much rather have statistical significance than more clicks on my buy button / signups for my site / etc.

The point is that with less significance you are less confidential what your conversion rate "would have been" under other courses of action. You can't say "this or that test has higher conversion" because of some simulation that has totally different structure than your real website.

Statistical about managing unknowns, not creating knowns, and everyone ignores this!

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact