Hacker News new | comments | show | ask | jobs | submit login

"Stopping a test when you reach a "statistically significant" result is the wrong way to do A/B testing."

Nobody said that it was. But when you do regular split testing, you can use power analysis to estimate the length of time you need to run an experiment to get a significant result at a certain precision:


You can't do this (at least, not easily) when you're using bandit models, because none of the assumptions are valid.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact