

Ask HN: A/B testing with only 1-2 sign-ups per month - timmorgan

I would like to employ A/B testing for my web service homepage and sign-up. My site only gets 1-2 sign-ups per month.<p>How long would I need to run the experiment to get a reasonably decent idea of which version is statistically better? (I have no statistical background/training.)<p>The wikipedia page (http://en.wikipedia.org/wiki/A/B_Testing) is very light on details.<p>Any help would be greatly appreciated.
======
nate
Here's a great post on doing the math for this stuff

[http://blog.asmartbear.com/easy-statistics-for-adwords-ab-
te...](http://blog.asmartbear.com/easy-statistics-for-adwords-ab-testing-and-
hamsters.html)

But yes, the bad news is it's going to take too long at your rate right now. I
recommend looking at things like <http://conceptfeedback.com> to get feedback
in lieu of being able to test right now. You could also consider running a
poll with Mechanical Turk workers. The turkers would answer a poll like which
design is better for about a $0.05 a person.

I've done the conceptfeedback.com for testing some new elements in
<http://tgethr.com> before we had the traffic to start experimenting and the
feedback was great.

~~~
timmorgan
Wonderful advice. Thank you.

------
Alex3917
Based on your current traffic? About 55 years.

~~~
timmorgan
So you're telling me there's a chance... :-)

