

Ask HN: Please help with an AB test. (Live results in comments.) - notauser
http://theplanis.com/app/mst

======
notauser
My last day at my full time job was Tuesday, now I'm working on my own
projects. It's a lot of fun so far (except for testing in Internet Explorer -
this works in 8, haven't tried 7 yet).

I wanted to get something up quickly to test my tool chain/environments and I
thought that running a simple experiment was a great way to do that. I'm AB
testing two versions of requesting e-mail addresses from visitors and I'm
happy to share the results with HN.

I'll post a more detailed (anonymous) breakdown later on but you can see the
initial results here: <http://theplanis.com/app/abtestresults/>

~~~
ErrantX
just as important are users who DIDNT respond :) are you collecting that
information?

~~~
notauser
Good point, yes I am - It's not going into the live results but I'll put that
data up too at an appropriate point.

(For the interested the short link points to a tiny Django web app that's
doing both the tracking and handling AJAX e-mail uploads. Everything else is
client side JavaScript.)

------
jsm386
a/b testing is a great tool, but wouldn't telling people they are
participating in an a/b test trying to determine a specific question
'requesting e-mail addresses from visitors' bias the results?

~~~
notauser
I'm logging referrers so I can see how much of an effect that has - if it
turns out to be significant I'll segment the results.

~~~
jsm386
interesting..i await the results

~~~
notauser
After 20 hours:

310 unique visits / 22 addresses captured (7%)

50% of traffic from news.yc - and generally didn't leave an email address at
all.

Users from other sources were 10 times more likely to leave an addresses if
shown the pop up on page load.

