

Ask HN: how to iterate or A/B test while you have small conversion rates? - thenomad

So, I have a little side project that's showing some promise and occasionally converting.I want to get down with the A/B testing, of course, because it's been very useful in other projects.<p>Problem is, the conversions to sales don't occur often enough to be statistically useful - with one conversion per day on a very good week, from what I understand of A/B testing I'd need to run for about a month to get meaningful statistics. Not exactly rapid iteration!<p>So, how can I iterate in this case? It's a single-page sales letter with a "buy" link at the bottom - I guess I could try to capture email addresses initially or even split the pages up, but I'm not sure how statistically relevant the results would be, or if changing my sales model like that would actually reduce conversions!<p>I'm guessing I'm not the only guy in this position. Any ideas?
======
pdenya
I'm in the same boat with my site. For now I've given up on A/B testing until
I can at least double my traffic. This SEO service seems fairly cheap and came
highly recommended by @aymeric: [http://taskarmy.com/services/342-get-your-
webpage-on-1st-pag...](http://taskarmy.com/services/342-get-your-webpage-
on-1st-page-of-google)

------
asharp
Define a series of heuristics.

For example, lets say you have one heuristic that is the number of page views
on a particular page, then the number of people who fill out a particular
form, then the number of people who.... (Basically various things that should
correlate with sales).

The easy way is to then go through and first off measure the correlation
between each heuristic and sales and then define some facsimile for the change
in sales through something like the sum for all h(corr(h, s)*delta h) with h
being the heuristic and s being sales.

You can then use that as a new test heuristic for A/B testing.

~~~
thenomad
Sadly it's not a "read, register, play around, 30 days run out, pay" kind of
deal.

The useful events are pretty much -

1) Arrival on page 2) Read down page 3) Click buy button 4) Pay.

That's what's doing my head - too few things to measure!

I'm debating artificially adding more steps in the chain, but I'm far from
certain about that.

~~~
asharp
Yeah, however you know more then that per user.

ie. how long the user is on the page, how many pages they view, where they
came from (did they come from google/news.yc/etc?)

If you're still stuck, send me an email! (You have no email on your profile
page, btw)

~~~
thenomad
That's wierd - it's listed there...

Thanks - I may just do that. Appreciated.

~~~
asharp
It's listed but it's only used for internal operations (iirc password
resets/etc.)

Only the user info section of your profile gets seen by others.

------
cschmidt
Try to move up the conversion funnel if you can. For example, track the number
of clicks on your buy link, as opposed to the total number of conversions.
That will give you bigger numbers to A/B.

