

Ask HN: High traffic sites use A/B testing. What can low traffic sites use? - briggers

Stated another way: why wait for your first few thousand visitors before making your first data-driven decision?<p>Is this a problem that needs solving?<p>[Edit: formatting]
======
_delirium
It would be possible to at least improve the situation. An approach common in
machine learning is, before a lot of directly-on-point data comes in, to
bootstrap predictions using a background dataset of existing data, which can
at least give you hints about the structure of the space, and where in it you
might fall.

In this case, if you (or some service) had a large data set of user
interactions collected from many other websites, then even though the first N
data points on your site might not be enough to make strong conclusions on
their own, they could be used to match your website up to scenarios in the
background dataset that are most similar. There are a lot of different
approaches, ranging from Bayesian ones where the background dataset is used as
a prior in one way or another, to more clustering-based ones, where your small
initial dataset is used to match you to a set of "similar, as far as user-
experience lessons go" websites in the existing dataset.

------
coopertmynas
You can't make a data based decision without data.

What you can do is test on an individual basis. Companies have been doing this
long before computers came along. Look up videos by Steve Krug on how to
properly run user testing.

There are a lot of tools geared toward this already. Some offer testers,
others just give you the capability to test with your own anonymous users. My
rule of thumb is if someone else is making money off it, there is still money
to be made.

This kind of testing doesn't identify things like a red button converting
better than a green button, but it can tell you if some users aren't seeing
the button at all.

~~~
briggers
Right. Unless I've misunderstood you, this is the angle I am investigating
with www.uithing.com

Thanks for the tip re: Steve Krug's videos.

------
Travis
In my most recent experience, I had so many different tasks to do that were
more important than optimization. By the time I had the bare essential stuff
done, I had enough data to start to make decisions on.

------
carbocation
You need data to make a data-driven decision. Can you elaborate on how you
propose to solve the problem of "making a data-driven decision without
sufficient data"? I'm pretty skeptical, as you can see.

~~~
briggers
In site usability, ecommerce, and conversion in general speed is well
established as being very important. Google and others have made speed a core
goal because their data shows that speed helps conversion.

However, I would suggest that site speed is merely a proxy for usability
speed. ie. it's more important for conversion to have a form that is easily
comprehended and filled out than it is to have a fast loading page.

What I'm working up to is the idea that usability speed might be a useful
proxy for a high-conversion design. It's also data that can be measured
accurately without bias, and can be generated in lieu of real visitors by
throwing a horde of disinterested people from Mechanical Turk at (eg.) your
sign-up page with a well-defined goal.

------
pkamb
Write down the top 5 tasks people will be doing on your site.

Get 3 people, tell them to attempt to complete those tasks.

Watch.

