

Ask HN: How to split traffic in A/B testing? - shadowz

I've been reading up a lot on A/B testing lately on HN. Since sequential A/B testing is not recommended due to external factors, I'm just curious as to how someone is suppose to split web traffic (say 50/50 without using hardware to balance).
======
patio11
The way A/Bingo handles this is to assign every user a unique ID, and for each
test take the MD5 hash of their ID concatenated with the test name. That gives
you some big ol' integer. Modulo the number of choices gives you the number of
the choice to show them. This is extremely fast, durable, and has very little
state, which are nice things to have in an A/B testing algorithm.

~~~
euroclydon
So that means, that all you need is their UID, and for each subsequent page
hit, for a particular test, the test choice to show them is deterministic.
But, where does the even distribution of test choices come from? The hash or
UID or the combo of the two?

~~~
patio11
MD5 hashes mod a small number can be assumed to be randomly distributed.
Hypothetically assuming I had evidence to the contrary, I would shortly be
very famous in some circles. Practical results bears the expectation out.

------
aymeric
Which programming language do you use?

Maybe someone has already built a framework for you to use?

I personally used ABingo for Ruby on Rails
(<http://www.bingocardcreator.com/abingo>), Visual Website Optimizer
(<http://visualwebsiteoptimizer.com>) and Google Website Optimizer
(<http://www.google.com/websiteoptimizer>) for any technology.

The advantage of running a technology specific tool is mainly because of
performance. Most technology-agnostic tools execute a javascript on your page
to be able to execute the A/B testing logic where ABingo for example is
executed on the server side (no additional http request)

What generic tools give you though is the ability to change your tests without
changing your code.

