I use A/Bingo [1] on CareLogger. Although at work (Learnhub.com) we use Vanity [2].
I found that Vanity splits the participants more equally. I noticed that with A/Bingo one would have 50 more trials than the other ones. Not a big deal unless you check the dashboard constantly.
They both do a simple task well so either will work fine.
Regarding the sustainability, this is something I've noticed as well. The conversion rates fluctuates heavily depending on the days of the week (middle of the week is best). It swings back and forth but 25% is the new middle ground and not just the good days as it was before.
Both A/Bingo and Vanity split participants in essentially the same fashion: each new participant is assigned totally randomly. (Not only is it the same effect, the algorithm we use for it is practically identical, too.)
This tends to produce a phenomenon well-known to coin flippers: the more coins you flip, the closer the percentage of heads and tails will converge to 50/50 and the farther your counts of heads and tails will diverge from each other.
I figured as much. It's likely that I noticed it because in the environment I use Vanity theres about much more traffic than on carelogger so the numbers even out much more quickly.
I found that Vanity splits the participants more equally. I noticed that with A/Bingo one would have 50 more trials than the other ones. Not a big deal unless you check the dashboard constantly.
They both do a simple task well so either will work fine.
Regarding the sustainability, this is something I've noticed as well. The conversion rates fluctuates heavily depending on the days of the week (middle of the week is best). It swings back and forth but 25% is the new middle ground and not just the good days as it was before.
[1] http://www.bingocardcreator.com/abingo/
[2] http://vanity.labnotes.org/ab_testing.html