
Announcing ABTests.com - prakash
http://davidcancel.com/announcing-abtests-com/?awesm=1RAO&utm_campaign=twitterfeed&utm_medium=awe.sm-twitter&utm_source=brizzly.com&utm_content=twitterfeed
======
tdavis
WARNING: Nothing to see here but a vague description and newsletter sign-up...

------
paraschopra
Much better resources:

WhichTestWon - <http://whichtestwon.com/> Testing Thursdays -
<http://testingthursdays.com/>

------
holdenk
I wonder if its any different from <http://www.google.com/websiteoptimizer> .
Hopefully someone posts once theres something there.

~~~
jdagostino
Yeah I agree - What are they planning to offer and charge for that Google
website optimizer doesn't already offer for free?

What is interesting is that it from the guys behind Crazy Egg - Which is a
_simple_ but great tool for click tracking

~~~
patio11
Google Website Optimizer is the costliest version of "free" you'll ever
experience. I'd recommend it over not doing A/B testing at all (well,
probably), or for someone who wanted to dip their toe in the water with a
personal blog or something and didn't want to write code.

But for business owners here? Who are generally fairly proficient with coding?
No, no, just no. Friends don't let friends use Google Website Optimizer. There
is almost certainly a better option available for your platform of choice. Use
it. If not, _write it_ , it will pay for itself at any sort of scale. (I
should know, I had to do it myself.)

GWO is optimized to ease deployment by non-technical marketing folks. It makes
significant compromises in user experience to make that possible. In
particular, using Javascript for the basic two-alternatives-on-two-urls A/B
testing will cost you an extra page load. If its on your personal blog, OK. If
not, that just isn't acceptable. Everybody who looks into the effect of page
load times on dollars-in-your-bank-account finds the effect to be
_staggering_. Google and Amazon famously have published studies showing
measurable sensitivity to differences of 100ms. Causing an extra page load is
just about the worst thing you can do for page load speed. It is so bad that
Google would swallow broken glass before ever using Website Optimizer, or any
technology which required an extra page load, on any of their public sites.

Then there is the 2 URL requirement. This has a lot to anti-recommend it: from
a SEO perspective, putting the same content on 2 URLs visible to users
encourages them to link to both, splitting your link equity and exponentially
decreasing the amount of earned organic traffic. (This is because having 2x
the links results in far more than 2x the traffic, because of the steep drop
in click-through rates on SERPs.)

Users also might bookmark or link to the alternative you deactivate at the end
of the test, which means you get to maintain it for forever. I still have 3
URLs on my site from using GWO years ago which I have to remember to support
every time I rewrite my URLs or, whoops, Mrs. Smith's English Resources page
breaks. (Good thing I didn't have a web app at the time -- can you imagine
having some users bookmarked to a signin page which 404ed? Not fun!)

It also plays very, very poorly with dynamic content. You end up essentially
having to rewrite any variant to be displayable in Javascript.

And testing behavioral differences rather than display differences is just
downright frightening. Really: you don't want to do it. Flee. FLEE. Save
yourself while you still can.

Disclaimer: I wrote Rails plugin which competes with them, well, to the extent
that two free things that solve similar problems for different people can be
said to compete. Treat everything I say with a grain of salt. You can find
these thoughts at more detail and length at my site:

<http://www.bingocardcreator.com/abingo/compare>

~~~
paraschopra
Plus - GWO won't let you play around with the data. In the end, what you would
just know is that one variation performed better than other one. You wouldn't
know if one variation resulted in more time spent, less advert clicks, more
user interaction (in terms of comments), more purchases. And you won't know
how it affected your overall funnel behavior. GWO would give you just one goal
tracking, that too a simple convert/no convert.

In short, as Patrick pointed, GWO is developed from a perspective of ease-of-
use and simplicity. You cannot do advanced stuff with it.

Even though I agree with Patrick with the impact of load time with site
performance, the case against using JavaScript must be seen from a broader
perspective. Google, Amazon and other companies stress for millisecond-level
optimization because their site time is already hyper optimized. But if you
see an average e-commerce (or some other) website, a couple of 100
milliseconds due to JavaScript look like a good bargain when it comes to
keeping _view_ layer separate from _code_ layer.

But, overall, I think it should be seen from business objectives. If one cares
about milliseconds, best to do testing at code level. If you care more about
doing tests quickly and not touching your code every now and then, JavaScript
based testing is the way to go.

