
Optimizely (YC W10) Launches Interactive Mode, Cross-Browser Testing, and More - dsiroker
http://blog.optimizely.com/lots-of-new-features-from-optimizely-test-dyn
======
jessedhillon
A/B testing is one of those things that, as an engineer, for me it's very
tempting to just write it myself. I've tried a couple of solutions:

\- Split Test Accelerator may actually have been written by a monkey who was
accidentally taught PHP, judging from the code.

\- Google Web Optimizer seems to have been written as an afterthought, judging
from the simple nature of the tools you get. When I last used it, it was
actually just Javascript which you include, that on loading it simply does a
search and replace of elements. Sure, it gets the job done, but as a somewhat
savvy user, that approach doesn't leave me feeling confident in it.

I don't know what mechanism Optimizely uses to execute tests but it seems
similar to the Google way -- which I admit is the only way I could think to do
remotely hosted A/B testing like this. Optimizely looks a little better in
that they host the assets you might want to test.

I welcome anyone who wants to build a solution that doesn't seem like it's
bolted on.

~~~
patio11
I recommend doing this for anyone whose development platform of choice doesn't
have an existing decent A/B testing option. You can do much, much better as a
developer with something that has tight integration on the backend.

That said, Visual Website Optimizer / Optimizely are _godsends_ for non-
technical stakeholders like the marketing department at Random Insurance
Company who want to be able to split test without getting into an argument
over getting engineers assigned every single time. There's enough of a market
there for both of them to do _quite well indeed_.

~~~
avichal
You hit the nail on the had. I think the much larger market is the non-
technical user. Most "business people" (for lack of a better term) want access
to lots of data to drive their key metrics. But being blocked on an
engineering team to make and push changes to a website is a huge barrier to it
ever happening.

A/B testing for marketers instead of engineers is a killer value to offer to a
business.

------
iamelgringo
I've been using <http://Browserling.com> for interactive cross browser
testing, and I love it. They don't really focus on A/B testing, so it's a
different market than optimizely, but I haven't seen anything better for
walking through how my site looks than Browserling.

~~~
hugs
You should also watch what Sauce Labs is working on.
<http://saucelabs.com/bug-beta> (Leveraging all our existing desktop cloud
infrastructure for interactive x-browser testing.)

------
dtran
Way to go Dan and Pete! Didn't get a chance to play around with it, but the
cross-browser testing and interactive modes sound like really useful features
even outside of the context of A-B testing.

------
WA
Could someone explain me, why a 4.5% increase in conversion rates increases
sales from 1,000$ to 4,500$? For me, 1,000$ * 1.045 = 1,045$ As seen in their
info page here: <https://optimizely.appspot.com/whatisabtesting>

~~~
dsiroker
We should definitely make this clearer in the infographic but the 1% and 4.5%
numbers represent the conversion rates for those two variations. They are not
the INCREASE in conversion rates. Hence going from 1% to 4.5% is an increase
of 350% and that's why they get $4,500 in sales instead of $1,000 in sales.
Does that make sense?

------
jdp23
Kudos to Optimizely for continued progress. It's great to see A/B testing get
steadily broader adoption and it's great to see them riding the wave while
helping it along!

------
maneesh1
Much easier than using Google Web Optimizer. Cross Browser Testing is a huge
add-on and going to save a bunch of time.

------
usagi7
Looks great, I'll check it out when I get a chance.

