
Increase Signups by Adding Video to Your Page - dcancel
http://www.abtests.com/test/38004/landing-for-geomoto
======
harrybr
The heading is rather misleading. It actually should say "Some site somewhere
increased sign-ups by adding video to their page. You may or may not get the
same effect, depending on a vast myriad of factors." But I guess that's not
very catchy is it.

~~~
brown9-2
I seem to remember a very similar posting from abtests.com last week. I would
hope that people don't expect that you can magically copy the results of
someone else's A/B test to your own domain.

~~~
briancooley
I am sure they do. Ever since the "You should follow me on twitter" post
(<http://www.abtests.com/test/19013/other-for-dustin-curtis>), I see that
phrase everywhere.

It could just be that I am noticing the phrase since I read about it, but it
would be a strange coincidence to see the exact wording so often.

~~~
charliepark
I believe the guys at ABTests.com would encourage readers to consider the
results on the site, to evaluate whether or not running a similar test would
make sense for their own site, to then run a test, and to then post the
results at ABTests.com. _Everyone_ would benefit from the same test running on
multiple sites, to see whether the results are universal or not.

I'm sure they don't think that a featured test would result in identical
results on a different site. But opening up data like this for peer review and
replication is a good thing.

Edit: Oh. dcancel said as much, just above this.

------
josefresco
Is the form for trials or paid customers? Seems the form is for trials which
effects only the # of trials which was only marginally higher. What is the
process to convert from trials to paid and is the video really the cause in
the massive jump?

There were also other small subtle changes like the addition of "Request
Instant Price Quote" headline etc.

~~~
dmix
Those small changes can be significant.

A/B tests need to be as specific to one adjustment as possible.

I've noticed flaws like this many times on ABtests.com. But I guess they are
like any war stories, its the overall idea that counts not the details.

~~~
dcancel
Appreciate the feedback. Would love to make our site better, any suggestions
would be very much appreciated.

What are the primary flaws you're seeing?

~~~
dmix
I was referring to the accuracy of the test results on abtests.com in general
not the site in the link. Many of the A/B examples had multiple changes to the
site but only noted one as making a difference.

In your case adding a headline to the form was overlooked.

You were using form submissions as a metric from what I can tell. So changes
specifically to the signup form itself (such as the header) is likely to have
as much of an impact as the video.

Did you track how many people watched the video then signed up as a metric?
That may be more telling.

------
aaronsw
The Obama people, in a much, much bigger test, found that having a video on
their splash page decreased signups.

~~~
carbocation
Is this (esp the embedded video) the reference for your statement?
[http://websiteoptimizer.blogspot.com/2009/11/new-video-
how-w...](http://websiteoptimizer.blogspot.com/2009/11/new-video-how-we-used-
data-to-win.html)

------
cd34
How was the testing done? Every other pageview delivered over a 7 day period
had the video?

If you tested by running one for 7 days and the other for 7 days, there are
numerous other possible factors. Even the comparison of Friday versus Saturday
traffic could have enough variation on a small sample to skew results.
Internet congestion might have slowed the site down during one test, awareness
might predispose people to be more likely to convert beyond trial or even the
fact that the video version renders more quickly to the surfer proving that
html design and page render time may have been the deciding factor. I didn't
see any mention on the site how the trial was actually conducted, but, based
on the other test results, I don't see enough isolation of variables to
decidedly say the video was the sole factor.

------
wgj
The lesson in A/B testing isn't to find magic bullets in other people's
results. It's to actually _do_ A/B testing. If you could generalize from other
people's results, you could just catalog what works and be done. It doesn't
work that way.

------
yannis
This is especially interesting for me (because normally I am put off by
video), do you think is because the video highlighted the installation
procedure better or do you think generally people prefer video?

------
dcancel
I think it was because the video helped the user understand the product
better. All about education.

