

Ask HN: Do online demos without sign-up actually increase sign-ups? - thibaut_barrere

Hi HN!<p>I'm working on a SaaS product and I wonder: is there anyone here with concrete feedback about how online demos that do not require sign-up (a bit like https://www.optimizely.com/ and stripe) actually increase sign-ups (or not)?<p>As a user, it would feel nice, but does it have any measured effect?<p>Any feedback is most welcome!
======
patio11
Test this if you're curious what it will be like for YOUR application, YOUR
audience, and YOUR demo implementation.

To the extent I am capable of commenting publicly on this, I have not ever
seen solid numbers which make gradual engagement (technical term for what
you're talking about) look like a big win for the company, and to the extent I
have seen numbers on gradual engagement, it has been the opposite of a win.

The only numbers I can comment on publicly are the gradual engagement test for
BCC. That was _my worst Product idea ever_ , hands down. It 10xed customer
support costs because users expected their accounts to magically work (even
when, e.g., accessing from home and school and why did your computer delete my
cards you just ruined Halloween for my class you monster). The conversion rate
for guest accounts to paying accounts was 2. Not two percent. Two accounts
_ever_ , in six months, actually saved their email addresses and then, some
time later, paid for the software. (This was at least partially because I did
not roadblock users at any point and e.g. ask for an email address with a
greyed-out Skip option, which is a virtual necessity to get gradual engagement
to work. #microUXtip) I have no email addresses for guest accounts nor
permission to contact them, something which I have in abundance for trial
accounts, and I can tell you that is worth a good deal of money over the long
run.

Optimizely and Stripe are, indeed, the _two_ YC companies that I can think of
which do gradual engagement. The mafia are very generous with what they know,
have an active mailing list, and are universally capable of knocking out a
gradual-engagement enabled account system within a week or two. What does this
suggest to you as to internal sentiments regarding gradual engagement?

If you want to do something like this, use a _post-signup_ fully capable
account for a product tour. Product tours, as compared to gradual engagement,
have a) vastly easier UX problems to resolve and b) empirically are _full of
win_. That video I have about the first run experience explains this in a lot
of detail.

P.S. Global endorsement of everything btilly says on this thread.

~~~
foxhop
I never new about gradual engagement until using the stripe website. In this
case it worked for them and I continually added information into my "guest"
account until it was my full fledged account.

This might not work for all web services, but I think it works well for
stripe. I honestly doubt I would be using stripe today if they didn't have
gradual engagement. When I was in the market I found stripe during my
"research" mode. I stumbled into their site and as I "learned" about their
product I was also "building" my account. In the end I felt little friction
and I integrated stripe payments into LinkPeek.

tl;dr Gradual engagement success appears to depend on the type of person or
market a service targets. Pulling from BCC we could deduce that gradual
engagement may not work well for teachers, or that your implementation did not
work well. I know from experience that this technique converted me from a
spectator to a consumer. My personality might not be typical but I generally
don't like pressure and often have a bad case of buyers remorse after making
purchases. I didn't feel this when "researching" on stripe and I ended up a
customer.

------
btilly
This is a great question. Why don't you set up an A/B test?

Yes, I know, you're hoping someone else will know the answer. But if you don't
develop the habit of testing, you won't be in the habit of testing, and you
won't be testing things that are unique to your business.

This is what patio11 keeps on complaining about. When there is a discussion of
A/B testing, everyone says how great it is. But over 90% of you simply aren't
doing it. And in a thread like this you can see that people really aren't
doing it.

~~~
joelrunyon
One of the big problems some startups might face is actually getting a
significant amount of traffic in order to do a/b testing properly. Even if you
set it up correctly, drive traffic and get a "result" you may or may not have
enough data to make that result statistically significant.

So, while it might be nice to a/b test everything - sometimes you just want to
start out on the right foot and then test from there.

~~~
btilly
Yes, lack of traffic is hard. In that situation you should try to find
companies that you believe run a lot of tests and copy what they do until you
have sufficient volume to run tests for yourself.

~~~
adrianhoward
_Yes, lack of traffic is hard. In that situation you should try to find
companies that you believe run a lot of tests and copy what they do until you
have sufficient volume to run tests for yourself._

Hmmm.... I'd caveat that advice a bit. I'd have to be pretty sure that the
company I'm copying is facing the same sort of problems I am. Especially
since, after N instances of being wrong, I've learned that I'm often quite bad
at figuring out whether company Foo is facing the same problem as company Bar!

Especially when you have limited resources and solving one problem means that
you're not solving another.

I've seen far too many instances of people coping what Amazon or Facebook do
and it not helping in any way because - well - they're not Amazon or Facebook.

Personally I'd try and spend a more time talking to / observing actual users
if I don't have the numbers for more quantitative techniques. Something,
sadly, most companies seem to do just about as much as they a/b test :-/
That'll give me some strong hints on what the most significant problems are.

------
hcho
I have a freemium app and I recently stopped asking for an account for the
free tier. Traffic tripled and so did the premium accounts. So yes, I can
attest that frictionless on boarding increases sales.

~~~
btilly
Did you A/B test? Could there have been another reason that traffic tripled?

~~~
notatoad
A/B is such a buzzword here. He recorded the results in one situation, changed
something, and then recorded the results in another situation for comparison.
That's what A/B testing is.

~~~
btilly
No, that is emphatically not what A/B testing is.

A/B testing is a statistical technique for teasing out the significance of
changing one thing, in a world where lots of things are changing all of the
time. If you just change stuff and throw it out there, you can make an
educated guess about what mattered, but you don't really know and have no idea
how good the data actually was.

If you want to learn more about what A/B testing is, and how to do it,
<http://elem.com/~btilly/effective-ab-testing/> is a good starting place.

------
nwienert
I truly think most of the future classes of web apps will operate with auto-
generated guest accounts. It's a no brainer. If you truly have a great
product, let users dive right in (I don't even believe you should have to
click to get to the trial, just dump them right into it!).

It's like bait. Once they begin to use your product and put their data/actions
into it, you simply remind them to save it with with their email (and password
if necessary to save it). Their investment in trying it is a much stronger
motivation to sign up than some features list with a promo video ever could
be.

EDIT: Not to mention as we move more towards desktop style web apps, it will
be the natural progression.

~~~
jmonegro
I remember Stripe used to be like this, but not anymore. I wonder what
reversed their decision.

~~~
pbreit
Stripe accounts are free and they have a test mode where you can easily create
subscriptions and transactions. I think that's beter than a demo.

~~~
jmonegro
Yeah, I know, but they used to let you try it without creating an account, and
then if you wanted to keep using it you'd just give them an e-mail and a
password and you got to keep the API keys and other info.

It wasn't so much a demo as it was an instant trial.

~~~
zzimbler
They still let you try it out instantly, they just made the font pretty small
on the link to actually do that.

<https://manage.stripe.com/register>

Note: see the skip this step link

------
the_bear
I recently started highlighting a demo option next to the sign up button on my
site. The total number of people who signed up for a free trial dropped, but
that's only because there are a bunch of people who aren't a good fit for the
service, and the demo weeded them out. So the total number of signups dropped,
but the total number of people that paid at the end of the trial period went
up significantly.

~~~
btilly
This is a great data point.

Did you run it as an A/B test to be sure that the eventual rise in people
paying wasn't because of some other change that you made which you're not
thinking about?

~~~
the_bear
Yes, it was a pretty informal test, but the results were clear. Before adding
the demo option, about 21 people signed up for the free trial each day. After
adding the demo option, that dropped to about 16/day, but 30+ people tried the
demo each day (including the 16 that signed up). So basically, and extra ~10
people per day tried the product with the demo option.

There's a 30-day period before those people have to decide to pay, so I didn't
run a full A/B test during that entire period because we were clearly getting
more qualified signups, so I don't perfect data on exactly how big the
increase was.

------
adrianhoward
There isn't "an" answer. You need to test and explore. I've seen nice online
demos make significant differences to final revenue. I've also seen them make
zero difference. Depends on the product and the market and the quality of the
demo.

One issue with a/b testing something like this is that the cost to do a good
online demo can be non-trivial - and judging whether it can be a success can
be non-obvious without doing a good online demo first. Egg. Chicken. Chicken.
Egg.

If I were me I might go about it something like this (assuming that building
the demo feature isn't trivial - in which case just a/b test it :-)

0) I'd already be doing quick'n'dirty usability testing and talking with users
regularly. This is even faster at showing up low-hanging fruit than a/b
testing in many situations. I'd want some hint that the kinds of problems
demos solve (e.g. not understanding what the product did) were showing up and
there were no bigger fish to fry.

1) I'd guesstimate how much time/money it would take to build a good online
demo.

2) I'd guesstimate how much of an increase in final conversion it would take
for it to be worth while taking that time (considering that I could be
building other features).

3) I'd add a "demo" button. I'd have it link to a "demo coming soon" page with
an email signup so I could grab some "demo wanting" users for future quizzing.
I'd split test that to see if anybody clicked on the demo button.

4) I now know what percentage of people are even vaguely interested in an
online demo. So I have an upper bound for how much of an increase in
conversions it could make. If it's worse than the number I came up with that
would make it worthwhile - I can stop now.

5) I also now know whether "demo seeking" potential customers convert
better/worse/same than non-demo-seeking potential customers.

6) I'd probably go talk to the demo-seeking users I gather on the e-mail list
no matter what - to find out more about why they wanted the demo. Did they not
understand the product? Were they looking for a particular feature? Did they
want to play to figure out whether it was worth the money? Something else?
Maybe you can fix their problem without building the demo. Maybe everybody
wants to know whether it has FooFeature and you just need to add a bullet
point to the home page.

7) From the previous step I now know more about why folk want an online demo.
I'd try and build as little as possible to fix that particular problem. Stick
that on the demo page - and see if we got any more conversions in the next
round of testing.

Repeat until no longer improving :-)

------
endergen
One thing you have to account for is that by capturing their emails you can
also do email marketing to increase the chance they return to app as you
develop it out more and add cool new features. The caveat being, your product
has to be interesting and your emails infrequent and well designed.

~~~
rhizome
Indeed. Don't just start sending weekly "WE'VE BEEN MISSING YOU" emails right
off the bat...or ever, really.

------
wim
I would say one of the most important aspects to consider is the workflow of
being a 'demo user' to signing up. Sure, rewriting your app's sign up process
into an 'online demo' by automatically generating a guest account won't take
that much time. But the real challenge is to make sure you're designing the
whole process in such a way that you're not adding more confusion for the
user. For example - and this is probably more relevant for B2B/enterprise apps
- users might be really used to having to sign up with a password (or even
worse, having to fill out a Request for Demo form and wait for a call from a
sales rep). In that case I'd try to make sure your users understand whether or
not the data they enter is private or shared, if it's saved once they "sign
up", etc.

We also notice that our users sometimes create an account, play around a bit,
but then leave and come back again a few days later because they were in the
middle of something. I'm curious what the effect is of having no way of
contacting them, or leaving a reminder in their inbox of where to find back
the website. Maybe just asking for an email address would help.

In any case it would be interesting to see some more data. So far we've just
been using sign up forms, but always try to keep them short and 'friendly'.
We're definitely considering giving this a try, so hopefully we have some
meaningful data to show in a while.

------
clu3
This question comes almost at the time for me. A few weeks ago, i was
wondering the same question for my first startup, an online task manager,
namely eekip. In the end i thought, there are so many task management software
out there, and each single person likes it different way, so I'd better give
people a look inside to see if this is suitable for them. And hell, it
wouldn't be that much of a pain to code it up anyway. I ended up spending
about 3 days straight to get it done. What happens is users can try the demo
account with only 1 click, and once logged in the demo acc, i tried to present
the best stuff to the user, just so that user wouldnt be walking away and say
: "Move along, nothing to see. Just another dumb task management software!".
If you're curious, you can see how i pulled it together at <http://eekip.com>
(still in alpha)

~~~
adrianhoward
And did your three days of work creating a demo (when you could have been
spending three days building features :-) result in better conversions?

------
daveid
I have a similar question, I'll use this opportunity to ask it:

If I present the visitor a thing or a service he can buy, is it bad that I
need him to sign up separately and confirm his e-mail before he can actually
press "place order"?

I guess it sounds bad from how I'm saying it, but my concerns are the security
(confirmed e-mail = you can reach the person) and work investment (creating a
form that would do two logically-separate things at the time is more work).

So what do you guys think? Should I let people do anything without a confirmed
e-mail address and work on merging sign-up, sign-in and place-order forms
together?

~~~
pbreit
Requiring email confirmation almost always decreases conversion and frequently
in a major way. You should have a very good reason for confirming an email
address (for example, PayPal tying payments to email addresses) or don't do it
(or do it passively; ie, if someone happens to click on a link in an email you
send them, mark the email address as confirmed).

~~~
daveid
I see. Thank you!

I disabled the limits for unconfirmed users and will create a merged form as
soon as possible.

The development flow looks like this to me right now: First I built a strictly
REST/model-centered back-end, and now I have to smooth the corners between
user-friendliness and those standards. Interesting observation.

(Also, if anyone is curious what I'm building, it's
<https://artistsnclients.com/>)

------
philhill
I posted pretty much the same question as you a couple of weeks ago for our
product <http://flashissue.com>. at the end of the day it's important to just
measure what's important in YOUR funnel.

it's taken us a while to get mixpanel installed so we can measure this
properly.

our key metric is not how many people we get logging in but how many people
move through our product and send out an email newsletter. we'll be a/b
testing this with a "sign-up" button and an "open" app.

we should compare findings :-)

------
foxhop
I don't think it would hurt conversions. Instead I question the amount of time
spent building the demo versus working on making the product itself better.

In case of LinkPeek the demo was basically just a free version of the SaaS
product. It only required one webpage to be built and took on the order of
about 2 hours to create.

If you are interested in seeing the demo go to linkpeek.com and click try now.

~~~
thibaut_barrere
That's exactly my point. In the case of our app
(<https://www.wisecashhq.com/>), it would be totally doable, but would still
require a bit of work to create two sub-apps based on the same core features,
and handle the scaling differently (I don't want the demo traffic to interfere
with my actual, paying customers).

Thanks for you link as well!

------
makuchaku
Chances of me signing up for a service which has an online demo - increase -
because I can actually know what am I getting into.

However, at the same time if the demo looks crappy enough, chances of me
signing up decrease exponentially.

~~~
sachin132
If the idea sounds interesting to me. I would anyway try it. The demo button
infact reduces the chances of actual signup as after trying out i may or
maynot signup.

~~~
thibaut_barrere
I'd love to reduce sign-ups that could end up in cancellations straight away!
It would only make more support.

------
lvh
In my experience: yes. But, always measure! If it's something you can
accomplish in under a week, I'd say go for it.

~~~
thibaut_barrere
Given how I have architectured the app (modular enough) and thanks to DotCloud
(for scaling up the demo without much setup), that's definitely something I
could deploy in a week. Thanks!

------
GigabyteCoin
Definitely. I don't have any numbers, but over the years just have come to
accept that as fact.

------
ktizo
I would always let people test free stuff without an account, and for features
that can only work with an account, create a guest one automagically that the
user can convert into a proper account if they wish to use those features for
more than one session.

