Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Do online demos without sign-up actually increase sign-ups?
52 points by thibaut_barrere on July 14, 2012 | hide | past | web | favorite | 52 comments
Hi HN!

I'm working on a SaaS product and I wonder: is there anyone here with concrete feedback about how online demos that do not require sign-up (a bit like https://www.optimizely.com/ and stripe) actually increase sign-ups (or not)?

As a user, it would feel nice, but does it have any measured effect?

Any feedback is most welcome!

Test this if you're curious what it will be like for YOUR application, YOUR audience, and YOUR demo implementation.

To the extent I am capable of commenting publicly on this, I have not ever seen solid numbers which make gradual engagement (technical term for what you're talking about) look like a big win for the company, and to the extent I have seen numbers on gradual engagement, it has been the opposite of a win.

The only numbers I can comment on publicly are the gradual engagement test for BCC. That was my worst Product idea ever, hands down. It 10xed customer support costs because users expected their accounts to magically work (even when, e.g., accessing from home and school and why did your computer delete my cards you just ruined Halloween for my class you monster). The conversion rate for guest accounts to paying accounts was 2. Not two percent. Two accounts ever, in six months, actually saved their email addresses and then, some time later, paid for the software. (This was at least partially because I did not roadblock users at any point and e.g. ask for an email address with a greyed-out Skip option, which is a virtual necessity to get gradual engagement to work. #microUXtip) I have no email addresses for guest accounts nor permission to contact them, something which I have in abundance for trial accounts, and I can tell you that is worth a good deal of money over the long run.

Optimizely and Stripe are, indeed, the two YC companies that I can think of which do gradual engagement. The mafia are very generous with what they know, have an active mailing list, and are universally capable of knocking out a gradual-engagement enabled account system within a week or two. What does this suggest to you as to internal sentiments regarding gradual engagement?

If you want to do something like this, use a post-signup fully capable account for a product tour. Product tours, as compared to gradual engagement, have a) vastly easier UX problems to resolve and b) empirically are full of win. That video I have about the first run experience explains this in a lot of detail.

P.S. Global endorsement of everything btilly says on this thread.

I never new about gradual engagement until using the stripe website. In this case it worked for them and I continually added information into my "guest" account until it was my full fledged account.

This might not work for all web services, but I think it works well for stripe. I honestly doubt I would be using stripe today if they didn't have gradual engagement. When I was in the market I found stripe during my "research" mode. I stumbled into their site and as I "learned" about their product I was also "building" my account. In the end I felt little friction and I integrated stripe payments into LinkPeek.

tl;dr Gradual engagement success appears to depend on the type of person or market a service targets. Pulling from BCC we could deduce that gradual engagement may not work well for teachers, or that your implementation did not work well. I know from experience that this technique converted me from a spectator to a consumer. My personality might not be typical but I generally don't like pressure and often have a bad case of buyers remorse after making purchases. I didn't feel this when "researching" on stripe and I ended up a customer.

This is a great question. Why don't you set up an A/B test?

Yes, I know, you're hoping someone else will know the answer. But if you don't develop the habit of testing, you won't be in the habit of testing, and you won't be testing things that are unique to your business.

This is what patio11 keeps on complaining about. When there is a discussion of A/B testing, everyone says how great it is. But over 90% of you simply aren't doing it. And in a thread like this you can see that people really aren't doing it.

One of the big problems some startups might face is actually getting a significant amount of traffic in order to do a/b testing properly. Even if you set it up correctly, drive traffic and get a "result" you may or may not have enough data to make that result statistically significant.

So, while it might be nice to a/b test everything - sometimes you just want to start out on the right foot and then test from there.

Yes, lack of traffic is hard. In that situation you should try to find companies that you believe run a lot of tests and copy what they do until you have sufficient volume to run tests for yourself.

Yes, lack of traffic is hard. In that situation you should try to find companies that you believe run a lot of tests and copy what they do until you have sufficient volume to run tests for yourself.

Hmmm.... I'd caveat that advice a bit. I'd have to be pretty sure that the company I'm copying is facing the same sort of problems I am. Especially since, after N instances of being wrong, I've learned that I'm often quite bad at figuring out whether company Foo is facing the same problem as company Bar!

Especially when you have limited resources and solving one problem means that you're not solving another.

I've seen far too many instances of people coping what Amazon or Facebook do and it not helping in any way because - well - they're not Amazon or Facebook.

Personally I'd try and spend a more time talking to / observing actual users if I don't have the numbers for more quantitative techniques. Something, sadly, most companies seem to do just about as much as they a/b test :-/ That'll give me some strong hints on what the most significant problems are.

I'm not hoping that at all (I do measure my data actually! I'm even a data-freak, so to speak).

Deploying a demo requires a certain amount of effort.

We're bootstrapping so I'm careful with our efforts.

I'm only asking around to figure out if for most people that did it, the effort was worth it.

My point of view is that if you have an A/B testing platform in place (pretty easy to get up these days - for a first pass you can just insert a piece of JavaScript and pass the work off to various companies that do the stats for you), and you have a demo prepared, it is very little additional work to A/B test two landing pages for said demo. One of which requires signup to see the demo, the other of which shows you the demo before asking for signup.

If you have any traffic at all, the habit of A/B testing will quickly pay for itself.

I do have a couple A/B testing platforms ready and I will definitely use those, but setting up the demo requires a week of work or so.

I see your point though and do agree that once the demo is in place, A/B testing will make sense.

I have a freemium app and I recently stopped asking for an account for the free tier. Traffic tripled and so did the premium accounts. So yes, I can attest that frictionless on boarding increases sales.

Were you able to track via pixels or cookies or something how many of those premium accounts signed after using the free version?

Did you A/B test? Could there have been another reason that traffic tripled?

A/B is such a buzzword here. He recorded the results in one situation, changed something, and then recorded the results in another situation for comparison. That's what A/B testing is.

No, that is emphatically not what A/B testing is.

A/B testing is a statistical technique for teasing out the significance of changing one thing, in a world where lots of things are changing all of the time. If you just change stuff and throw it out there, you can make an educated guess about what mattered, but you don't really know and have no idea how good the data actually was.

If you want to learn more about what A/B testing is, and how to do it, http://elem.com/~btilly/effective-ab-testing/ is a good starting place.

I don't think so. I am not developing this app at a great pace for a while. I changed this one thing and see a continuing impact on traffic.

Impressive return - thanks!

What is your app?

http://this11.com - It's a soccer tactics app.

I truly think most of the future classes of web apps will operate with auto-generated guest accounts. It's a no brainer. If you truly have a great product, let users dive right in (I don't even believe you should have to click to get to the trial, just dump them right into it!).

It's like bait. Once they begin to use your product and put their data/actions into it, you simply remind them to save it with with their email (and password if necessary to save it). Their investment in trying it is a much stronger motivation to sign up than some features list with a promo video ever could be.

EDIT: Not to mention as we move more towards desktop style web apps, it will be the natural progression.

I remember Stripe used to be like this, but not anymore. I wonder what reversed their decision.

Stripe accounts are free and they have a test mode where you can easily create subscriptions and transactions. I think that's beter than a demo.

Yeah, I know, but they used to let you try it without creating an account, and then if you wanted to keep using it you'd just give them an e-mail and a password and you got to keep the API keys and other info.

It wasn't so much a demo as it was an instant trial.

They still let you try it out instantly, they just made the font pretty small on the link to actually do that.


Note: see the skip this step link

That's my guts feeling as well.

IMO it splits the difficulty into two steps (starting to use the tool, then sign-up up / paying).

For a product that is easy to get started, it seems to make sense.

Thanks for your feedback!

I recently started highlighting a demo option next to the sign up button on my site. The total number of people who signed up for a free trial dropped, but that's only because there are a bunch of people who aren't a good fit for the service, and the demo weeded them out. So the total number of signups dropped, but the total number of people that paid at the end of the trial period went up significantly.

> the total number of people that paid at the end of the trial period went up significantly.

That's what I would have suspected as well. Weeding out the users that are not a good fit is very interesting indeed!

I suspect too that asking for CC info straight after the demo would be less of a repellent, once the visitor has actually tried the app (and it will probably lead to less chargeback, too).

> The total number of people who signed up for a free trial dropped, but that's only because there are a bunch of people who aren't a good fit for the service

This is the real benefit of free/no signup. If you can weed out your customers that aren't a complete fit your support costs drop dramatically.

This is a great data point.

Did you run it as an A/B test to be sure that the eventual rise in people paying wasn't because of some other change that you made which you're not thinking about?

Yes, it was a pretty informal test, but the results were clear. Before adding the demo option, about 21 people signed up for the free trial each day. After adding the demo option, that dropped to about 16/day, but 30+ people tried the demo each day (including the 16 that signed up). So basically, and extra ~10 people per day tried the product with the demo option.

There's a 30-day period before those people have to decide to pay, so I didn't run a full A/B test during that entire period because we were clearly getting more qualified signups, so I don't perfect data on exactly how big the increase was.

One thing you have to account for is that by capturing their emails you can also do email marketing to increase the chance they return to app as you develop it out more and add cool new features. The caveat being, your product has to be interesting and your emails infrequent and well designed.

Indeed. Don't just start sending weekly "WE'VE BEEN MISSING YOU" emails right off the bat...or ever, really.

I would say one of the most important aspects to consider is the workflow of being a 'demo user' to signing up. Sure, rewriting your app's sign up process into an 'online demo' by automatically generating a guest account won't take that much time. But the real challenge is to make sure you're designing the whole process in such a way that you're not adding more confusion for the user. For example - and this is probably more relevant for B2B/enterprise apps - users might be really used to having to sign up with a password (or even worse, having to fill out a Request for Demo form and wait for a call from a sales rep). In that case I'd try to make sure your users understand whether or not the data they enter is private or shared, if it's saved once they "sign up", etc.

We also notice that our users sometimes create an account, play around a bit, but then leave and come back again a few days later because they were in the middle of something. I'm curious what the effect is of having no way of contacting them, or leaving a reminder in their inbox of where to find back the website. Maybe just asking for an email address would help.

In any case it would be interesting to see some more data. So far we've just been using sign up forms, but always try to keep them short and 'friendly'. We're definitely considering giving this a try, so hopefully we have some meaningful data to show in a while.

This question comes almost at the time for me. A few weeks ago, i was wondering the same question for my first startup, an online task manager, namely eekip. In the end i thought, there are so many task management software out there, and each single person likes it different way, so I'd better give people a look inside to see if this is suitable for them. And hell, it wouldn't be that much of a pain to code it up anyway. I ended up spending about 3 days straight to get it done. What happens is users can try the demo account with only 1 click, and once logged in the demo acc, i tried to present the best stuff to the user, just so that user wouldnt be walking away and say : "Move along, nothing to see. Just another dumb task management software!". If you're curious, you can see how i pulled it together at http://eekip.com (still in alpha)

And did your three days of work creating a demo (when you could have been spending three days building features :-) result in better conversions?

I have a similar question, I'll use this opportunity to ask it:

If I present the visitor a thing or a service he can buy, is it bad that I need him to sign up separately and confirm his e-mail before he can actually press "place order"?

I guess it sounds bad from how I'm saying it, but my concerns are the security (confirmed e-mail = you can reach the person) and work investment (creating a form that would do two logically-separate things at the time is more work).

So what do you guys think? Should I let people do anything without a confirmed e-mail address and work on merging sign-up, sign-in and place-order forms together?

Read this http://www.uie.com/articles/three_hund_million_button/

The company was Best Buy. Look at the stats which show that requiring registration essentially never helps. About the only time it would is for (very?) frequent visitors.

You don't need "security" for orders other than making order ids non-guessable. For customer service people can give their order id and other details if necessary (eg zip code, name).

Requiring email confirmation almost always decreases conversion and frequently in a major way. You should have a very good reason for confirming an email address (for example, PayPal tying payments to email addresses) or don't do it (or do it passively; ie, if someone happens to click on a link in an email you send them, mark the email address as confirmed).

I see. Thank you!

I disabled the limits for unconfirmed users and will create a merged form as soon as possible.

The development flow looks like this to me right now: First I built a strictly REST/model-centered back-end, and now I have to smooth the corners between user-friendliness and those standards. Interesting observation.

(Also, if anyone is curious what I'm building, it's https://artistsnclients.com/)

Reduce friction whenever possible. Allow the user to place an order before the confirmation, but limit the account until the user does. Also show an alert after a successful order that an email confirmation is being sent out and an option to resend it.

I would say you should combine them. As long as you provide an easy way for contacting in case they have a problem, they best thing you can do is simplify the registration/purchase process.

The only way to know for sure is split testing.

On the one hand, you build a bigger mailing list + extra friction.

On the other, smaller mailing list, but more users get to try your service.

I recommend a split test.

I posted pretty much the same question as you a couple of weeks ago for our product http://flashissue.com. at the end of the day it's important to just measure what's important in YOUR funnel.

it's taken us a while to get mixpanel installed so we can measure this properly.

our key metric is not how many people we get logging in but how many people move through our product and send out an email newsletter. we'll be a/b testing this with a "sign-up" button and an "open" app.

we should compare findings :-)

I don't think it would hurt conversions. Instead I question the amount of time spent building the demo versus working on making the product itself better.

In case of LinkPeek the demo was basically just a free version of the SaaS product. It only required one webpage to be built and took on the order of about 2 hours to create.

If you are interested in seeing the demo go to linkpeek.com and click try now.

That's exactly my point. In the case of our app (https://www.wisecashhq.com/), it would be totally doable, but would still require a bit of work to create two sub-apps based on the same core features, and handle the scaling differently (I don't want the demo traffic to interfere with my actual, paying customers).

Thanks for you link as well!

Chances of me signing up for a service which has an online demo - increase - because I can actually know what am I getting into.

However, at the same time if the demo looks crappy enough, chances of me signing up decrease exponentially.

If the idea sounds interesting to me. I would anyway try it. The demo button infact reduces the chances of actual signup as after trying out i may or maynot signup.

I'd love to reduce sign-ups that could end up in cancellations straight away! It would only make more support.

There isn't "an" answer. You need to test and explore. I've seen nice online demos make significant differences to final revenue. I've also seen them make zero difference. Depends on the product and the market and the quality of the demo.

One issue with a/b testing something like this is that the cost to do a good online demo can be non-trivial - and judging whether it can be a success can be non-obvious without doing a good online demo first. Egg. Chicken. Chicken. Egg.

If I were me I might go about it something like this (assuming that building the demo feature isn't trivial - in which case just a/b test it :-)

0) I'd already be doing quick'n'dirty usability testing and talking with users regularly. This is even faster at showing up low-hanging fruit than a/b testing in many situations. I'd want some hint that the kinds of problems demos solve (e.g. not understanding what the product did) were showing up and there were no bigger fish to fry.

1) I'd guesstimate how much time/money it would take to build a good online demo.

2) I'd guesstimate how much of an increase in final conversion it would take for it to be worth while taking that time (considering that I could be building other features).

3) I'd add a "demo" button. I'd have it link to a "demo coming soon" page with an email signup so I could grab some "demo wanting" users for future quizzing. I'd split test that to see if anybody clicked on the demo button.

4) I now know what percentage of people are even vaguely interested in an online demo. So I have an upper bound for how much of an increase in conversions it could make. If it's worse than the number I came up with that would make it worthwhile - I can stop now.

5) I also now know whether "demo seeking" potential customers convert better/worse/same than non-demo-seeking potential customers.

6) I'd probably go talk to the demo-seeking users I gather on the e-mail list no matter what - to find out more about why they wanted the demo. Did they not understand the product? Were they looking for a particular feature? Did they want to play to figure out whether it was worth the money? Something else? Maybe you can fix their problem without building the demo. Maybe everybody wants to know whether it has FooFeature and you just need to add a bullet point to the home page.

7) From the previous step I now know more about why folk want an online demo. I'd try and build as little as possible to fix that particular problem. Stick that on the demo page - and see if we got any more conversions in the next round of testing.

Repeat until no longer improving :-)

In my experience: yes. But, always measure! If it's something you can accomplish in under a week, I'd say go for it.

Given how I have architectured the app (modular enough) and thanks to DotCloud (for scaling up the demo without much setup), that's definitely something I could deploy in a week. Thanks!

Definitely. I don't have any numbers, but over the years just have come to accept that as fact.

I would always let people test free stuff without an account, and for features that can only work with an account, create a guest one automagically that the user can convert into a proper account if they wish to use those features for more than one session.

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact