

How Ghost increased conversions - eamonncarey
http://blog.ghost.org/ghost-onboarding/

======
patio11
This sort of segmentation exists in many, many free trials. Sometimes the
causational arrow can be difficult to tease out. For example, one's best
customers often engage higher with the app on every conceivable dimension, so
if you happen to test a particular feature, you'll see adoption of that
feature correlates with conversions and might be inclined to e.g. try to juice
adoption via promoting in onboarding. That works only some of the time.

In general, though, strong endorsement of figuring out what people care about
quantitatively and qualitatively, instrumenting that, and then testing early
in the funnel improvements to it. The gold standard is, of course, not doing
retrospective analysis but rather split testing the intervention. I sympathize
that this is difficult for a lot of SaaS apps as you need high trial volumes
to make it work.

I can't share client results but my blog probably has two or three write ups
of 10%+ lifts for BCC doing this.

------
jasonlfunk
The title is wrong. Ghost didn't increase their conversion rate by 1000%. They
found a segment of their user base that converted 1000% more than another
segment.

It is not included in this post whether the overall conversion rate actually
improved substantially after the changes were made.

~~~
johnonolan
Hey Jason, that's actually pretty true. We found a segment which converted
1,000% better and then managed to increase the number of people in that
segment by 370%

The title of the post is "How we Figured Out What Makes People Love Ghost
1,000% More" \- which I think is still pretty accurate :)

~~~
gabemart
> The title of the post is "How we Figured Out What Makes People Love Ghost
> 1,000% More" \- which I think is still pretty accurate :)

I disagree. If you choose the bucket "people who subscribed to Ghost Pro",
they would convert ∞% better than the bucket "people who didn't sign up for
Ghost Pro". That doesn't mean that signing up for Ghost Pro makes people love
Ghost ∞% more.

More generally, how can you tell if people installing a custom theme makes
them love Ghost more, or if people install a custom theme _because_ they love
Ghost more?

Getting more people into the bucket of "people who installed a custom theme"
doesn't actually help your bottom-line conversion rate until you can establish
unequivocally that the people you've added to the bucket convert at the same
rate as the bucket did before they were added to it.

~~~
fredophile
> Getting more people into the bucket of "people who installed a custom theme"
> doesn't actually help your bottom-line conversion rate until you can
> establish unequivocally that the people you've added to the bucket convert
> at the same rate as the bucket did before they were added to it.

This is incorrect. Let's say we have two buckets, A and B, where bucket A is
people who haven't used a feature and bucket B is people who have. If some
testing shows that people in bucket B convert far better than people in bucket
A you make a change to get more people into bucket B. The change is successful
if more people end up in bucket B and bucket B still outperforms bucket A and
the total contribution coming from bucket B is higher. Bucket B doesn't need
to perform equally well before and after the test.

Using the numbers from the link bucket B was performing at 10x the conversion
of bucket A. Out of 10,000 users bucket B contributed 70 subscribers. After
the changes bucket B received almost five times the users as previously and
converted at four times the rate of bucket A. Out of 10,000 users bucket B
contributed 884 subscribers. If the change had no impact we'd expect more
users in bucket B but not more subscribers. These numbers aren't really
accurate since I took them from the diagrams in the article and they change
what they're looking at between the two. To be correct you need to compare the
same thing before and after changing a feature.

------
joosters
How do you know that you have the cause and effect the right way around? After
all, it could well be that users add a theme _because_ they are about to buy
the service, and figure that they should spend some time customising their
pages now that they have decided to sink money into it.

Ok, so that may not sound totally convincing, but in general I think it is a
big problem with the analytics in the article. If you can't tell if A caused B
or B caused A, then you can't reliably act on it.

------
foxylad
There are "ah ha!" moments, and there are also "oh no!" moments.

I love Ghost's simplicity for blogging, but then I spotted an egregious
spelling mistake in one of the articles I'd published. I went back to the
editor to check where I'd missed the ubiquitous red squiggly line, and found
there wasn't one. No squiggly line, and in fact no spellchecker at all!

Given that spelling is very important for blogs, and that all browsers have
very effective spellcheckers built in, I'm at a loss to understand why this
wasn't bug number 1 to the developers. They use a markdown editor that somehow
kills browser spellchecking, but there are several alternatives out there and
this was a definite "Oh no!" moment for me.

------
erroneousfunk
This data intuitively makes sense -- the added theme gives users a sense of
"this is _my_ product," where before it was "this is a Ghost product that I'm
using right now."

As a user, however, I'm annoyed by videos, and hardly ever watch tutorial
videos. Why spend a minute and a half watching something when I can read it
much faster? Especially if setting up a theme is as easy as you're trying to
make it -- shouldn't it be obvious how to do it?

------
adamcowley
I find it a little suspect that you had exactly 3400 people watch your
training video and exactly 6600 not watch the video. Those seem like very
convenient statistics. Am I missing something?

~~~
johnonolan
Hey Adam, great question! I took the conversion %'s measured and applied them
to a base sample size just to make the numbers in the post a little bit more
easy to read/understand. Hope this helps clarify

------
davidholmesnyc
This goes to show how important analytics is in a business. They found
something they could market better in their own software by understanding
their business more. Big ups to Ghost for that discovery.

------
siscia
[Disclaimer: Off topic and reaching for help]

Me and my girlfriend are looking for some gigs so we can buy a ticket plane
and meet in February (she is in Italy right now while I am in China).

She is working through her PhD in Statics while I am a developer with a lot of
interest in data analysis and visualization.

If somebody need the same analysis they did a ghost it is something we can
handle pretty well.

If you want to collaborate just drop a line at: simone (at) mweb (dot) biz

[End off topic, and sorry about that]

