
How our freemium plan failed - Shpigford
https://baremetrics.com/blog/freemium-saas-implode
======
cooperadymas
It sounds like the freemium experiment was a success (11% conversion rate),
and the failure was your lack of preparedness to scale to support the extra
load it brought with it.

The 60 customers that were lost during that time period - were they pre-
existing customers from before the freemium switch, or new customers? Did they
leave because the technical issues or because they saw the freemium version
and decided they didn't want to pay any more?

~~~
Shpigford
Pre-existing and they left because of tech/service issues. We interview all
outgoing customers.

And I agree our lack of preparedness to scale was part of this, but it's
shortsighted to only look at a simple conversation rate for determining
success.

"Success" is relative and we define it on a higher level than just conversion.
The fact is, software isn't infinitely scalable...or rather it takes orders of
magnitude more resources for it to approach that level.

There's a certain inflection point where the time spent "getting prepared"
simply outweighs the potential benefits. We could have spent _years_ "getting
prepared" but there was a point where we just had to say "ship it"...and
that's what we did.

~~~
encoderer
Can you share your experience on getting this feedback during a cancelation?
We have mixed success getting users to share. It's about 50/50 for us. We have
even lower response rates from users who have finished their trial without
conversion.

~~~
Shpigford
At the crux of it is not allowing self-serve cancellation. I know, boo, hiss,
burn us at the stake. ;) We wrote a little bit about that here:
[https://baremetrics.com/blog/how-we-reduced-
churn](https://baremetrics.com/blog/how-we-reduced-churn)

But ultimately we exchange at least a few emails with every cancellation and
try to understand exactly why we were no longer addressing their need. 99% of
people are extremely willing to help out here.

Also, Exit Interviews: [http://www.extendslogic.com/business/jobs-to-be-done-
cancel-...](http://www.extendslogic.com/business/jobs-to-be-done-cancel-
interviews/)

~~~
encoderer
Thank you. We're on the same page wrt emailing us to cancel. A quarter of our
churn is actually users who have their cc declined and do not fix it, despite
nag emails, in two weeks. I don't think we've ever heard back from a single
one of these users. For the other 75% I guess we should just be asking more
questions before servicing their request.

~~~
aculver
Hey, I used to run a SaaS product that solved the failed payment problem very
specifically ([http://churnbuster.io](http://churnbuster.io)) and I'd
recommend you extend that 2 week period out to something like 4 to 6 weeks.
Some of our most successful customers were sending out 7-8 emails over a 6
week period. Every business is different, so YMMV, but it might help. A bunch
of other thoughts on the subject that may be helpful to you here:
[https://www.quora.com/What-are-the-industry-average-rates-
fo...](https://www.quora.com/What-are-the-industry-average-rates-for-
involuntary-churn-on-credit-card-subscriptions) .

------
the_watcher
One of the counterintuitive things I learned fastest after entering the
workforce is that there is often an inverse relationship between amount of
money being paid for a product and the level of support demanded from the
users. Almost without fail, our least difficult clients pay the most money,
and our lowest value customers make the most demands.

~~~
nostrademons
I've noticed this too, but the flip side of it is that there's often an
inverse exponential relationship between the amount of money being paid for a
product and the number of users.

I suspect that both observations can be explained by assuming that the amount
of value generated by a product follows a power law. Charge a lot, and you
will restrict your market to only the head of the distribution. A nice
consequence of this is that the consumer surplus (the amount of value you
generate in excess of your price) is highest in this part of the distribution,
which means all your customers are very happy even if you're charging them an
arm and a leg. When you drop your price and service the long tail, you get a
lot of customers who derive only a tiny bit more value from your product than
it costs them, which means they get mad at you really easily.

~~~
lubos
This is the most insightful thing I've read on HN in a while.

The beauty of this argument is that it doesn't put customers (who are less
demanding while paying more) on pedestal.

I run SaaS where there is one plan only. So all customers pay the same price.
One observation I've made is that customers who are the heaviest users (thus
extracting the most value) are the most forgiving ones and the easiest to deal
with.

It's not about the price they pay. It's about net value they extract. The more
value extracted, the higher tolerance threshold.

~~~
the_watcher
> It's not about the price they pay. It's about net value they extract. The
> more value extracted, the higher tolerance threshold.

This is probably the single best way to explain the phenomenon that I've read.
You're absolutely right. My boss and I talk about this a lot, and I'll
absolutely be stealing this in our next conversation about it.

------
compumike
We went through this transition at CircuitLab
[https://www.circuitlab.com/](https://www.circuitlab.com/) \-- now we have
free demos that don't support saving to the server, so they're almost free in
terms of server resources. Also, don't allow forum posts from non-paying
customers.

As the article hints, the real limit isn't computing resources
(CPU/storage/etc), but people resources. Committing these resources to non-
paying customers is generally not a winning proposition.

Upfront-card-capture free trial periods and liberal money-back guarantees give
your customers the zero-risk ability to guarantee that a product is a win-win,
but filter out people who don't take your product seriously enough to consider
paying for it.

I'd advise any founders working on SaaS products to provide some sort of free
trial mechanism, but probably less than they imagine!

~~~
greggman
I don't know why but upfront-card-capture (by which I'm guessing you mean must
enter CC number to try and cancel later) is almost always an instant NO for
me. It feels scammy like a porn site.

I'm not trying to be cheap. I just have no reason to believe you aren't going
to try to grab a few dollars from me hoping I'll forget to cancel. And while
you might have a liberal refund policy many companies don't so the risk I feel
includes previous scams. On top of which, from experience, many companies make
you jump through incredible hoops to cancel and there's no way for me to know
if your company is one of those.

And, if I was an employee of some company. The odds of me feeling comfortable
using a company credit card to trial or even having access to one seems low.

~~~
compumike
Yup, that's completely understandable! We eventually settled on a three-tiered
system:

(1) A limited free demo - just one button click from homepage and it loads in
your browser!

(2) A more extended free demo - but requires e-mail signup / free account
creation.

(3) Fully-featured free trial period, includes saving to server, more export
options, multi-user collaboration, other operations - but requires entering CC
number and choosing plan.

 _Even after all that,_ our default customer service policy is to cancel
subscriptions (available on website without talking to a customer service rep)
and to give refunds immediately when asked, extending into 60-days of past
charges.

At the end of the day, I definitely agree with your sentiment: nobody wants to
get stuck paying for something that doesn't work for you. I think we've
achieved that for our customers, but it took some iteration to get all of
these tiers together, and other companies will have to come up with their own
free trial plans. Just like Baremetrics is doing per their blog post.

(The other side of this is chargebacks: businesses do not want chargebacks!
Stripe, for instance, charges merchants like us a $15 fee for chargebacks
[disputed charges reported by a customer to their bank], plus the time it
takes our customer service staff to respond to the dispute, plus losing the
payment from the customer! We are incentivized to avoid chargebacks.)

------
brwnll
Is there anything in this experience that's actually a result of a Freemium
model?

It sounds like any usage spike would have resulted in the same outcome. If you
had been featured in Oprah's "Favorite Things", and ended up attracting the
same number of users to a _paid_ plan, which of the events would not have
occurred?

1\. Server resources would have been strained.

2\. Engineers would have been reassigned to scale instead of features.

3\. Existing customers would have churned as a result of inability to scale.

The only notable difference is that you did it to yourself. On the flip side,
it also enabled you to turn it "off" by limiting the free account, which you
would not have been able to do if the users were from a referral source
expecting to pay for account anyway.

~~~
sanderjd
If they had a massive influx of paid users, they would have been able to spend
their way out of their scaling problems by first buying server resources and
then hiring.

------
masonhipp
We've had to deal with an onslaught of of free users at our startup
[https://glyphs.co/](https://glyphs.co/) quite a bit over the past few weeks:
last week we took in something like 4k new user accounts.

I'm definitely a big fan of freemium, and specifically I really like limiting
accounts by features (less by usage), but I think it's 100% imperative to
control the rate at which new users are given access to your system. We're
currently in beta which makes it easy (by only allowing users at our pace) but
I think in your case (given the amount of processing required) it would have
been very advantageous to build a line for the free account and let people
wait for a bit. That builds demand, lets you control user-flow, and people who
are very interested can pay to skip the line. Anyhow, just my thoughts,
obviously every scenario is different and you guys know this area best.

~~~
HappyTypist
Counterpoint: This can backfire. Scarcity can make the free plan to be
perceived as having a value that is greater than free.

------
pbreit
I don't think this is an indictment of "freemium" so much as the decision-
making around it.

Also, there's a misnomer that you're supposed to link pricing to features or
usage. No, you link pricing to the customer's ability/desire to pay. Sometimes
this correlates with features and pricing but often times not. Multi-user
access is a good example of a feature that naturally appeals to a company with
resources (i.e., ability to pay). Automatic collections is not.

~~~
jon-wood
I know its just an example, buy multi user access is something I hate not
being on every tier. It's a basic requirement for any business with more than
one employee in my opinion.

The alternative is shared password lists that never really go away.

~~~
encoderer
Oh, interesting. We use this to segment between individuals/consultants and
businesses. We want to be able to offer individuals an affordable plan while
charging businesses an amount commensurate with value they are getting and the
addl load they add to customer support and infrastructure. Any thoughts on
this model?

~~~
jon-wood
As richardwhiuk below says, this approach appears to distinguish between
individuals and businesses, but many small businesses will just create an
individual account anyway and share the password for a single user account.

They're still going to use customer support and infrastructure as much as
other customers, but there's going to be that little bit of extra friction for
users as they find out what the password is, and communicate any changes to
it.

Personally in the context of Cronitor, I'd drop the user levels, and just
segment on monitor counts and features. Maybe bump Slack integration up to the
team level, since that's the sort of feature which is genuinely useful to a
team, and less so to an individual.

~~~
encoderer
Thanks for the feedback. We'll be giving this more thought!

------
venning
An "11.5% conversion rate" is a bit disingenuous. Their advertisement method
attracted users who were not "actually eligible to even think about becoming a
paying customer". I don't think you can so easily exclude them. Put them back
in and you have "over 1,000 free accounts" and only 53 that upgraded.

Assuming "over 1,000" means "over 1,060" then 53 paying accounts actually
means "less than 5%" which fits with their assertion that "the average B2B
conversion rate is around 3-5%".

While I applaud converting 461 "potential paying customers" into 53 paying
customers, the 11.5% rate seems more like an internal metric. It's important
to focus on what you can achieve and so segmenting out those with the
requisite "subscription revenue" to be able to convert is useful for refining
internal goals, but there is a reason why the average conversion is so low:
you always get customers who can't pay.

If other businesses were excluding those customers the quoted average would be
greater than 3-5%.

------
PaulHoule
I think also freedom is not free.

I often download a 15 or 30 day trial of some program and that sounds like a
lot of time, but it can take a day or more to really evaluate something and I
need to multiply that against some guess of the probability we will really
adopt it. Well we have lots of tasks that have a 100% probability of "must be
done" and those are more important and then the trial ends and I forget about
it.

If you make somebody put some chips in the pot upfront it makes them more
serious, like they not only have something to gain but also to lose.

~~~
encoderer
In your case, if confronted with a cc form to start a trial, what do you do?
Do you need near 100% probability to continue?

It's a timely subject for me. We're preparing to test this at Cronitor and
we'll write a blog post with our findings.

~~~
PaulHoule
If I really want or need it and the pricing seems fair in terms of value I
will put in the credit card number. If I am going to spend an hour of my time
to evaluate your product I can spend for the equivalent of an hour of time to
do the trial, more or less.

------
brianwawok
For sure this is interesting to think about. Is all the work I need to do for
the unpaid people who may or may not ever pay me a penny worth it to get a few
more users?

if the "work" is just spinning up some new AWS nodes, the math is easy. If
lifetime value of a free guy > cost of AWS node, do it. If < cost of AWS node,
don't do it.

If the work is lots of staff related hand-holding - then it depends. If your
staff is at 100% capacity and you need to hire more staff, it is closer to the
AWS example above. If you have staff sitting around doing nothing anyway, then
maybe it is "free" as in an already sunk cost.

In their case it seems like they had some engineering flaws in their system
(who doesn't), that made the scaling up part harder. Good to think about how
to scale up early so you don't hit a wall.

~~~
jacquesm
It's actually a bit more complex than that. If the only way you measure the
lifetime value of a free guy in dollars and cents than that value will always
be pegged at '0' so applying your formula literally would lead to 0 times the
freemium model would be given a chance.

But free users can be put to work in other ways, and that value may be very
hard to peg down accurately. A good 'free' user that promotes your service to
paying users, for instance, an employee that uses the product at home that
advocates its use in the business where he/she works. Or maybe free users give
you content that you can monetize in some other way.

And that's the hard part, assigning a value to the contribution. The cost part
is the easy bit, servers are cheap. But support is costly and even then it may
still make sense to have a free tier, it's never going to be an easy decision.

~~~
mcherm
Although the conversion rate to paying customers provides one simple-to-
estimate portion of the value and at an astounding 11% conversion rate, that
alone might pay for it if the support costs weren't so high.

~~~
jacquesm
Support is super hard to scale. The best way is to design your product so well
that you rarely need to answer calls/emails.

------
jusben1369
This is a poster child for the value of raising money vs solely bootstrapping
(I hope DHH doesn't downvote me!) It appears this added $60,000 ARR. I'm not
sure how big the market is (TAM would be all Stripe subscription users?) but
assuming it's 10x of what they found in just 60 days and growing (as Stripe is
growing) Therefore are you looking at $500,000 to $1 mil more per year if
properly resourced to go after this opportunity? Raise $1 million to staff up
for it; the good news is you have real data now.

~~~
anthony_franco
The economics don't make sense by just simply pouring more money into the
problem. If they need to hire an additional employee to support that $60k ARR
they'll be heading into the red in terms of net profits.

They also probably figured that's true given they already raised $500k so they
actually do have additional capital at their disposal. They're not
bootstrapped anymore. Smart move to pull the plug and rethink things.

~~~
jusben1369
Yep. Not at $60K per annum. That's why I talked about how big the opportunity
overall was (what % of the market did they capture in those 2 months?) Maybe
being just on Stripe it's too small vs supporting other subscription
offerings.

------
nostrademons
Did you think about technical solutions to this? You list one obvious one
(limiting the Stripe import for free plans to the 30 days you actually
display), but the other one I would've looked at immediately is to avoid
storing the raw data for free plans entirely, compute your metrics on the fly,
and store only the computed metrics. 1000 customers * 30 days * 25 metrics
(based on your homepage) * 4 bytes/metric = 3MB, which is trivial for any
database to process.

In general, I think most engineers these days overuse databases. At Google,
common wisdom was "never, ever hit the disk during routine serving", and when
I was in the financial industry we'd regularly process the NYSE TAQ data
(50GB, about 2B trades) on a single machine nightly.

It'd be really, really nice to have growth gated on technical problems. :-)

~~~
zrail
The biggest problem with that plan is Stripe's API limits, which (as far as I
understand) are placed on the parent key, not the account key. Also, having to
maintain two different ways to compute all of the metrics shared between the
two account types seems problematic from a complexity standpoint.

Edit: I can't speak for the team and I've never worked on Baremetrics' code
base, but I have developed against the Stripe API quite extensively.

------
hayksaakian
You can't do freemium unless you are prepared to upsell the crap out of your
free users.*

Otherwise you're wasting your time and your users time.

* Unless you want to eat the costs for a few years. In that case you need to really develop your brand to become top of the line in your market.

------
wangii
Additional 1k accounts crashed production servers? I think fixing the software
stack is much more important than business strategical adjustment.

~~~
zrail
Why bother fixing the software stack for accounts that aren't going to pay for
it? Also, remember that Baremetrics imports your entire Stripe account. Every
single event, from day one, so it can analyze and chart them. For any decently
sized business this is a ton of data, let alone 1000 of them.

~~~
wangii
b/c you'll have 1k paying customers sooner or later. if you can't scale, you
won't have a viable business. Look, I'm not suggesting to throw away your
entire stack. It's very likely that you spend few days, find out the bottle
neck and fix it with a scaleable/distributed method. Computing is cheap, and
will forever getting cheaper. Paying customers are rare, deserve deep
appreciation.

~~~
zrail
Just sprinkle Distributed Scaling Magic on your servers and voila! Your
problems are solved!

Nobody is saying scaling out to an additional 1k customers is easy. The great
thing about scaling slowly is that you can grow into the problems and address
them as they come up. Suddenly doubling your userbase, most of which will
ultimately churn out without paying you a dime, and having to deal not only
with the technical but also the _support_ ramifications, is a recipe for
failure. As the article says.

~~~
wangii
If your app is impossible to scale, then your pricing model isn't aggressive
enough. You should have a proper margin to grow your business b/c it'll
ultimately benefit your paying customers.

If it's not, someone else would happily figure out a way to take care of your
customers.

~~~
zrail
It's not a pricing issue, it's a time issue. Scaling a production application
like Baremetrics isn't a turn-key thing. They can't just up the number of
dynos or whatever you're thinking it's like. It takes time, time they decided
was better spent servicing paying customers and new trial customers, instead
of customers that statistically would never pay them.

I'm not sure what we're disagreeing on, really. The blog post literally has
"failure" in the subject. It was an experiment that didn't turn out well, thus
they ended it and are now moving on.

------
PythonicAlpha
I think, the freemium model works best, when the resources needed for one
additional customer (including computing-power, storage and support) are
relatively negligible.

Many companies operate the freemium model with much less than 11% conversion
rate -- but the additional customers they have to carry with the others are
not so a heavy burden.

The advantage of the freemium model is faster scaling and adoption of the
service -- but with the cost of an additional burden ... and you have to know,
if your business can carry the weight or has to do differently.

------
encoderer
I'm sure they made a real development investment to add Freemium features on
top of their free trial. At Cronitor we recently moved in the opposite
direction -- from a limited freemium model to a free-trial -- for similar
reasons.

Our free plans were limited to an extent that they didn't blow-up our service.
Our primary concern was that as we've built out our product, introduced more
advanced features, and signed bigger customers, the risk that a free user
could degrade service for paying customers no longer seemed worth it. We
considered isolating free users in a separate silo but maintaining that
infrastructure also didn't seem worth it.

Ultimately these are hard problems to test because rolling out a change like
this requires a lot of work. Even if you release it via b-test you've already
made a real investment to get that far.

For those curious, we are keeping the free trial model but we do hope to
continue improving trial conversion.

Edit: For clarity

~~~
magic_beans
Baremetrics DID move to a free trial model, as they mentioned.

------
davemel37
I have witnesses several companies scale customer acquisition too quickly,
scramble to get their ducks in a row, become addicted to the growth due to
their bloated operations and eventually collapse under the weight.

This doesn't just go to being prepared to handle growth, but realizing that
most customer acquisition strategies either decay over time or reach a point
of diminishing returns, and putting yourself in a position where you need
growth and volume to survive and betting on your existing customer acquisition
is a formula for failure.

Like all thing, Growth is neither good nor bad on it's own, and should never
be more than a means to an end.

------
fndrplayer13
Freemium is definitely hard. We do this with Quill Engage (quillengage.com)
for Google Analytics reporting and it certainly took us a long time to fully
understand the workload associated with maintaining a free product while
trying to support our main paid product.

For what its worth, we'll be signing up and trying out BareMetrics shortly. I
love what we've seen of your product so far and look forward to seeing what
your team has delivered.

One other comment -- I love how you pulled in the output of your product to so
clearly support the point of your blog post. Not a bad job selling your
product while discussing technical issues :)

~~~
Shpigford
Thanks! Quill Engage looks cool. Let me know if we can help with anything!

------
huhtenberg
> _in two years ... grew the business ... to over $30k /mo in revenue_

Combined with the fact that your About page shows 5 people, this is easily the
most interesting part of the post :)

~~~
Shpigford
Glad to hear at least one sentence was interesting. There are 83 sentences in
the article for an Interesting Conversion Rate (ICR) of 1.205%. Not. Too.
Shabby. :)

~~~
huhtenberg
Oh, I appreciate all of the write-up and it's interesting in its entirety. But
getting a chance to look at core numbers is that much more useful. That's it -
not _interesting_ , but _useful_.

~~~
Shpigford
Then you'll looovvveee this:
[https://demo.baremetrics.com](https://demo.baremetrics.com) and
[https://baremetrics.com/open](https://baremetrics.com/open) :)

------
codegeek
Good learning and write up. Running a bootstrapped product myself, I learned
that the best thing to do is to offer a Free Trial but never unlimited free
account for B2B businesses like these. Also, trying to restrict with features
never really works out at the end as people will always find a way to put
stress on your system along with tons of support issues. Just offer a free
limited trial with ALL features and when the trial ends, ask them to upgrade.

------
EGreg
I think David Heinemeier Hansson and Basecamp would love this.

