
Your average revenue per customer is meaningless - matm
http://data.heapanalytics.com/your-average-revenue-per-customer-is-meaningless/
======
mrkurt
It depends on your funnel consistency. If you are selling a self service
product with transparent pricing, fully realized ARPU can be a solid number.
It will change as your customer base changes, and there still could be
outliers, but it's decent and easy to reason about (and project with).

If, however, you are kind of self service, but do an Enterprise sales deal ...
you have two funnels with two vastly different customer profiles. Not only is
ARPU bad in this scenario, most numbers are.

The trick is narrowing the scope of a given metric enough that you can use it
to make good decisions, and but not so much that it ignores important parts of
a business.

------
programminggeek
Well, you can think that averages are meaningless, but they do mean something.
If your average sale is $10 and you are spending on average $9 to acquire
them, you have a 10% margin on average.

Segmenting is great and can actually move the averages wildly, but that
doesn't mean the averages are pointless. It means that averages are just one
view of the data.

If I want a really high level view of the data, an average is great. If I want
a super low level view of the data, looking at each customer or small groups
of customers is great too. It depends on what you want to know and what
problem you are trying to solve.

Use the right tool for the job.

------
twic
So what indicator statistics _are_ good, then? "Take a look at the entire
picture, not just the average" is great, but what am i going to put on my
dashboard? What am i going to make my goal for the year?

On the technical side, when trying to boil down scads of metric datapoints
into a single number, the statistics i usually end up using are the median and
the 95th centile (or some higher centile). The median gives some sort of rough
idea of where the middle of the main mass distribution is, ignoring the
extremes, and the high centile gives an indication of where the top edge of
the main mass is, ignoring freak outliers.

Would those be any use for revenue per customer?

Would we be better off with the median and some number that tries to capture
the shape of the power law? Something that says "for every dollar you go up in
revenue, the number of users drops by X%", or something like that?

~~~
raviparikh
One of the things to take away is that it's often counter-productive to take
one summary statistic like "mean" or "median" and try to just optimize that.
The correct strategy is likely to be a bit more nuanced and depend on your
specific situation.

For example, maybe you have a few different pricing tiers, and instead of
optimizing for ARPU, you optimize for the number of people in a given tier
that you can up-sell into a higher pricing tier. Now you're optimizing for
something meaningful. Note that even if you did a good job in accomplishing
this goal, your ARPU might actually go down, if you simultaneously saw a lot
of growth in your lower pricing tiers from new signups.

------
Mz
I am vaguely reminded of a recent discussion* about negotiating with Steve
Jobs where kind of the reverse point was made: The author was advised to make
the number fit the scenario Steve claimed he wanted. The author did so,
creatively, without lying.

A lot of people do not understand the substance behind the numbers and this
leads to garbage in, garbage out. That's what this article is about:
Understanding what's behind the numbers and not being fooled by them. The
previous piece was also about understanding the substance and knowing how to
work the numbers to make other people happy with the proposed deal.

A good read on similar topics: How to Lie with Statistics.

* [https://news.ycombinator.com/item?id=7451018](https://news.ycombinator.com/item?id=7451018)

------
brockf
I'm not sure I follow this, or buy into the suggestion of the post.

First, I don't see how it's true that data with a relatively large amount of
variance will tend to be power law distributed. Defining what a "large amount"
of variance is is tough (it depends on your intuition and choice of variance
metric) but there are lots of distributions with considerable variance that
are, for example, normally distributed (many more than are power law
distributed, as far as I can tell).

Second, if you find that this is misleading your projections, why not just use
a different kind of average? For example, if you just want to know, "How much
is the next customer likely to spend?", you might use the mode. Or, if you
want a more robust average (i.e., less likely to be seriously thrown off by
outliers), why not use the median? You can even complement these with
confidence intervals if you want to get a sense of their precision.

Like twic already said, you need some indicator to understand what's going on
with your business. I think that in many cases, this will be the mean. But if
you want something more robust or more practical, perhaps the median or mode
might suit you better.

~~~
raviparikh
To address your first point – that's fair, I didn't really dive into the
statistics in detail (since it wasn't really the point of the post). If
variance of a distribution is finite then the central limit theorem applies,
and given a sufficient number of trials, the distribution will begin to
approach a normal distribution. However some data sets have infinite variance
and may (under some conditions) begin to approach a power law distribution;
or, they have finite but extremely large variance, in which case the number of
trials it will take to begin to look like a normal distribution is very, very
large.

For your second point – there are legitimate uses for other summary statistics
(mode, median), but they can still be very misleading. For example, you
mention using the mode of the distribution as a predictor of what the next
customer will pay – this definitely wouldn't work for a company with a metered
pricing plan, for example. Distributions are often not well characterized by
singular summary statistics, Anscombe's Quartet (mentioned elsewhere in this
comments section) is a good example of this:
[http://en.wikipedia.org/wiki/Anscombe's_quartet](http://en.wikipedia.org/wiki/Anscombe's_quartet)

~~~
brockf
Re: power law distributions. I'm not sure what you mean about finite versus
infinite variance. Are you referring to whether you are analyzing a bounded
versus unbounded scale? Even if a scale is unbounded, that still wouldn't make
a dataset any more likely to power law distributed. A power law distribution
would be, however, somewhat more likely to be observed in cases where there is
only an upper- or lower-bound (though again, not always... it really depends).

Re: appropriate averages. Your case (metered billing) is an interesting one. I
don't see why the mode is necessarily wrong - the next customer is most likely
to spend the amount that is currently your most popular amount). However, in
order to calculate a mode, you likely would want to bin your amounts into
ranges so that $10.11 isn't treated as distinct from $10.12, etc.

You're definitely right about one summary statistic not being sufficient. I
would advocate for visualization and summary statistics, with some estimate of
your confidence in the estimate displayed visually.

------
daemonk
Anscombe's quartet is relevant.
[http://en.wikipedia.org/wiki/Anscombe's_quartet](http://en.wikipedia.org/wiki/Anscombe's_quartet)

~~~
raviparikh
Absolutely – one of my favorite examples about the importance of
visualization, and why summary statistics often don't do a great job of
summarizing a distribution.

------
moments
While I agree with the spirit of this, the conclusion is not necessarily
correct. The mean can be highly informative, but should never be used alone.

Assume you know only the mean revenue and the maximum revenue (but forgot to
measure variance). You could make an extreme scenario with the maximum
possible variance to generate a "worst case" distribution. In this scenario,
all customers either provide zero revenue or the maximum. This distribution
has the maximum possible variance for a given mean and maximum.

Will you be profitable this year? Your chances will be better than the worst
case scenario described above! If higher moments are known (variance, skew,
etc.), more accurate bounds can be found.

In conclusion, the mean can be very useful, especially if higher moments are
known.

------
mildtrepidation
It's trivially easy to say "metric x is meaningless" if you assume the person
interpreting it doesn't actually understand the context of that metric. You
might as well say "metrics that aren't based on a solid understanding of your
business model and user behavior are meaningless." Of course they are; this is
not useful information.

If you run analytics on bad or incomplete information or information that
doesn't realistically relate to something important to your bottom line, or if
you try to interpret useful data without knowing what you should actually be
looking at in the context of the service you're providing, it'll obviously be
meaningless.

