
How VCs See Your KPIs - dpaluy
https://www.nfx.com/post/how-vcs-see-your-kpis
======
3pt14159
> preferably with a dashboard

Nope.

Daily / weekly / monthly plain text email or emailed spreadsheet is way
better. The problem with dashboards is that when the underlying numbers
change, they can change the figures for the past and there won't be a record
of it. If it's in an email someone can bring up this discrepancy.

Also, lots of startups waste a lot of time building a dashboard not realizing
that they're now building two apps. "We don't need login, we'll just IP
whitelist it." Then a VC wants to see the numbers. Then he wants to forward it
to his partners. Now if your dashboard goes down the CEO starts freaking out
about how it makes the company look during their raise.

The rest of the advice is spot on though. Especially keeping the number of
KPIs down.

This isn't to rule out pulling in other sets of numbers, but those should be
done by a Data Analyst and they should be one-time analyses that can be re-run
by the analyst later if needed.

~~~
iooi
> Daily / weekly / monthly plain text email or emailed spreadsheet is way
> better.

> Also, lots of startups waste a lot of time building a dashboard not
> realizing that they're now building two apps.

Setting up a Grafana dashboard is a lot easier than setting up crons that send
emails. Auth is built in, including OAuth.

It's also trivial to have your data adjust to time bounds that you can change
on the interface.

> when the underlying numbers change, they can change the figures for the past
> and there won't be a record of it. Any important numbers should be graphed
> on a time series chart, and you won't have to track down emails and compare
> across different tabs.

~~~
3pt14159
> Setting up a Grafana dashboard is a lot easier than setting up crons that
> send emails. Auth is built in, including OAuth.

Not really. It would take me about 5 minutes to setup an extra worker that
sent emails for an existing app.

Plus, it's a strategic decision to share numbers with a potential funder. That
decision should be made each time, rather than allow ongoing analysis.

> Any important numbers should be graphed on a time series chart, and you
> won't have to track down emails and compare across different tabs.

I agree that important numbers should be graphed on a time series chart, what
I'm saying is that sometimes changes in code can change _numbers in the past_.
If each email has that month's spreadsheet, complete with numbers going into
the past, and someone says "hey wait a minute, I thought churn in July 2017
was 4.5%, I remember because it was our worst month!" Then you can go back to
see what the email said.

------
rabidrat
"And tracking towards the wrong goals can be even more counterproductive than
failing to track anything at all."

The author doesn't seem to recognize that KPIs would be subject to Goodhart's
Law. ("When a measure becomes a target, it ceases to be a good measure.")

~~~
tryitnow
Really? I think Goodhart's Law applies more to complex systems with many, many
actors, like whole economies (which is what Goodhart originally applied it
to).

I am willing to bet that something like "retention" or "net new accounts" are
relatively resistant to GL.

~~~
rabidrat
Maybe in a startup you're right, because it's too small for any kind of
metric-hacking to go unnoticed and become systemic. But it definitely happens
in larger companies; some might say that making the metric of revenue/profit a
target is the source of our capitalistic ills today.

[https://money.cnn.com/2017/08/31/investing/wells-fargo-
fake-...](https://money.cnn.com/2017/08/31/investing/wells-fargo-fake-
accounts/index.html)

------
peterwwillis
> I’m a big fan of making KPIs as visible as possible when the time is right,
> like having big monitors with your KPIs displayed all around the office so
> everyone can see them.

I've been in tech 14 years, and I've never worked anywhere where a manager
discussed KPIs with people on my level. But the companies I worked for also
had billions in the bank, so maybe this just implies that companies of that
size don't need engineers to look at KPIs.

Get rid of the stupid TV dashboards and put up a decent _Monet_. It sucks
working in a physical manifestation of a computer.

~~~
mi100hael
_> But the companies I worked for also had billions in the bank, so maybe this
just implies that companies of that size don't need engineers to look at
KPIs._

Indeed. Things are a little different at a startup when there are ten of you
sitting in one big room and the business lives or dies based on what each and
every person does in that room.

------
manishsharan
Could someone share a few examples of KPIs,excluding the well know AARRR
metrics, tracked by successful startups ?

------
loco5niner
> As a four-time CEO, I had to learn early-on how to...

For some reason, this line put a bad taste in my mouth. Probably an
association with too many bad advice articles on the internet where the author
felt the need to establish authority on the subject using a line like this
rather than letting the article stand on its own merits.

------
Alex3917
> I’m a big fan of making KPIs as visible as possible when the time is right,
> like having big monitors with your KPIs displayed all around the office so
> everyone can see them.

> If they don’t have a daily report or tell me it's going to take a few days,
> the conversation is over.

So he's a big fan of having the company KPIs displayed on monitors throughout
the office, but if anyone actually uses those monitors to get their info on
KPIs then the conversation is over. Got it.

~~~
EnFinlay
Both statements are saying the same thing - KPIs should be highly accessible
and visible.

~~~
Alex3917
He’s saying he wouldn’t invest in any company that doesn’t have a daily email
blast with the KPIs. But if you already have dashboards for everyone then
surely a daily email is a nice to have rather than a need to have.

~~~
citrablue
Daily emails are a great notification method (the point of KPIs is to
socialize them, and some people do better with push instead of pull). Further,
emailed reports are a append-only log of previous reports, which has
additional value.

Whether that's worth the half day to setup is a different question.

------
davidivadavid
So basically set up OKRs.

------
graycat
"Key", in what sense "key"? There needs to be a good reason. Just applying the
label is not good progress.

(1) To go to the trouble to measure something and publicize it to employees,
investors, etc., first need a good reason for measuring that something. E.g.,
why do we measure MPG, MPH, engine temperature, time since last oil change,
..., unemployment rate, labor participation rate, economic growth rate,
inflation rate, prime rate, money supply, balance of payments, mortgage
default rate, federal budget deficit/surplus? For each of these, and jointly
for several of them, we have good reasons, from owning a car to
macroeconomics.

(2) When we seek to measure something, we should be clear on what in principle
we are trying to measure and, then, pay due attention to the accuracy of our
measurements, e.g., reliability and validity.

The OP ignored (1) and (2).

My wife, Ph.D. in mathematical sociology, understood reliability and validity
and much more. Similarly for my brother, Ph.D. in political science. The
social sciences are such _indefinite_ fields that they have to be careful
about what they are measuring, why, what the measurements mean, and the
accuracy.

As a graduate student, Ph.D. in applied math, I understood reliability and
validity, and at least several, likely nearly all, my fellow graduate students
and faculty did also.

The OP is by a VC who wants "warm introductions". Well, likely the people he
knows don't know about reliability or validity, and the people I know do. So,
we won't have colleagues in common with no possibility of a "warm
introduction" in either direction. E.g., one of my dissertation advisors was
later President at CMU. I'm sure he knew about reliability and validity, but
it sounds like we couldn't get a "warm introduction" via him in either
direction. The Chair of my Ph.D. orals committee was a Member, US National
Academy of Engineering, and I'm sure he knew about reliability and validity;
same problem, not much chance of an introduction!

Uh, the author of the OP is interested in "huge ideas". Hmm .... From people
who don't know about reliability and validity? How about measure preserving in
ergodic theory and applications to machine learning and computer security?

My startup is supposed to be of high interest to nearly everyone on the
Internet -- "huge" enough? But to evaluate the work, need to dig into and
evaluate the crucial, core, original applied math _secret sauce_ based on some
advanced pure math prerequisites -- the OP author is qualified to evaluate
such work?

In what sense is he really interested in and able to evaluate or work with
"huge ideas"?

Or, if he invests and we have our first BoD meeting and his big contribution
is to insist that we allocate time, money, and effort to lining the walls with
big video screens with "key performance indicators" changing in real time, uh,
to "motivate" the employees? Uh, might we have a study of the effect on the
employees, motivate them, distract them, or irritate them? My _a priori_ guess
-- at best another case of the Hawthorne effect

[https://en.wikipedia.org/wiki/Hawthorne_effect](https://en.wikipedia.org/wiki/Hawthorne_effect)

