
Are “Better” Ideas More Likely to Succeed? An Analysis of Startup Evaluation [pdf] - jforman
http://www.hbs.edu/faculty/Publication%20Files/16-013_201e071c-9dd8-4838-b5aa-492f4f202822.pdf
======
Brushfire
Full Abstract:

Entrepreneurs face high uncertainty, and often make costly investments in new
business ideas without knowing the expected payoff. This paper empirically
examines whether ex-ante assessment of earlystage startup ideas can predict
their subsequent commercialization. We leverage an entrepreneurship program at
the Massachusetts Institute of Technology in which early-stage venture ideas,
presented in the form of succinct standardized summaries, elicit subjective
evaluations from a large set of experienced entrepreneurs and executives.
Using data on 652 ventures in multiple industry sectors, evaluated over an
8-year period, we find that ideas that elicit more positive evaluations are
significantly more likely to ultimately reach commercialization. We further
show that these results are driven by venture ideas with documented
intellectual capital in research-and-development-intensive sectors, such as
life sciences and medical devices. We find no evidence, by contrast, that
experts can effectively assess the commercial potential of venture ideas in
non-R&D-intensive sectors such as consumer web and enterprise software.
Finally, we find that industry-specific and scientific expertise is not
critical to experts’ collective ability to predict ventures’ commercial
viability.

~~~
tjradcliffe
Their results may be significant to VCs, but likely aren't to anyone else.

They get about a 4% increase in good outcomes, from 23% to 27%, for a 1 sigma
increase in expert positive response. This is significant at the 5% level.

It's enough to say that "Experts do slightly better than chance in IP-heavy
fields", but about 3/4 of those businesses will still fail, and that remains
true regardless of expert evaluation.

If you're a big investor this could make a difference to your bottom line. If
you're a small investor or entrepreneur, it is pretty much irrelevant.

------
graycat
IMHO: Venture capital has its own rules.

Rule 1: Find a startup with traction significantly large and growing rapidly
in a huge market and where, still, the founders are very short on cash and
will sign a term sheet with some really onerous conditions. E.g., the founders
suddenly go from owning 100% to owning 0% and may get back to 60% on a four
year vesting schedule. The BoD, dominated by the venture capital partners, can
fire the founders before their stock is vested.

Rule 2: Claim a lot of items on a long list of what venture capital is about
other than Rule 1. Claim it's about technology, innovation, brilliant
founders, great leaders, changing the world, with the venture firm helping by
using their long experience, "deep domain knowledge", etc.

Rule 2 is 100% noise, just _misdirection,_ to keep people from noticing Rule
1.

Other major parts of our society are really good at innovation, creating it,
evaluating it, exploiting it, etc., but not the venture capital firms. They
won't evaluate innovations. Instead, again, they just follow the two rules.

As in the title, "better ideas"? The venture firms never consider the quality
of the ideas -- instead, they just follow Rules 1 and 2. Or, on evaluating the
ideas, they just let the market decide and then look at traction to see what
the market has decided.

Sometimes they make a lot of money, but on average their returns are poor.

~~~
nekopa
Are VC returns _really_ poor on average? I thought their hit rate (1 company
out of 50 being successful) was poor, but their returns are stellar because of
the few unicorns.

If your conclusion is correct, how are the VCs staying in business?

~~~
didgeoridoo
The trailing 10-year average of VC returns is approximately the same as the
S&P, but with orders of magnitude more volatility.
[http://www.inc.com/kimberly-weisul/venture-capital-funds-
mak...](http://www.inc.com/kimberly-weisul/venture-capital-funds-make-decent-
money-this-is-news.html)

------
Gys
From the abstract:

'We find no evidence, by contrast, that experts can effectively assess the
commercial potential of venture ideas in non-R&D-intensive sectors such as
consumer web and enterprise software. Finally, we find that industry-specific
and scientific expertise is not critical to experts’ collective ability to
predict ventures’ commercial viability.'

~~~
wwweston
> venture ideas in non-R&D-intensive sectors

We're probably moving in to near tautological territory, here. If your venture
doesn't depend in genuinely novel technology with significant R&D investment,
then... its success probably depends on fundraising, marketing, and luck. All
of which don't require industrial or scientific expertise.

~~~
dhaivatpandya
This is not really true. There are plenty of ideas that don't involve a
significant R&D phase (i.e. consist of more or less the implementation of the
idea rather than the research-based refinement of an idea) which can immensely
valuable.

Plenty of examples exist such as Uber, Basecamp, etc. Although the success of
these ventures is dependent on fundraising and marketing, these are certainly
not the only factors and these factors apply to research-heavy startups as
well.

------
unabst
This paper provides evidence for what some might consider an obvious
conclusion, but there is always value in documenting evidence, better yet,
generating it.

The paper says established sectors are easier to predict than radical ones,
just as progress is easier to measure in established sciences, as opposed to
where paradigms may shift. And when faced with a paradigm shifting value
proposition, no one really knows what to make of it until the rubber hits the
road and the model is tested in the market.

But once established, even a paradigm shifter becomes a crowded R&D sector,
no? The iterations are faster, but consumer web and enterprise software do
seem more crowded. Search, social networks, and web technology plays also come
to mind.

What I didn't find the paper mention were how open source and the lack of IP
monopolization as an option could be a factor, and also how the entrepreneurs
themselves stacked up in comparison. For me personally, it would suck if "the
entrepreneurs didn't really matter" (the conclusion asserts better human
capital gravitates towards the ventures with greater potential, but doesn't
say anything about any influences they actually have).

------
louithethrid
I found a rather strange index to rate the success-rate of future projects.

(MentionedInConversation x InfectiousnessToOther) _EnduranceOfIdea_ (
1/TechAffinity)

Meaning, if your aunt talks to your mum about it, and the general idea comes
up once or twice a week and migrates on, even without mentioning the company -
and your mum&aunt is not enthusiastic for just the technology, then its going
to make it.

Case: AirBnB - mentioned once or twice - misspronounced, then mentioned only
as concept for three months - the idea stuck to rent out rooms to total
strangers without big formalitys. Such buisness_idea_memes are easy to scale,
propagate themselves in a way and the technology has basically some catalytic
function to a allready existing small scale buisness concept that a person is
allready familiar with but did not dare the risk previously.

"I could make a small hostel.."

One could imagine this for allmost anything.. Make a app for people who wish
they where at partys and able the enjoy them and are even willing to pay for
that, and connect them with party-animals lacking the money, willing to drag
somebody shy along. Idea could run without your app, but your app propells it
onwards. Voila!

------
ykumar6
I think this study is trying to make statistical predictions based on a
limited dataset. The industry has shown the outliers (or statistical
anomalies) are the ones that result in the biggest companies.

------
hooo
> Using data on 652 ventures in multiple industry sectors, evaluated over an
> 8-year period, we find that ideas that elicit more positive evaluations are
> significantly more likely to ultimately reach commercialization.

It sounds to me that ideas with positive evaluations are more likely to be
funded through to commercialization. Not a very interesting finding?

~~~
jldugger
Just below that:

> We find no evidence, by contrast, that experts can effectively assess the
> commercial potential of venture ideas in non-R&D-intensive sectors such as
> consumer web and enterprise software.

~~~
hooo
Doesn't this support what I said? They studied which ideas reached
commercialization -- not how successful they were once commercialized.

