All successful startups are lucky, but they're never just lucky. If they were, we could save ourselves a lot of work reading applications and doing interviews. We could just pick randomly instead. If we did that for one batch, you'd really be able to tell the difference, believe me.
I would absolutely love to see a double-blind randomized experiment with controls for YC applications. In one batch, randomly and secretly assign CONTROL or EXPERIMENT to each application (in whatever proportion you are comfortable with). Of the CONTROL batch, do regular reviews. For the EXPERIMENT batch, randomly accept the same percentage of groups as the control. Then have people outside of the reviewers try to label each group correctly at different stages and see how they do. These people could be: YC alums, YC staff not on the interview loop, VCs making investments, outside experienced CEOs, successful startup founders.
For full double-blindness you could even do interviews with the EXPERIMENT group, then use a program to automatically either use the interview result or not. This might be aggravating for reviewers, though, seeing their work wasted by random choices.
How can you really do any experiments on this kind of shit, when the real winning investments exist in front of a zipf curve?
You get one or two huge winners out of hundreds, and they are so huge that they are bigger than all the others combined. The investors care most about getting these rare winners, and if YC does any kind of controlled study on who becomes these winners, I don't believe there will be any statistical significance to speak of.
Sorry, statistically speaking the difference in results of interviewing and not interviewing are insignificant.
I have a feeling that eventually there will be a Vangaurd of startups that will massively outperform traditional VC/incubators due to reduced management fees similar to index funds vs. regular funds.
On the one hand, it's probably true that experts overweight their own ability to pick winners. My guess is, PG is likely overestimating the ability of YC staff to actually identify good patterns simply because it's easy to draw conclusions from just a few datapoints but it's hard to draw statistically significant conclusions that hold up out of sample. However, I doubt that a startup index fund will outperform the best VCs and incubators simply because good deal flow is key, whereas in publicly traded markets, everyone has access to all deals. Having said that, I have no doubt that a VC index fund would outperform many of the VCs that are not that good
There's also a question of how do you define the index fund. The S&P is pretty non-ambiguous. How do you define which startups are in the index? By the time they're definable as a real business, they're no longer early-stage startups.
So pg's years of experience getting better than random results is irrelevant and you'll only believe him if he does a long experiment (that would cost him lots of money if it's not just luck) to prove something he already believes?
It's unclear whether he's getting better than random results. You're assuming the consequent.
It's well-known that the most successful startup in a group of startups is likely to earn more in profit than all the rest combined. We're here on HN talking about PG because YC has been the most successful of all incubators. However, there's always going to be one incubator that significantly outperforms all the others as a result of the distribution of startup success.
I don't know enough about how well PG's investments have done to make informed judgments, but only a very select few people are likely to have that info. I suspect those people have also done their due diligence to figure out if YC is a value-add compared to other incubators (it should be possible to do this without a convoluted double blind experiment), but anyone who's not them really can't say for sure that that's the case.
This is oversimplified. In many cases, it is the conjunction of a reasonably strong prior belief and the absence of preexisting "proof" that motivates a scientist (or mathematician) to try to prove something in the first place.
The key is to perform proper Bayesian updates in the face of evidence in either direction; if you do, as long as your prior wasn't totally insane it doesn't really matter where you started.
Launching a successful startup is luck BUT selecting entrepreneurs/startups who have luck is no random. You can definitively find some metrics to evaluate the current luck of a startup and expect that the founder has enough skill to continue to maximize luck in the future.
It'd be unfair to great startups in the experiment batch -- not that it's pg's job to make life fair for people, but it'd put a big dent in YC's reputation. Why spend time flying out there if there's a chance your interview doesn't even matter?
You can still do a pretty similar experiment. For instance, startups 1-50 get into yc like normal. Startups 51-100, together with 50 randomly selected startups get grouped together and tagged as "the bubble group". Meaning that they were on the bubble of getting into yc. Pretty high honors. However, they don't get any access, consultation, etc. to yc itself. Then after a few months you unblind which were highly selected and which were random, to see who does better.
Can't you pick the best interviews, then hold your experiment with random groups as long as you mix in a the data from the interview groups in such a way to build a representative sample?
We would only be able to conclude that it's unfair for "great start ups" if the experiment showed that random acceptances performed worse than the interview process.
If it were just luck then the companies rejected by YC should statistically perform as well as those selected for interviews which should perform as well as those offered spots in a given batch.
Sorry, not an expert and I have never been an entrepreneur so forgive me if I am misunderstanding, but doesn't YC give them money? In order for this statement to be true either (1) having startup seed money has no bearing on success or (2) YC gives the same money to people it rejects as to people it accepts.
The amount of capital provided by YC is trivial...a rounding error relative to the value of a successful company. But more relevant is that YC almost certainly has some meaningful data on the companies it rejects.
So PG's claim could very well be supported by proprietary information.
Trivial once the company is successful, but isn't the purpose of the money to ensure that the founders don't have to give up and get "proper" jobs before the company succeeds?
I think you're interpreting the idea of luck incorrectly. Luck in this context is simply the phenomenon of being one of the start ups with high success. To say that success depends solely on luck means that the success of any startup is simply rand(). That doesn't mean all start ups are equally successful.
Giving selected startups money confounds two factors: the selection criteria and the extra funding. This means that any differences in the success rate can't simply be assigned to the selectors choosing wisely.
Unless being a part of YC adds value which I suspect it does. Impossible to really test this unless YC would provide the same level of support and resources to other companies.
Depends on what "luck" entails. Success is measured by something--for the sake of argument, let's sake acquisition price. But there have to be proximal factors; to be a bit reductive, no one is arguing that large companies pick randomly for the world of startups and them for millions of dollars. Perhaps the most immediate proximal factor is number of users or revenue or...whatever. You wouldn't say, "if it was just luck, companies would preform just as well whether they had users or not".
So the question with luck is how far down the causal ladder you start considering things inputs (i.e. things which may or may not influence the outcome) instead of outputs (i.e. things which might be the result of those inputs or of luck). So with YC, is that getting or not getting into YC is an input variable that we could check against success, or is getting a YC spot one of those proximal factors, and the luck was just in getting that, at least in part?
So the "is it pure luck?" question isn't very well defined, but the GP's argument might still be useful in answering it. Moreover--larger questions of luck aside--it would be helpful in determining whether the YC selection process is actually useful. But probably only if they didn't tell anyone about it.
I think the OP's point is after filtering out obviously non-successful startups, the chance of success is potentially non-correlated with other "traits", "characteristics", or "metrics" etc. that people often associate with successful startups/founders.
It would be an interesting experiment if YC lower the acceptance threshold to say twice the original batch, and randomly pick from that bigger pool, and see if there's any difference between the ones that didn't make the original cut and those did. (A/B in its finest :)
In order for us to "believe you", you would need to test your hypothesis at least once - for example, by selecting a small number the startups you invest in at random and see how they fare compared to the other startups. Keep in mind that most stock pickers would respond exactly like you even though their choices are rarely (if ever) better than chance.
If I was to walk for the first time into your kitchen and tell you that your hotplate was turned on, would you not believe me until I'd also experienced your hotplate turned off? You would discount my previous experiences with hotplates, because maybe, just maybe, your hotplate is hot enough to boil water when it's unpowered?
The idea that the null hypothesis is the only source of information needs to die.
But on the other hand, there are well-known cognitive biases and counterintuitive facts that can cause people (including extremely intelligent people and even people aware of the biases and facts) to believe things that scientific experiment can show to be false.
Figuring out that a hotplate is turned on can be done though a very simple and direct observation, so most likely I would take your word for it.
Figuring out what makes people successful at starting companies is extremely complex. And without objective evidence, no level of expertise, be it PG or other VCs, will convince me to take someone's word for it.
It's not luck for you, it's luck for entrepreneur.
Your job is to select people who were lucky enough to meet the right people, meet the market, had luck to have enough traction, lived in a country where he could spend time learning and so on...
But when you are entrepreneur, you have to maximize your luck by maximizing number of opportunities.
Spot on. I think it's important to mention the context of Taleb's philosophy. You have to keep in mind that in certain jobs, like trading, it is fairly plausible that successful traders are plain lucky because all traders eventually blow up. In the startup world, this analogy doesn't hold for a number of reasons. If you are genuinely skilled, you will succeed. Perhaps not on the first try, but you will succeed eventually.
Some skilled people succeed, therefore all skilled people succeed eventually?
Alternatively, it's a tautological statement that defines "genuinely skilled" people as those who succeeded, therefore unsuccessful people must not be skilled.
I was discussing a somewhat related subject yesterday, on an in-company product development workshop.
For a DropBox-like product, feeling safe you won't lose your files is the baseline. If you cannot reliably ensure the files won't get randomly deleted, you don't have a product. Once you have a product, the value you add gets increasingly more subtle - better metadata capture, nicer ways to organize your files, an outstanding slideshow or music app - and what determines your ultimate success may feel completely random and totally unrelated to the original way you defined your product.
It would be interesting if we could measure luck based on previous YC batches.
Basically, the random choice is pulled from the pool of all the talented and productive people. So the best we can do is try our best to continuously be talented and productive, and hope we get lucky one of the times.
More interesting than double-blind test is if each of the yc partners play fantasy startup.
a) each pick his/her top 5 yc companies per batch that they think will be most successful (and main drivers why)
b) each pick his/her bottom 5 that they think will be least successful (and main drivers why)
c) each assign the value of each of the 5 most successful companies 5 years from now.
(and build a bridge as to how that value will be achieved).
and then run some fun numbers on your predictive capabilities.
when i ran "c" for my private equity firm on all investments made over 20 years in various vintages and industries, i found my answer as to why my job of building complex financial models was pretty much worthless number painting. very low predictive ability in value drivers and financial projections being met (these are later stage businesses, too). also, i realized that i should get out when every newer vintage had a larger % of the driver of value stemming from leverage (i.e. financial engineering) than ebitda outperformance.
This has been on my mind lately as well, for some reason. That, and the point of this story http://sivers.org/horses which is basically that even after the fact, it's really hard to know anything[1].
I don't know if it's just from growing older but I feel a lot more humble when looking at peoples ideas and actions than earlier. I literally feel that I have no clue whether they will succeed or not whereas I used to quickly get quite strong feelings in one direction or the other. In one way, this feels more mature and zen but at the same time, it annoys me a bit. I feel that conviction is the trait of a leader and something I need to possess if I want to convince anybody to engage in my projects. Maybe I'll just practice faking it and hope they didn't read this comment :-)
Edit: [1] One of my favorite examples of this is in Felix Denis' book How To Get Rich where he describes that the problem with Apple is Steve Jobs. This was written 7 years ago when Apple was struggling and it looked like Steve Jobs was driving it into the ground with his stubbornness. (The book is excellent by the way, in spite of the tacky sounding title)
Thank you for posting a link to that story. It reminds me a bit of Boethius:
"It's my belief that history is a wheel. 'Inconstancy is my very essence,' says the wheel. Rise up on my spokes if you like but don't complain when you're cast back down into the depths. Good time pass away, but then so do the bad. Mutability is our tragedy, but it's also our hope. The worst of time, like the best, are always passing away."
My intuition is that it's very random. A business professor and economist (Luigi Zingales) once commented that lots of firms tend to find new ideas all at the same time. When the market is ripe and the arrows point in the same direction, everyone seems to find it. Execution is important, but a lot of randomness hits the process too.
One important point to realize for the author is that games are not services, they are works of art. As such, you can't say this is an incredible game with the same objectivity as this is an incredible service.
Ryzom was a great game, but MMOs are built on one thing: atmosphere. Blizzard made WoW precisely because WarCraft was its longest running and most popular brand to date. Blizzard already had a massive fan base of WC fans ready to play WOW. The lore was considered a masterpiece, which immediately immersed the players in the world before they even played it.
With services, things are much more objective, because we don't care about branding. Using a service like Dropbox is not a deeply emotional experience, but it does work very well. Apple is one of the few companies that can instill emotion into its services and hardware, and that is due to their marketing and design philosophies, which are considered the best in the world. However, this is nowhere near the level of emotion that art invokes.
As such, I would advise the author to carefully distinguish between art and service. When making art, you should never make it with the expectation that others will like it. You should make it such that it satisfies your own tastes. If others like it, you know that your taste is shared. If not, your tastes are unique. It's a win win.
Don't regret making works of art. It is all that succeeds us when we leave this realm.
>Don't regret making works of art. It is all that succeeds us when we leave this realm.
Scientific discoveries also persist after your death. While making art is fine, it's hardly the only thing that people are known for.
On top of that, practically any action can leave a lasting impact, even terrible ones. So "effects persisting after you die" isn't really a way to determine value of your actions.
Yeah, any member of animal kingdom can rationally analyze situations and make logical decisions based on facts, but art, on the other hand, is unique to humans. </s>
> When making art, you should never make it with the expectation that others will like it. You should make it such that it satisfies your own tastes.
I don't know, to me both are businesses and have to deal with things such as user acquisition, user retention, average revenue per user, etc.
We've read about the author releasing the game, but don't know the back story about user acquisition - what channels were used, what was the performance of those channels, what viral mechanisms were put in place, what A/B tests were run, how user stickiness was determined and improved upon.
Behind every "luck" story there's some dude or team of dudes whose entire job is user growth.
Critics said he won many battles simply because of luck; Napoleon responded, "Give me lucky generals," aware that "luck" comes to leaders who recognize opportunity, and seize it.
Bizarre - the last link I was on was a random ESPN quiz with UVa star Joe Harris in which that quote was one of the questions. (I realize this is a pretty meaningless and useless comment, but seeing this quote 2 times in less than 10 seconds was too much not to mention).
Is it bizarre? Given the number of people on HN, the number of things you're likely to read -- it seems inevitable that "bizarre" coincidences will certainly happen.
I'm unable to find a definitive reference, but Gary Player (golfing legend) is also attributed with the quote: "The harder I practice, the luckier I get." Perhaps that's why it was on ESPN.
Pretty sure George Simmons has this quote of Player's on one of the chapter pages in his "Precalculus Mathematics in a Nutshell" book.
The upshot is that its origin is unknown, but goes back at least to a Walter Winchell column from 1949. Gary Player used it and became associated with it. The attribution to Jefferson is nonsense, of course.
It was just because Thomas Jefferson said it and Joe Harris goes to the University he founded, it's just bizarre that I saw it 2 times in less than 10 seconds.
As a former player, and customer service representative of the game, i can say confidently that the reason Ryzom failed had nothing to do with bad luck, and everything with bad decisions and incompetence.
They had no qualms about changing large parts of the game. The prime example being that they changed the experience distribution in the game such that where it was first that any large group of players could play together and cooperate, after beta it was changed such that only squad-sized groups could cooperate and squads being in the same area only meant difficulties, since experience could not be shared anymore.
On the technical side they had a lot of terrible code as a result of much of their 50 man staff being students doing an internship for free there. They also had a very bad attitude towards volunteer support staff, where, for example, an issue on the web site would be reported, the dev in question would say nothing, and when asked five minutes later would say "What problem, i see none?" after obviously having fixed it.
Lastly, they also had a very bad attitude to players. The game had guilds and an ingame browser (kind of hidden, since it was the backend for 90% of the game UI), so some enterprising players had figured out how to use that to provide guild forums. The result being legal threats.
As much as i love the game still to this day, and still hold on fondly to the screenshots and memories i collected in its wonderful world, it wasn't luck that killed it. It was Nevrax' incompetence and shortsightedness.
Well, according to statistics (see Kahneman) CEO 'performance' has very low correlation with perceived CEO performance. Also, past success has low correlation with future success. In other words, there isn't really a 'CEO skill', or a 'successful manager' skill. In other words, company success is not repeatable given the same set of people. In other words, yes, luck is extremely important.
There is no moral of the story other than: failure does not mean that you didn't do well.
"There is no moral of the story other than: failure does not mean that you didn't do well."
While this lesson is pretty well internalized, I think there's some difficulty with the converse: just because you succeeded, doesn't mean what you were doing was right. Life is contingent.
Yes, we (should) all know that but somehow we are still deluged with books and blog posts from these lucky few telling the rest of us how to do it. Worse yet too many are taking it as gospel when they really should just do it their own way.
We are deluged by these narratives because they are easy to understand. Same thing with a lot of money is synonymous with success; money is easy to quantify.
When we start talking about randomness and luck, my sense is, that people begin to feel uncomfortable because it makes the world a much more less understandable place.
Are those statistics for startups or Fortune 500s? Because the way I see it, at a small scale you are much more in control of your own destiny simply due to high agility and low communication overhead.
I know it's not the point of the article, specifically, but it's very difficult to draw any startup conclusions based on a game company. Games are notoriously difficult to make into successful products, even more-so than your average startup.
That said, I do think the author has a lot of good points. Surely there is a portion of "randomness"/luck, but there's also a healthy dose of focus & determination that make things work.
To play a bit of devil's advocate, I do feel that in the last few years the idea of "success" has somewhat diverged from what it used to mean. Is Twitter successful? To their founders & early investors, certainly they are. In 5 years, will Twitter still be relevant & a worthwhile investment to their millions of new stockholders?
Was Zynga "successful"? Again: to early investors, they were wildly successful. They were able to dump their shares onto a greater fool.
To me, those are two examples of "lucky" startups. A startup like Salesforce is less "lucky" and more a well-executed product. It's not as sexy as a Facebook/Twitter/Zynga, but it's a solid business addressing a specific need. It isn't reliant on "scale massively and figure out a sustainable business model later", which to me is what the aforementioned companies are.
To paint a picture, some startups have fisherman tactics: throw as wide of a net as possible, catch as many things as possible and some small portion will be valuable. These would be startups to whom luck would seem to play a larger role (in my opinion).
Others are more focused: identify a specific set of targets to go after, and hunt them down. This would be more like Salesforce. To me, these are the startups that don't rely as much on luck.
I agree with you, I don't know the Saleforce history but I'm quite sure that at the beginning of the company, they had lot of luck. I'm sure at least 10 other companies tried to address the same problem on the same market and failed.
Execution is mandatory but lot of the execution is also based on lot of uncontrollable parameters, hire the right guy at the right time.
Look at Evernote for example, they were almost bankrupt even if the execution was good, the market here and so on. It's a pure luck if the are still there and so strong.
Have you heard of the concept of increasing your luck surface area? If my memory serves me correctly is that all of the work, networking, etc. you do the more opportunities you have to get lucky. You can't have chance encounters in your basement. You can't stumble onto investors for a product that doesn't exist.
I think it goes along the lines of the quote from The entertainment industry that every overnight success is a result of years of hard work (paying your dues in the form of acting/music lessons, community theater/coffee house performances, tons of auditions).
The game industry is a special case because it is at an intersection between technology and entertainment. Success is very difficult to predict or repeat. You can't just fill a need in a niche because any other game, really any other form of entertainment, is a substitute for your product.
I used to think I was really smart and that my intelligence would carry me when I finally set out to start a company.
Wrong, while I still consider myself to be smart, I don't think I'm savvy enough to rely upon good decisions to carry me. There are plenty of people smarter than me that have failed in their ventures. So, I've reverted to a brute force strategy coupled with an unrelenting resolve to see projects to completion.
Just lucky? Not just. When you build your companies with a willingness to change, frequently self re-evaluating, you will amaze yourself with how much you grow as an entrepreneur. It is ridiculous what I know today when compared against my assumptions 2 years ago.
When the author was talking about the failed MMO, I took him at his word when he said his team created an amazing game. So the point of failure is obviously marketing. Who knows what could have been if those game developers had a comparable marketing team in their corner. What if they launched a beta 1 year into development to help develop a core user group as well as aid in gamer feedback which could then be redigested by the marketing team to help them create buzz.
I love this rumored quote assumedly from Jack Dorsey "It only took 10 years for me to become an overnight success." The entrepreneur that doesn't die is hardened and polished in failure over a number of years. When luck finally strikes, he will have the tools, experience, insight, vision to take full advantage.
I'm 2 years into this journey, hopefully I'll find luck in year 3, and hopefully I will have grown enough to take full advantage of it.
You missed the point of the article. Ultimately, the universe is unfair. No special insight, ability, breakthrough, or perseverance is sufficient to change that.
I didn't say anything about fair. I didn't even say that you don't need luck. I believe you do need luck. But, you can remove a lot of your dependence on luck by getting better at the game over time, as well as by playing a lot. Just ask anyone that plays Texas Hold'em.
Want to score more? Get your number of shots on goal up. Luck is just the positive result of chance being realized. Chance to succeed vs chance to fail. So as long as you don't care about the number of failures you stack up, you just keep rolling the dice until you succeed.
Perseverance is sufficient as long as it is accompanied by adaptability. You simply need the ability to recognize your strengths/weaknesses/personality type and play the game accordingly. Either invest in improving your weaknesses, or spend time networking to find a partner to complement them. Even better, pick an idea where your weaknesses will not be a factor.
What makes startups so hard is all the opportunities to fail that come along as you try and build something new. Every month, or every week, there is a new thing that wants to kill you. Sometimes these things have nothing to with the company. But, there are sooo many ways to mitigate these problems and you learn how to do so better and better the longer you play the game.
If you'd like me to go into some more personal details to show some examples, just let me know.