Hacker News new | past | comments | ask | show | jobs | submit login
The Minimum Viable Testing Process for Evaluating Startup Ideas (firstround.com)
117 points by yarapavan 37 days ago | hide | past | favorite | 30 comments

This article really vibes with me in general and is just a great piece of content.

I was a little disappointed in the examples though, running a course is an MVP and delivering a meal is an MVP. MVPs don't need to be software, in these cases, the software is actually definitively not a minimal approach, proven with a non software example of the MVP.

So is a test viable? I have asked people to pay for things I hadn't built yet and didn't intend to build if they wouldn't prepay, is that an MVT and not an MVP? Are waitlists and crowdfunding campaigns MVTs?

As Sprig was shuttered after 4 years and $56 million in funding, it doesn't seem like it was a good test.

Being able to cook meals and deliver them doesn't seem like a test. Thousands of restaurants do that every day, and national chains do it at scale. The test needed to be is there a market at the price point necessary to sustain the kind of meal he envisioned and I guess the answer to that was no.

What would have been enlightening in the article are examples of negative tests -- something one of the firms he was involved with tested and failed at, which led them to reevaluate.

Further down in the article he mentions a failed test for a tour marketplace where locals will show you around, but I agree that I would have liked to have seen more negative result examples than just the one.

> running a course is an MVP and delivering a meal is an MVP.

Yes I too was hoping for something a bit more radically different to my existing definition of what an MVP is.

I definitely agree that it's easy to over engineer MVPs - I'm guilty of that myself, a lot, but many/most of the "MVT's" described here really just sound like non over-engineered MVPs.

Nevertheless, the article is still useful as a reminder to really pair it down to the absolute bare essentials at the beginning. So essentially what we end up with is a "minimal viable MVP"!

I was curious to see no mention of doing any kind of research via some sort of survey - to not necessarily canvass potential customers on whether they'd buy the product, but more around the problem space, what problems they currently have etc. To me this seems like quite a low-hanging fruit to help validate ideas?

Paying someone to plan a trip for you is definitely not a product, so not a MVP. Running a course is closer to a full-fledged product (more than a MVP) for a company that sells courses to students, but Maven is a platform for creating courses...

By the definitions laid out in the article waitlists and crowdfunding campaigns would be MVTs to address marketing or "building something people don’t want" risk.

That’s a long-winded way of saying “try to test a new business hypothesis before building anything”.

None of this kind of testing that VCs love and founders do is real testing, statistically speaking. Just because you can sell a bunch of lemonade at the street corner on a hot summer’s day doesn’t mean that you should be going into the lemonade business…

Conversely, Jeff Bezos may have put up a pseudo website when he started Amazon to “test” whether ppl wanna buy books online. After three months maybe a few ppl would’ve bought a few books, but that would’ve told him nothing about whether it’s a good idea.

Yes, lots of startups come out with crap products that no one needs, but I’d argue that much of it isn’t as “testable” as articles like this (which are written ‘from hindsight’) make it seem.

There are plenty of large companies that test extensively and then demand is either spurious or not sustainable. Other products “grow” on ppl, or some celeb makes them fashionable if otherwise, they would’ve died a quiet death.

At the end of the day, these “tests” are just a good persuasion technique for when you need to raise money, but they lack scientific basis, and they won’t “derisk” much. But they make things sound salient and rational like all persuasion techniques, and for that, they certainly have their place.

I just don’t think this needs a 5-page blog post that is abstracting and “frameworkizing” something pretty basic, obvious, and not that interesting.

Oh, and I still don’t know wth a “cohort based course” is supposed to be. Sounds to me it’s just a regular real-time class taking place online?

MVP vs MVT is a false dichotomy. It is made up by the author to sound smart and be convincing. From the article, describing what it calls MVT:

“Build an initial product to bring all of your insights together and test them with your target customer.”

Yeah, that initial product is the MVP.

Maybe we read different articles, but "Build an initial product" is what he said to do _after_ the hypothesis testing.

I think this is often where people are confused by MVP. Your minimum viable product isn't (necessarily) your initial product. It's the product you build to test a hypothesis.

This is why the article read fairly false to me, not that there wasn't the odd nugget of value, but coming up with a new MVT, is just adding to the confusion.

I quoted directly from the article. MVP is built after the hypothesis testing, but it’s still a test, a test of the independent hypotheses combined into a material product.

I think the point is that the P is not necessary if the T's fail. So, you're creating a stop loss for your time/money/blood/sweat/tears

> I quoted directly from the article.

You did, but your quote comes directly after he said:

"Once you’ve finally tested enough hypotheses to have more confidence about your product viability, then go to the next steps:"

The MVP is the next step and it distinctly different from what he calls an MTP.

An MTP is not a useable product, it doesn't have a login, stack or maybe any code at all.

A Minimum Viable Test works better when you use Key Risk Indicators (KRIs). KRIs are a lightweight tactic to list your risk estimations, quantify them as possible, and aim for mitigations. KRIs are very useful (in my experience) to accelerate projects because they are quick and easy to write and discuss, and they guide teams toward the best Minimum Viable Tests.


I clicked through because it sounded interesting (and still does) but without any kind of examples it's hard to get started.

Thanks, I will add these examples now:

System risks e.g. unplanned unavailability count (UU#), mean time to recovery (MTTR)

Marketing risks e.g. customer complaint count (CC#), net detractor score (NDS)

Financial risks e.g. unbooked letters of intent (LOI$), investment value at risk (VaR$)

Process risks e.g. dropout rate (σ), error ratio (ε)

I think all KRI accomplish is force you think about risk systematically but in terms of actual numbers they are still just guesses.

I read this when it was on Twitter. To be honest, it doesn’t really sound very different than the MVP method. The whole point of the MVP is that it is minimally viable so the founders can test around it before building something full scale that the customer wants.

the main difference is that a "minimum viable product" nowadays is basically just the product itself. It's hard for founders to distinguish between MVP and P when companies have such simple, growth-minded core ideas. Instead, this essay describes how to try as hard as possible to not build a product at all, and how that can lead to strong insights that precede engineering work.

I find all this mvp stuff just over complicating everything and just complex fluff.

What you need to know is if you can reach at decent scale people that will pay enough to give you a fat margin of profit for your product.

So you need a product that does the job, you need a scalable distribution channel and you need customers to pay enough to give you 60%+ margins.

That’s it. There’s no way around it. With both physical and digital products you can let people pre order them.

You need distribution-product-price fit. When you achieve that you can pour gas on the fire.

TL;DR Just create a damn landing page connected to stripe and take pre orders and make sure the price gives you lots of margin. Then spam The distribution channel. That’s it.

It's over complicated to you because you're already thinking about making something small and iterating. It's becoming second nature to a lot of people, but there are a lot of people who still think they need to make the "perfect" product.

How do you figure out what problem to solve and the distribution channel?

You can copy products that are growing fast or solve your own problem or better, start backward from the distribution channel. Like there’s a reason that YouTube personalities sell mostly merch, that’s what’s good to sell on YouTube as a creator.

The product is the easy part.

The hard part is the distribution channel. How to reach lots of people that would pay for something for cheap. That’s the secret you have to uncover as it’s semi unique for everyone. Airbnb spammed Craigslist and Hotmail spammed emails w their hotmail signature.

Seems to me there is a continuum of vaporware-->MVP-->real product, and the art of it is to know where on that line to start.

As a repeat tech co-founder, I easily fall into the trap of being a perfectionist & a complete-ist, always wanting to add one more feature or refactor one more poorly designed aspect of the project. So I appreciate this article as it pulls in the other direction.

However I think doing what he calls MVT's -- basically sub-standard, feature-poor, non-scalable MVP's -- can lead you astray, or simply fail to properly test the hypotheses in question.

In the end it comes down to judgement and experience. I've taught myself to put time-to-market at the top of my list of priorities, but it is still a struggle between wanting to 'get it right' vs missing the opportunity or chasing the wrong assumptions.

I love this. It puts into (better) words my long-held belief that ideas do, in fact, matter.

Startups live and die by their ideas. The nuance is that ideas can be detailed or abstract, and those details can be some combination of right/wrong and relevant/irrelevant.

I've got back and looked at some of the early pitch decks of my current unicorn employer, and they're shockingly accurate. The founders / early team have been saying the same thing since before launch. It's unreal how fast they figured out the core value prop, and so much of the growth of the company is living up to the promises of that early idea.

Figuring out what really matters needs to happen early, and it's something that founders and early team need to internalize as much as possible. If you can get that deep understanding of what needs to be done, quickly, the company's odds of success go up so, so dramatically.

I am starting to use different approach....


I) Build community - build something useful for segment you are interested in even it has no big future payout. Or forum/blog/discord/instagram/fbGroup/emailList/twitter/partner with existing influencer... just have access to audience. Talk to them, recruit few to build out v1, blast v1 to all audience.. boom you are in business.

II) Just brainstorm dozens of crude ideas about problems some industry/professional/consumer might be having. Then pick problems you like most. Then find existing solutions and see what's happening. Once you have some insights, you will have lot of questions. Ask existing users all of them. And you will start to learn about industry very nicely.

III) Try to partner with consultant who has helped many businesses. Or try to partner with veteran in industry that is lagging in tech. (more opportunities). Make him significant partner for just giving insights and providing introductions and connections.

IV) If all else fails build anything. Even if it looks a copy of existing thing and inferior. It's likely you will start to find users who still want to use your product, lake is really big. It's not winner take all. But what will happen is you will start making in ways into this new industry and realize why these businesses/users are unsatisfied with existing stuff.

Remember all companies have limited resources. So they always go for segment/problems that yield highest return. They may start with low ROI segment but move onto greener pastures. ie. individuals -> SMBs -> enterprise. FB -> a place for college students -> place for families -> place of businesses.. now it's no longer a place to discover your college peers and share things with them and them alone. Everyone and their grandmother is there. Early adopters/segments are always left in dust. So there are always unsatisfied segment of customers.

V) ME TOO in new markets - Lot of world is still offline and/or has very slow internet. You can be first there.

As others are saying, distinction seems right but practically it results in the same outcome. Test riskiest assumptions with least amount of resources and find problems first. If you tend to write code then use no-code tools to avoid the trap.

"MVP" is now a meaningless word that is used to mean "initial release" of something. The "minimal" part has been entirely forgotten. Silicon Valley people are incredibly stupid.

I think this aligns with the Lean Startup process and has a lot of merit. I may very well have accidentally used a similar process to develop Mach 9 Poker. I sat on my original idea and iterated it in my head and by conversing with live poker players. After a few years, I finally had an epiphany and when I tested that idea, the players came around as very curious. I'm still modeling it and building it, but the path is solid (so far).

There is so much misunderstand about the definition and the purpose of an MVP. It’s not a simplified version of your final product experience, it is something that allows you to get the maximum amount of validated product value hypothesis with the least amount of effort.

The "launch fast, fail fast way(launch an MVP and iterate it)" is to optimize for the company and investor, not the customer.

I got an idea and I want to test it using a landing page to collect emails with a waiting list. I know some backend and frontend but not an expert. Should I use some kind of Wix or other website builders to do it quickly and if yes any suggestions?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact