
How to validate your startup idea quickly - amanjaincorp
https://amanjain.substack.com/p/how-to-validate-your-startup-idea
======
andrewstuart
With all due respect, a Facebook person's thoughts on validating a startup
idea are pretty much irrelevant.

Whatever thing you are building at Facebook starts out with 10 hurricanes of
wind behind it.

Startups begin with all the wind of a baby blowing a raspberry.

~~~
jerzyt
Yup. The author made a poll of fellow FB employees. As if that is even
remotely representative of the overall market of the 25 to 44 year olds. Just
consider the average salary of the polled population vs the general
population.

~~~
amanjaincorp
Poll of "fellow FB employees" could be replaced with spending ~$100 on Google
Ads and using a certain targeting criteria.

~~~
forgotmysn
but its not

------
irjustin
A number of the comments are saying this is bad because from Facebook, numbers
are unrealistic ($100mm?!), signs tell you nothing.

Personally, I've done startups longer than I want to admit (10+ yrs). If I was
to start another one, the play book would look basically like this article.

Idea/Thesis, Hypothesis, Target, Test and some where later - build.

The problem I had when I started and why so many founders get it wrong is the
steps usually look like - Idea => Build or just ... Build.

There is a lot of good advice here. Sure the numbers might appear insane (just
replace, $1m, $300k, $300), but don't let that stop this from being a good
framework. The goal is to remove as many assumptions about what you're
supposed to build as fast as possible because chances are, more than not, the
assumptions are wrong from Idea => Build. This is a good way to validate a
number of assumptions and it's only just the beginning

The only thing I would have added is - ask for Money.

Ideas will iterate very quickly (or die) once money is involved.

~~~
onion2k
_Idea /Thesis, Hypothesis, Target, Test and some where later - build._

This is a common strategy if you just want to 'do a startup' but it misses the
_need_. Every successful business I can think of started with a founder
needing a product to exist. All the startups I've been involved in have been
products I wanted to exist because I needed them myself, or I had a close
relationship with someone who needed them. Turning them in to something that
could make money was a key goal, but I'd have built _some form of them_ anyway
even if it wasn't.

If you're just thinking up things people might want and throwing up a landing
page to see if people claim they're interested I suspect you're going to get
bored quickly, and you'll move on to the next _even better_ idea very quickly.

That's not to say it _can 't_ work if you're making something just because 300
people signed up to your Launchrock page, but I think the motivation to
succeed won't weather the storms that your startup will face if that's the
_only_ reason you have to build something.

~~~
mpfundstein
the whole Lean Startup methodology is snake oil IMO. it is used primarily by
consultants to lure big corps into "innovation methodologies".* While its
certainly an improvement over status quo, results were mostly meh anyway. you
just cant force good ideas

the best startups are the ones mentioned by OP. visionary founder who needs
that stuff himself.

* i was director at a small 7mm/yr consultancy and did precisely that

~~~
webmaven
The main point of lean startup isn't to have better ideas, it is to identify
_and discard_ bad ones faster, and focus your limited resources on the wheat
among the chaff.

------
codingdave
Maybe it is because I'm older, but whenever I see this advice to build a fake
landing page and see if people sign up... I recall that selling vaporware was
reviled in the 90s, but somehow now it is considered a wise business tool. It
still feels scammy to me. If you have a market in mind, but don't have people
in that market to talk to directly to validate your idea, you are aiming at
the wrong market. Otherwise all these techniques prove is that you can collect
emails, not that those emails will actually become paying customers.

~~~
codysc
You're not alone. This has always felt scammy. It's a little more gray-area
than some of the traditional vapor-ware which was more about driving
competition out of business by large players, but still something I try to
avoid.

~~~
nogabebop23
I'm older too. Maybe the issue is if the central premise is vapourware or
components/add-on features So maybe if the author automated the metadata
collection, ran some filtering on it and then had a fitness pro sanity check
the results? I agree that this oft-recommended "idea validation" process
sounds a lot like this scene from Silicon Valley, without the clarification at
the end:

[https://www.youtube.com/watch?v=Q1QPXyebhiY&t=8m26s](https://www.youtube.com/watch?v=Q1QPXyebhiY&t=8m26s)

------
whatl3y
Even if slightly unrealistic, I actually think the nicest part is the back of
the napkin math and derivation of the numbers to give you context for how big
a market is and what you’d need to attack to meet your goals.

I do however have an issue with the wizard of oz method of validating your
product. It’s not because of the perceived ethics and I totally understand too
many people today build something before ever knowing anything about whether
anyone will use or pay for it. Hopefully I didn’t miss it but nowhere does the
author discuss the amount of time/people/money it would take to build a
working MVP, not to mention how much of your time will be tied up from working
on said MVP if you’re oz (i.e. the man or woman behind the curtain until the
MVP is built). This is important since it drives how you (or at least I) would
move forward: Do you need to quit a full time job and spend 3-6 months 60+
hr/week building your MVP, can you build an MVP in a few weekends, would you
need outside help/money to build an MVP, etc etc.

The example in this post mentions a recommendation engine of sorts for
individualized fitness programs. This sounds like a fairly high amount of
expertise in the field of fitness, some possible AI with trained models
sprinkled in and a good amount of curated fitness routine info or I guess
individual exercises based on the equipment the user enters that they have. So
a lot of time getting with experts (or becoming an expert) AND building a new
machine learned model possibly with a nice, usable UI. This sounds like a full
time job for 5+ people for 6+ months at least to get any sort of real
prototype.

------
brainless
I started reading the post but the moment I saw big numbers in hypothesis I
felt like the author has no clue how hard reality is. Facebook is a great
cocoon to try random numbers. Outside that founders take months to even find
the right set of people who fit your need space.

Just __talk __, generally, about pains you are trying to solve. Everything
else is quite unnecessary.

Did you know that even founder of Zapier had to get on a call to explain their
first adopter how the product actually worked? Yeah these are early days.
Listen to their podcast of IndieHackers.

~~~
amanjaincorp
I'm trying to understand your feedback. Perhaps I'm missing something.

The point of the big numbers in my post is not to look at them in absolute
terms. But to look at the ratio A/B. And then apply it to a much smaller
audience. Because that tells helps validate that your idea will meet your
expectations (whatever they may be).

~~~
ALittleLight
I read the comments first, and then eagerly read your post - because I hate
Facebook and wanted to scorn you, but having read it I see that it's not
really Facebook-centric at all, containing only a glancing reference.

That said, I do think there are problems with your use of statistics and you
kind of do need to consider the numbers in absolute terms. If I check my idea
out with ten people I know in my target demographic, and five of them like it,
I probably shouldn't expect 50% of the target market will be interested in my
idea. My sample size is too small and also not really representative of the
demographic because there is a selection bias at play - they're people I know,
so maybe I know the only five people in the world who have the same problem
I'm trying to sovle.

Likewise, in your example, you talk about showing a concept to people on a
Facebook social group, but people on a Facebook social group aren't really
representative of your target demographic either. "Only 20 bucks a month?
Sounds great, I love interacting with apps!" may be something a Facebook
engineer would say, but that may be less likely from the mouth of the average
25-44 year old.

I do think testing your ideas out and seeing how users respond to them is a
good practice but there is inherently danger to doing tests and coming up with
statistics if you believe your own tests. You might run the experiments you
describe and feel good about your numbers and invest time and money to go to a
wider audience only to find out that a lot of people think 20 dollars a month
is too expensive for an app that gives workout advice (which, I predict, I
could find on the current app store for free) or that they don't like working
out at home or from an app, etc.

If you know you don't know what's ahead you'll proceed cautiously. If you
falsely believe you know what is ahead due to biased statistics or
insufficient sample sizes, you may blunder painfully.

I also caught a few references to Thiel's "Zero to one" in there, but I feel
the approach you're describing, at least the hypothetical example, is somewhat
contradictory to the advice in that book. I'd characterize your approach as:

1\. figure out the market - (Big number, everyone interested in fitness)

2\. guess what piece you can serve - (small number - only 2% need to try!)

3\. figure out if you can serve them - (small number - maybe 10% retention)

In other words, "We just need a small piece of this big market!"

From Zero to One:

>Now I think the opposite version of this is always where you have super big
markets and there is so many different things that went wrong with all the
clean tech companies in the last decade. But the one theme than ran through
almost all of them is that they all started with massive markets. Every clean
tech powerpoint presentation that one saw in the years 2005 to 2008, which is
the clean tech bubble in Silicon Valley, started with where the energy market,
where the market was measured in hundreds and billions of trillions of
dollars. And then once you're sort of like a minnow in a vast ocean, that is
not a good place to be. That means you have tons of competitors and you don’t
even know who all the competitors are.

>You want to be a one of a kind company. You want to be the only player in a
small ecosystem. You don’t want to be the fourth online pet food company. You
don’t want to be the tenth solar panel company. You don’t want to be the
hundredth restaurant in Palo Alto. Your restaurant industry is a trillion
dollar industry. So if you do a market size analysis, you conclude restaurants
are fantastic business to go into. And often large existing markets typically
means that you have tons of competitions so it's very very hard to
differentiate. The first very counterintuitive idea is to go after small
markets, markets that are so small people often don't even think that they
make sense. That's where you get a foothold and then if those markets are able
expand, you can scale into a big monopoly business.

In other words, if I looked at "2% of people who saw my Facebook post left
their email" I'd probably think the same thing I'd think prior to doing any
experiment, which is "There is a lot of fitness stuff, including apps and
gyms, this probably won't work". But, I'm also a pessimist, so maybe that's
just me.

Anywho - sorry if I've been too negative. I did enjoy reading your post, and I
hope you hit the target numbers you want on your substack subscriptions from
this post! :)

~~~
xondono
I thought the same thing. These kind of analysis will point you towards
already flooded markets (how many fitness apps there are already? My local gym
has one..), when you’d be way more interested in going after markets unknown
to most people.

------
jariel
This is a neat idea, and there may be some enlightening bits of feedback, but
it won't tell you what matters, and that is if people, in a specific audience
will like the product.

Sign ups and likes are very noisy singals.

There's just nothing that can take the place of market validation, so the 'new
standard' model of 'release early' is probably the best way.

Sometimes it takes a while to find the right user base and to get fit.

~~~
amanjaincorp
I agree different levels of "skin in the game" mean different signal.

Simple link click is pretty lousy signal and I would ignore it.

I think email is okay. Phone number is probably better. Somebody putting their
credit card is really good.

I didn't mention all of that for brevity. But you could device a point based
grading system based on what signal you get.

~~~
quickthrower2
I’ve done ok with a link to a calendly on the email marketing campaign - you’d
be surprised how many people we up for a chat! And as a bonus it solves the
awkward timezone issues for me and im not tempted to take 2am calls to be
nice!

~~~
amanjaincorp
That's a really interesting idea!

------
AznHisoka
The last step where you manually do what the product is meant to do really
works in 1% of products. Think of all the successful products today: Docusign,
Snowflake, Wix, yeah.. not going to happen.

It’s 2020. All the low hanging fruit ideas are gone. Any meaningful product
idea requires a bit more complexity in implementing it.

~~~
lvturner
I know several people who would pay good money to have lengthy and repetitive
forms (school permission slips for field trips) filled in 'automatically' and
doing this manually should be a cinch!

So I'm not sure I agree with your hypothesis.

~~~
AznHisoka
Really? Paying for someone to fill in forms? If it’s trivial like a field
trip, I don’t see anyone paying for that. If it’s important like a legal
document, you _dont_ want someone to fill it in for you.

~~~
lvturner
I thought the same, but I saw the size of the form for the field trip and the
detail they were asking for (most liability issues to absolve the school of
responsibilities) - though, I don't think it would bother me personally enough
to want to do it.

And important legal documents... Wouldn't you hire a lawyer or a solicitor to
deal with them?

I think even a will - I would probably pay a nominal fee to have one generated
for me, on the assumption that I can modify the final version before I
personally sign it off on it.

Though the point wasn't really the forms, the point was more that there is a
near infinite level of low hanging fruit, it's just not always apparent based
on your own situation (and to an extent country!) and set of
tasks/frustrations.

Another example that came to mind would be automatically updating all my stuff
when I change address (banking details, car registration, driving license etc
etc) sure there is a huge trust issue with this but with care, probably not
totally insurmountable.

------
ultrasounder
Not sure I can add anything more to this discussion than any more that has
already been said, but the real problem about validating ones idea is,” you
start out with an extreme bias” towards your idea. Conversely, if you are
validating an idea, you the only true and tried and tested way is to Mom test
it. It’s not enough you run a statistical poll and use that to run a product
development program. You have to actually talk to people. In this case, OP
needs to actually talk to the people in his demographics and validate the idea
that they would like to get a personalized email from his trainer with a
custom plan. If not it’s all shadow dancing with no real validation. Mom test
is an excellent book to drive this point home.

~~~
ultrasounder
Was on the phone and am not good with touch typing. But, I actually did what I
mentioned above. Being stuck at home like the OP, I actually had a lot of time
on hand. So, I started calling Coffee shops that had recently pivoted to
Shopify model(that change is happening in a massive scale) to try and see if
they would be interested in a subscription model for their coffee beans and
grinds. The outcome is for a different day, but the moral is "Always try to
Invalidate your idea". Your ideas are innately biased. And that's why Starting
with a clearly defined pinpoint is way easier than to apply Lean Startup
methodology to Your "ideas". Hope that helps. Cant recommend Ryan
Fitzpatrick's Mom Test enough. Guy even has a YT channel where he expands on
some of the things he talks about in his book.

------
hardwaresofton
Don't want to sound too negative but this article is basically:

\- Build a landing page with analytics on it

\- Early on, just do whatever automation manually

------
daenz
Something I am starting to learn about myself: I don't care if nobody wants
it, I still want to build it. Not a great trait for making money as a startup,
but it's an itch I just have to keep scratching...

~~~
whatl3y
I have the same itch and keep scratching it almost daily. I can say scratching
this itch over the last several years has absolutely been +EV overall. Maybe
not in a “I made $XXXX in sales over the last month/year/whatever” kind of way
but in that I’ve learned an incredible amount, have been able to use a lot of
the things I’ve learned at my job and have definitely improved my overall
market value as an engineer (validated with data I should say, not just an
opinionated, arrogant rant).

That said, I would keep scratching that itch if you still have the motivation
and drive to keep learning/building without seeing immediate material value
like money right away.

------
renewiltord
Read the comments first and thought this was going to be crap. Instead it's
great! Good read. Also read _The Mom Test_.

~~~
newbie578
The Mom Test is a must read! Only when you read it, do you realize how wrong
your previous approaches to business validation were.

~~~
amanjaincorp
I love that book. Such a tiny book but so much wisdom.

------
brahyam
I've done this before, it also appears clearly described in the "Start small
stay small book" which I recommend a lot. However after running it I found
myself a little lost, conversion was not as expected and it's very hard to
realize what failed. Is it the idea? or the design? or the speed of the
website? or the wrong target audience? or the wrong keywords?.

More importantly should I jump to my next idea? or should I invest more time
pivoting this one and pushing until I get where I want to be.

What happens when you're building an improved version of something that
already exist? Would you skip this kind of validation assuming that if people
pay for the big product they would pay for a small one? Specially if you're
not aiming for a venture backed monster but just a bootstrapped small
business.

~~~
amanjaincorp
RE: What failed?

Agreed this is a problem that your product needs to solve either way. This
idea validation playbook allows you to face the problem early on as opposed to
later.

RE: What happens when you're building an improved version?

I'd ask you why you think yours is an improved version? Is that a sure shot
fact or up for the market to decide. If it's sure shot, they yeah skip the
idea validation step. Sounds like your idea has more execution risk than
market risk.

But if you think its subjective, then that's probably something you could find
a creative way of solving using the Painted Door test or something similar.

------
sunilkumarc
Nice article. Most people make this mistake of building products first and
then finding customers for it. By following this approach they build something
that nobody wants. Ideally one should validate the idea and market fit first.
One shouldn't be building something for which there are no customers at all.

Follow this approach: -> demo \- sell \- build

And not: -> build \- demo \- sell

I bought this e-book recently on Gumroad and loved how it is right on point
about how to strategically validate your business and generate sales before
you even build it.

Here it is if someone is interested -
[https://gumroad.com/a/497906803](https://gumroad.com/a/497906803)

It provides so much value for the small price you have to pay. I highly
recommend buying this book.

------
not_a_moth
Maybe this is good advice for mass marketed consumer apps, but I don't believe
putting up a cheap SquareSpace template and faking automation instead of
building the product is a good strategy for getting initial enterprise
customers.

~~~
amanjaincorp
That's a fair point. You need a demo video / sales MVP and need to actually go
talk to customers most of the times.

There are certain enterprise specific channels where you could run an ad to
test. E.g. [https://appsumo.com/](https://appsumo.com/)

------
dissent
If the author ends a statement with a question mark, I find it very hard to
take them seriously. How can they be insightful but not thorough enough to
check a title? Does that make me a bad person? I mean just that, on its own?

~~~
chhavishah08
On the contrary, I have come through some articles / blogs that have delivered
insightful answers to the question that led you to it in the first place.
Doesn't make you a bad person, but you should definitely clean your judgmental
glasses.

------
Abishek_Muthian
When we say 'startup idea', it adds too much noise to the initial validation
process as the industry(Successful Entrepreneurs, Investors) almost
unanimously agree that the first step is solving the right problem[1].

So shouldn't the first step always be 'How to validate the problem you are
trying to solve?'

In my early life as an entrepreneur, I've failed multiple times since I didn't
understand this difference between Startup ideas vs Problems. Understanding
the importance of solving the right problem i.e. the problem with enough need
gap that people are willing to pay for getting it solved and of course the
problem which we want ourselves to be solved played a huge role in me building
successful products later on.

When I started coaching entrepreneurs a year back, I found this again to be
the common theme for failure among early stage startups i.e. building
something just because they can and hoping everyone needs it. It became a
professional necessity for me to come up with a way to validate problems/need
gap.

So, I started building needgap[1] a problem validation platform where problems
are the first class citizens. Posts are created by the consumers(incl.
entrepreneurs) who have a problem, want to find a solution for their problem,
check if others share the same problem, how ever small the problem might
be[3]. Builders can post their products in the comments if it solves that
particular problem or else discuss with others to create a product to solve
that problem(There are couple of products built/being built for problems from
needgap).

I'm not saying I've solved the issue of validating problems, far from it. I
think I now have a direction towards understanding the 'language, grammar of
problems and startup ideas' which might one day result in making validation
easier.

[1][http://www.paulgraham.com/startupideas.html](http://www.paulgraham.com/startupideas.html)

[2][https://needgap.com](https://needgap.com)

[3][https://needgap.com/problems/12-identifying-letter-o-
from-0-...](https://needgap.com/problems/12-identifying-letter-o-from-0-and-
vice-versa-typography-web)

------
kebman
This looks similar to the logline pitch suggested in _Save The Cat_ by Blake
Snyder, or really most books on scriptwriting. Hell, it could even be said for
writing your thesis. Always focus _hard_ on your idea, or you _logline,_ as it
were, or the premise of your thesis. Once clearly defined, worded and
outlined, it can both keep you on track and lead your project, _and_ it may
serve as a basis to get early feedback on your idea from the real world.

------
aajhasiudhasiu
As someone else here asked: What happens when you're building an improved
version of something that already exist?

I am currently building an android app that already a lot of competition.

Some of these competitors have in-app purchases and some have a separate paid
app version .

so the product idea itself is essentially validated - a nice chunk of people
are willing to pay for it.

------
cvaidya1986
Literally build whatever a cohort of users will pay for and go from there.

~~~
amanjaincorp
Yes, the idea talks about ways to figure out if a certain cohort will pay for
something before building the thing.

------
jwtnb
Give us a concrete & actual example that you have done instead of this
theoretical essay nonsense

------
dicefordeath
step one: work for facebook

------
sytelus
TLDR; Famous Painted door test: [https://briandavidhall.com/the-painted-door-
test/](https://briandavidhall.com/the-painted-door-test/)

 _A painted door test lets you gauge interest in a new feature, product, page,
or piece of functionality … before you’ve built it. It involves updating your
site with a link or callout that promises the new feature, then measuring
clicks to see how many visitors would try it if it existed.A painted door test
lets you gauge interest in a new feature, product, page, or piece of
functionality … before you’ve built it. It involves updating your site with a
link or callout that promises the new feature, then measuring clicks to see
how many visitors would try it if it existed.A painted door test lets you
gauge interest in a new feature, product, page, or piece of functionality …
before you’ve built it. It involves updating your site with a link or callout
that promises the new feature, then measuring clicks to see how many visitors
would try it if it existed._

------
jcun4128
Ask for money?

------
techslave
it’s official. substack is the new medium. a platform mostly for worthless
“expert” opinion.

