
How I Quickly Test and Validate Startup Ideas - dberube
http://startupbound.com/how-i-quickly-test-and-validate-startup-ideas/
======
powertower
I'm kind of biased against using Google AdWords since they banned my account
for no explainable reason (they lack complete customer care to the point where
they can't even display the ban message right:
<http://www.devside.net/images/adwords-account-suspended.png>), but consider
spending more money and getting better validation by going directly to
websites specific to your market and buying ad space there.

The AdWords traffic is just too broad and "general public" and it's incredibly
difficult because to get good CTR (click-through-rate), you actually have to
find keywords that produce bad or irrelevant results for the searcher while
being on topic, as otherwise the searcher will simply click the organic
results and never even see your sponsored result (when was the last time you
went to Google, did a search, got good organic results, and decided to skip
that and just go to the sponsored link?).

AdCenter is a little better IMO, but niche websites are absolute killer since
you're no longer screwing around, trying to filter out irrelevant traffic,
trying to get the right keywords and bid prices, and making sure your landing
pages have the right quality score. The moment you get a low CTR is the moment
Google AdWords start increasing prices, stops showing your ads.

Go right to the source to test your ideas. It will take more money, but you'll
get a confidence level of 95-99% for your validation, rather than something
much much less that you'll get using AdWords. It will be quicker too; AdWords
keywords can be incredibly difficult and time-consuming to get right.

~~~
suking
You have no idea what you're talking about with AW.

~~~
skulvr
ADW is overrated and very expensive. Specially if you need an expensive
keyword. I had a good experience with Facebook ads. It was cheap and high CTR.

~~~
suking
facebook ads don't convert into buyers - adwords ads do - that's why it cost
more. if you do the proper research, set up your campaigns and ad groups
correctly, add negative keywords, adwords can work really well. you need to
play their quality score game, get a good CTR on your ads and always be
testing new ads.

------
wooyi
Doctors don't buy software on the internet. You can't validate the idea that
way. The sales cycle is long and their buying process is totally different
from the "convert on the net" SaaS model. I did a startup called Maviq in 2008
for the patient reminder market. There is a v. high barrier to entry in this
market for startups.

~~~
vidarh
While you may not be able to validate the idea of the service in isolation,
you _can_ however validate whether or not there's a market for a service like
this _sold over the internet_.

In many cases, someone may consider it worth excluding an otherwise
potentially profitable idea if the sales cycle is long and the buying process
is different from the "convert on the net" SaaS model.

------
vsl2
I don't know about having "silver bullet" advertisements be your landing page
and text ad. It looks suspiciously like spam or some kind of con (think about
the majority of "Work From Home" ads). Every reasonably smart person I know
wouldn't trust "silver bullet" ads enough to seriously consider paying for the
product being advertised. In the particular example, most doctors are
supposedly smarter than the average population and so I think the conversion
rate would be even lower.

I don't claim to know a better way to garner attention through advertisement
than eye-catching (yet suspicious) claims, but I think the described method
helps you determine the interest level in your product from a small subset
(more trusting or naive, depending on your point of view) of your target
audience, not your total target audience. Maybe there's a known "naive-to-
total" extrapolator that I'm not aware of.

~~~
alabut
There's a balance to be struck because the polar opposite - discussing
features rather than benefits - is the norm and it's worse than ineffective
for the average user, it's confusing noise.

I just came back from running two days of usability tests on-site at a
security software company. I wish everyone could see the test subjects' eyes
glaze over the typical marketing text describing exactly how the thing works
and conversely, how they'd light up about 20 minutes into it when I'd guide
and moderate them towards the hidden nuggets of real tangible value, like "4x
faster than Norton" or a chart showing that the memory usage was a tiny
fraction of competitors.

~~~
vsl2
I agree that there needs to be a balance and furthermore, that finding that
right balance is very difficult. Probably why marketing whizzes get paid the
big bucks.

In your example though, there is difference in that you're an actual person
representing a reputable company on-site, not a random anyone-could-have-
created-this advertisement on the anonymous web. From more trusted sources,
I'd be interested in hearing the benefits and not necessarily the features.
But in an environment where seemingly every third website activates my anti-
virus software and I read about data hacks every week, I definitely don't
trust Internet ads filled with hyperbole and exclamation points.

~~~
alabut
It's true that people in a usability test don't behave exactly the same as at
home, but that actually proves my point. If they're obviously trying to be
positive and helpful (no matter how much you tell them to relax and be honest)
because they're constantly saying nice things about the site or app, then it's
even more noteworthy to hear any negative comments or see that they're
confused. Especially in the beginning of the session when their energy is
higher.

------
egiva
Actually, other people have done this in the past - Timothy Ferriss (the biz
guy and author of "The Four Hour Work Week") was famous for using this adwords
multivariate testing technique when he was deciding the future title for his
book! See a great interview here: <http://www.youtube.com/watch?v=3JHzLlvlVmQ>

------
asmithmd1
Clicking through to the authors blog home page and reading between the lines a
little it looks like he is the founder and was the CTO of Mofuse.com and was
kicked out just a week ago. Mofuse took $1.2M from 33 angels this past January
so I guess they had something to say.

What is the story dberube? Inquiring minds want to know.

~~~
dberube
I demanded $40MM a year in compensation and the use of a private jet. I was
told no.

------
mluiten
I actually like to add an intermediary step between the landing page and the
e-mail form: a "Create an account/Buy now for $X/month/year" button. When
people click on it, they get a "Sorry, we're not done yet, but give us your
email and you will be notified once we release" + the email form.

Not only do you know people are potentially willing to pay X dollars for your
product, you can also do a multivariate test on different amounts or pricing
schemes to see if they convert differently, giving you some rough idea how
much your idea is worth to them. I always add a "Make an account for FREE" as
well, as a baseline.

~~~
dberube
Well, the problem with doing that at this stage of the process is you risk
getting the initial price completely wrong and nobody clicks it.

Even though, if you had priced it just $20 lower, everyone would have clicked
it.

So by putting price in the mix before you even identify a willing market, you
risk missing the ball completely.

~~~
cbr
You could A/B on price to get the demand curve.

~~~
dberube
But you still leave open the possibility of not learning the basics of whether
or not someone wants the problem solved.

------
markbao
If you're looking for a fast landing page creator like the one mentioned in
the article, I'm working on one: <http://seekevidence.com>

~~~
hypnotist
Correct me if I'm wrong...

Are you using unbounce to create unbounce alternative? That is pretty badass.

:)

~~~
dberube
Hahaha

------
wwdevries
Can you define what you consider a "high conversion percentage" in this stage
of testing?

------
kamikazi
Fantastic article dberube, couldn't have come at a better time for me. Few
queries: a) What would you call a good collection, conversion rate? (Also
asked on yr blog) b) Will this work for ideas that might be seasonal? (eg: I
am thinking of selling a particular fruit from India online - it's seasonal
and the store will operate only abt 4-5 months. If it's successful, I'll see
what else I can do with it, but as of now there are no plans). So, have you
worked this strategy for any such ideas which are not 24x7? How early before
the season do you think I should run this. My season is still 5 months down
the line. If I run too early (now-ish) I run the risk of not getting anyone
interested as no one is searching for it. c) Can you give a few figures from
your 3 experiments about how many visitors you got from adCenter? And their
conversion rates? Thanks.

~~~
helen842000
You could always start by collecting e-mail addresses of those people
interested to buy it.

So they sign up and get notification once you have the items ready to buy.

Consider your shipping methods carefully else this type of product might be
tricky.

~~~
kamikazi
Well, that's my fear. That if I run this too early I may not be able to
collect any prospect email addresses (as almost no one/very few might be
interested in it or searching for it at this time).

I do have some queries/concerns on the shipping front too. Have you worked on
this type of farm/agriculture product or know any pointers where I can get
more info?

 _ps: Updated my email in my profile (if you want to take this offline). Don't
see your email ID in yr profile._

------
mva
I've read about this idea in the 4 hour work week. This is the way Tim Ferris
tested his ideas.

~~~
matdwyer
Agreed, the 4HWW covers this fairly well, along with some tips I hadn't
thought about previously

------
TamDenholm
Down for me, heres the google cache.

[http://webcache.googleusercontent.com/search?sclient=psy-
ab&...](http://webcache.googleusercontent.com/search?sclient=psy-
ab&hl=en&biw=1366&bih=566&source=hp&q=cache%3Astartupbound.com%2Fhow-i-
quickly-test-and-validate-startup-
ideas%2F&pbx=1&oq=cache%3Astartupbound.com%2Fhow-i-quickly-test-and-validate-
startup-
ideas%2F&aq=f&aqi=g4&aql=1&gs_sm=e&gs_upl=548l2995l0l3380l7l5l0l0l0l0l173l681l1.4l5l0)

------
eps
> _If everyone is busy, those calls are often not made._

I personally have not seen a single doctor's office that does NOT make
reminder calls or forgets to make them. Moreover virtually all of them also
have a $50 fee penalty for missed appointments (which is never charged, mind
you).

In other words be careful if you are validatng the idea just for the sake of
proving yourself right. Especially when your landing page is so ambiguous in
its message, that it's basically detached from the original idea.

~~~
dberube
Once again, the example given in the post was not an actual test. I was using
the idea as an example to guide the article.

The startup idea in the article isn't the focus.

------
dpcan
These landing page systems have a lot of room for improvement.

Upon getting a name and email address, it would be quite valuable to present
the new subscriber with some form of survey. Be it short and multiple choice,
but instead of JUST seeing if there is interest, you may be able to find out
what aspects of your product bare the most importance.

ie If you create a landing page for a new shopping cart product, you may want
to ask: Will you be selling digital goods? What country are you in? etc.

~~~
Vandy_Travis
Agreed, 100%. I've actually bundled a Wordpress install with the necessary
plugins and libs (google analytics, basic survey abilities, a/b testing
harness). This makes it super easy for me to spin up an ec2 or linode
instance. Then I can run a test and make a decision without touching any of my
core products/sites/servers.

Not too difficult to do.

~~~
lesterbuck
That sounds really interesting. Could you share some more details, or point us
to an example that is running? Which plugins work well for you?

------
wildbunny
Using adwords to provide enough traffic to test your product viability is
genius...

I think the HN effect has taken their site down, though!

~~~
dberube
It has! It's at Rackspace and I'm working on getting it on the largest server
they have.

Didn't expect to hit the front page.

~~~
tszming
Would you mind telling how much traffic (e.g. visits per hour from HN) you get
when your article hit the front page?

~~~
robinwarren
I got 10k overnight when I put www.jobstractor.com on as a show hn a couple of
weeks back. That was at the top spot for about 3 hours. NB, overnight to me in
UK is obviously not overnight for majority of HN readers.

I'm hosting it on the smallest linode instance going with nginx caching
responses and I'm serving almost no images so that worked perfectly for me.

~~~
tszming
Thanks for your information.

------
lbl
I wish you had written this a bit earlier :) I invented a playing card game
and was really excited about it because I got good feedback during play
testing. I tested it in few bars and they all liked the prototype. So after
making some edits to the game rules from the play testing feedback, I went
ahead and printed 2000 decks and I have a website where I'm selling the decks
( <http://acardgamewithattitude.com> )

I'm not seeing quite the success in selling online like the feedback I got
offline.

No sure how I would have tested this concept before printing because its a
niche market i'm targeting.

Now i'm confused what is wrong \- Is it the site design \- copy on the site \-
Am I over explaining \- Is it not clear if its ia deck of physical cards

Can i take the site down and do a teaser like it is going to launch? and then
sell it?

~~~
mrbgty
I don't really know what I'm talking about but I'm not sure this type of card
game translates well online. In my opinion, the site design is a problem. The
problem I'm seeing is that there's nothing making me want to learn how to play
a new card game. I have no idea what the answer to that problem is but there's
got to be some motivator on the site, I think, to get people to want to learn
it. I'm sure once the game is learned people will love it. Maybe you could
look at techniques other card games/board games use to get people interested?
If you made a digital version of the game, you may also be able to sell the
physical card game online?

The site is too red.

~~~
lbl
thanks mrbgty for taking the time to look at the site and giving feedback.
"Motivation" site is "Too red" I have added that to my list of to do. Thanks.
Anyone else?

~~~
lbl
After the redesign, is there any learning on if the buy button should be
upfront on home? or it doesn't matter?

Maybe this is a a really stupid question.

------
phzbOx
Really interesting and well written. I particularly liked the part when the
author said not to show feature as if they don't _buy_ the silver bullet, they
won't buy your product.

------
jdee
I'm interested in learning how people go about testing ideas that are
difficult to define in the small amount of screen real estate that an ad
offers, or dont have a clearly definable customer type.

How would one go about testing the waters for 'mass market' applications like
Twitter, Google+ etc where the end customer is potentially everybody? I cant
see how an ad like 'want to keep in touch with your friends?' would convert at
all. Any insight anyone?

------
roythunder
One reason I think the advice in this post is valuable is not just to test the
startup idea, but also to test the founder's appetite for traffic acquisition.

Many technical founders like building products, but those products will fail
without the majority of energy directed toward getting traffic. So if you
can't get a couple hundred email addresses of interested customers, you can
save yourself a lot of time by not going further.

------
brador
I don't know what you guys have been drinking last night but HN is full of
absolute gold this morning. Great article.

------
freshfey
You might be scrapping an idea off just because the landing page wasn't
optimized - could this be a possible problem?

------
mpunaskar
@dberube

I have used same method to collect beta tester for one of my idea. Google
adword has given amazing reults so far. Thats why i am interested to know if
cost factor the only reason to choose microsoft adcenter over google adwords ?

Does it give more ad impressions for less price + better click through rates?

~~~
dberube
The quality of the people coming to the site, in my experience has been the
same.

I actually have better CTR on adCenter compared to AdWords for the same ads on
the same keywords (marginally better).

So it's really come down to cost for me. In my experiences it's consistently
double or more for the top keywords in the bunch.

I've personally never run a massive campaign to drive tens of thousands of
visitors to a site through either service. My guess is, if you need high
volume of people quickly, you need AdWords.

------
drudru11
I think this has been common practice for over 2 years now. Where did you
learn this technique from?

------
ideaoverload
Zynga's 'ghetto' testing is the same idea, maybe even more radical in details:
[http://grattisfaction.com/2010/01/how-zynga-does-customer-
de...](http://grattisfaction.com/2010/01/how-zynga-does-customer-development-
minimum-viable-product/)

------
willpower101
"If people like your startup idea then odds are you're not thinking broadly
enough" -I can't remember who said this.

Cool idea though. Definitely works for immediate user solutions. Not sure it
would work as well for innovations.

"People don't know what they want..." -SJ

------
fdb
I've heard this called "pretotyping". The basic idea is _not_ to create a
working prototype (which still can take weeks/months/years) but to judge
interest.

<http://www.pretotyping.org/>

------
nikcub
This is an interesting idea that has done the rounds a few times in the past
few years, but it is also a bad idea in most cases.

First, if Google find out that you are advertising and collecting email
addresses for a product that does not exist - they will ban you. PayPal will
do the same if you are collecting any sort of payment information. There is
just too much that can go wrong and it can be easily mixed up as being a scam
or spam (which it sort of is, if you think about it).

You don't want to be banned from Google (or PayPal) - they carry that against
your name forever, even if you later signup as a company or with a different
account.

Second, this can only help you in a market where your potential customer knows
what their problem is and knows to look for a solution. In so many cases of
successful products this would not have worked. Not for Google, Twitter,
Facebook, YouTube, Microsoft, iPhone, iPad or most any large and successful
product.

It can also lead to false hope, or worse, an assumption that a product may not
work. Twitter started out as a messaging system for cyclist couriers. If Jack
had read Tim Ferris[1], took his idea to test the market with search engine
ads, he may have come away unimpressed and believing there was no room for
such a product - without even realizing the broader possibilities for such a
messaging system.

It has become a bit cliche to quote Steve Jobs, but from his bio:

> At the end of the presentation someone asked wether he thought they should
> do some market research to see what customers wanted. "No," he replied,
> "because customers don't know what they want until we have shown them"

The better way to test the market is to actively market (as opposed to
passively market - ie. having the market find you) - ie. go out and find your
potential customers, the influencers in the market, analysts that cover the
market, etc.

The whole idea behind minimal viable product is to lower the risk involved in
testing a market - so get the MVP built and then promote it, but not just with
search engine ads (the part of the market that knows what it wants, likely to
be the smallest part) but also by actively reaching out.

The post suggests a budget of $20-50 a day just to get some leads, when with
that sort of budget ($500-1000) you could just get a first version of the damn
product built.

You can also find out how many people are searching by just using Trends or a
similar product (keyword analyser). The PPC part just means you are paying $10
per email address - contacts that you may never end up using

I would like to see that similar posts to this one proposing this idea stress
that results from search engine marketing are not the be-all and end-all. The
language in this post suggests that the results of search engine marketing for
a non-existant product conclude the viability of such a product:

> If you weren’t lucky enough to have a high conversion rate you can scrap the
> idea or rinse and repeat this process and tweak or pivot on the original
> idea.

This should say that your market may not be people searching for the product,
not that you need to pivot the idea. This isn't how you validate startup ideas
or products, it is how you validate part of a potential marketing strategy for
a idea and product.

(Edit: and thinking about it, as a customer, I would _hate_ this. If I were
searching Google and comparing products that I need for a solution for and
bumped into your product and gave my details, after your website suggested
that the product exists - only to later find out that it * doesn't exist* - I
would be very very pissed off about it. Especially if I needed a product right
now and was falsely lead to believe that a solution exists).

[1] Tim Ferris is cited a lot as being one of the first to do SE testing (in
'4 hour work week'), but I also remember Josh Kopelman writing a blog post
about it a while before that as well. IIRC, Ferris suggests that you do this
when you already have a product, to test potential new features, pricing
ranges etc. The OP probably should have at least cited Ferris in his post,
since this isn't exactly a groundbreaking idea and nor is it the first time it
has appeared on HN.

------
inopinatus
This is a novel online take on the general concept of test marketing.

I would love to see a statistical analysis of the results.

~~~
markoma
Just don't know how better to do this.. Apologize for breach of etiquette. Saw
a post of yours about a month ago regarding Puppet vs Chef. Really impressed
with your comments and wanted to dive deeper. Alas, the thread seems to have
died and I cannot append or reply there. Is there any chance you would be
willing to discuss that topic more? Many thanks in advance.

~~~
inopinatus
what would you like to know?

ping me, @inopinatus on tw*tter.

------
yonasb
Great post. I've used LaunchRock (widget, not the whole page setup), would
definitely recommend it.

~~~
ww520
I've visited LaunchRock to check it out. I didn't get any account after set
up. Somehow I have a nagging feeling of being trolled, landing on the landing
page of a site promised to create a landing page.

The FAQ says I will get an account after getting 3 people to sign up. Bahh. I
just want to check out the features and I have to spend effort in marketing
your site? Thanks but no thanks.

~~~
swatermasysk
Warning...shameless self promotion...

Would love to hear your thoughts on my own start-up, Kickofflabs:
<http://www.kickofflabs.com>

No gimmicks, tricks, etc. Just a solid product with fanatical support.

------
chunkyslink
Any good alternatives to unbounce?

~~~
dberube
It's the best one I've seen. There is also <http://www.instapage.com/> but
it's not as good and I've run into problems with pages not loading -- and when
you're paying for traffic to hit a page, you want it to load.

------
dberube
It's back up -- upgraded the server.

