
Ask HN: Should you TDD a MVP? - fam
Is it beneficial to do so or does it simply add too much overhead?
======
DanielBMarkham
The question here is really "what's the test?" You have to reailze _the MVP
_is_ the test_.

For a startup, customers are how you pass the test. Anything else is a red
light. So in the most important way possible, as long as you have no
customers, you have a test which is failing.

This is important because the maintainability you're looking for in a startup
is your relationship with the customer. Manage that and the rest takes care of
itself. If you are already in a business, yes, "maintainability" means writing
code that will last. But if you've just got an idea or a dream, you've got
nothing worth maintaining. Nor will you ever.

Put differently, your technical debt can never exceed the economic value of
your code, which in a startup is extremely likely to be zero. (Different
scenario entirely for project-based work for ongoing businesses, which is why
TDD makes so much sense in that scenario)

~~~
freshhawk
> Manage that and the rest takes care of itself

Not quite, you are committing yourself to throwing away your MVP and rewriting
once/if you find the fit. That might be a very good idea if your MVP is
extremely simple and you expect it to either fail or succeed with minimal
iterations.

> Put differently, your technical debt can never exceed the economic value of
> your code, which in a startup is extremely likely to be zero

Excellent point, but you probably want to factor in opportunity costs as well.
You do not want to take on so much debt that it becomes impossible to succeed.

As soon as you start to iterate on the idea then the maintainability of your
codebase starts to matter because that determines how quickly you can iterate
on new ideas.

You do not want to get stuck doing even a partial rewrite while your
competitors keep iterating their original higher quality codebase.

The overhead that testing or even full TDD add is very minimal if you have
experience, so that's a tradeoff that must be made carefully. If you are
thinking about _learning_ TDD or testing while doing your MVP then that's a
different story.

This is the same question as "should I use version control for my MVP" isn't
it? The level of complexity for an MVP where this becomes a bad idea is
different but it's the same question.

~~~
TheMonarch
I agree, the assertion that you shouldn't test is absurd. For example, TDD can
save you time by catching bugs for you. When Iv'e worked on apps without
tests, I have to test manually each time I make a build, which gets annoying.
That's time I could spend coding. I think the solution is something in the
middle ground. TDD can sometimes veer off into a time wasting effort for the
sake of TDD. Just TDD the complex parts, or the ones that are likely to break.
Also write the tests that are easy, or quick to write.

------
pbiggar
I'm the founder of <https://circleci.com>, so I feel I have a little
perspective here from talked to dozens of startups about this very question.

You should test a different amount based on your goals. TDD does not deliver
value to your customer directly, so the only way its even usable in an MVP is
helping you iterate and deliver the MVP faster.

A common mistake, and it feels like you're doing it here, is to think that TDD
is a best practice that you have to do, rather than thinking about how to
apply it to improve your productivity.

So how will testing help in an MVP? It allows you verify that you haven't
broken anything when you write new code, and when you rewrite existing code.
But because you'll constantly throw out code, spending a long time doing this
will literally be wasted as you throw out the code you're testing.

Avoid writing the sort of massive extensive tests that many associate with
TDD. When you unit test a function, test or two cases at most, any more that
that is overkill. If it takes you 2 minutes to write a function, a function +
tests should take at most an extra 30 seconds. If you can't think how to test
it, don't.

So some specific advice:

\- when you write a function, you're likely to test it somehow manually
(refreshing the browser, at the REPL, etc). Take that manual test and put it
into the test framework. It should take 10 seconds more. If it is more effort
than this, don't bother.

\- write MVP-type tests: "what's the minimum I can write here to viably test
this"? 1-2 test cases, at most. A positive test and a negative test ideally.
And then you're done, move on.

\- avoid full-browser tests. Those things are a massive time sink, and nothing
will change more than your front-end.

\- use a CI service (self-plug: <https://circleci.com>). This means you don't
have to run tests before you push. Just push and let the server catch it.

------
biesnecker
No, if your MVP is a total throwaway that you're using solely to validate a
hypothesis, and once/if it's validated you'll be redoing everything.

Yes otherwise.

My experience with TDD is that the cost/benefit curve doesn't really pay off
until you need to refactor or significantly expand a project, at which point
having good test coverage saves you an enormous amount of time/pain.

~~~
rhizome31
My experience is that doing TDD doesn't take more time, even for initial
development. It doesn't take less time either. It's subjective and I haven't
done any measurement (I suppose it would be really hard to measure the effect
of TDD, all other things being equal, since writing the same software twice
using both methods would obviously be misleading).

I do have one clue that TDD takes the same time as non-TDD for me : I used to
work as a freelance developer, paid by the hour, for clients to whom I didn't
feel necessary to tell whether I would do TDD or not when negotiating the
number of hours a given project or task would take. Then I was doing TDD and
was still able to deliver on budget. Some of those clients were themselves
developers who presumably would know how much time a task should take. The
fact that I was doing TDD, or even writing tests at all wasn't part of the
discussion about the amount of time it should take. Doing TDD didn't seem to
have any effect on my ability to meet the schedule.

I think the reason it works like this for me is that I tend to do a lot more
manual testing when I don't write tests (and if I don't write tests first, I
tend to not write them at all because the motivation is much lower, the code
is already "working" after all...)

------
sprobertson
If TDD isn't something you normally do, it'll likely be too much overhead. If
you're a TDD expert and that's what you always do, go ahead.

The purpose of an MVP is to ship something that works as quickly as possible,
so it's not the ideal time to be trying out a new technique.

------
j45
What I'm writing below is meant in general, and not at the OP.

The purpose of an MVP is to find the minimum amount of value that a customer
can't live without. An MVP is finding the 5 things you should have done in
hindsight out of the 30 on your mind.

It sounds like you're focussing on yourself, and making your own life easier
with TDD and not the customer. Wrong priority. You shouldn't expect your MVP
to be here in 10 months.

Quit trying to prematurely optimize an empire that doesn't exist yet. Even if
you write everything perfectly, you'll rewrite it from coding yourself into a
corner and having to work out of it. You don't know what the problem-solution
fit is, let alone whether theres a market of people willing to pay for it.

As history has shown, customers don't care what you code in, or how you code.
It's more about building software that solves a pain and a problem, first, if
you're serious about using the term "MVP". Sometimes folks keep doing what
they've been doing and sprinkle some lean terms on it to make it sound like
they're different.

TDD helps where there is complex business logic, not getting and setting basic
information, which most MVP's do. If you have something that complex, it might
not be an MVP.

Focus on the MVP, finding a product-market fit, and a problem-solution fit.
Get out of the building and quit using methodologies as an excuse to analysis
paralysis yourself.

------
schmrz
I think that this is a wrong question to ask. You have probably so many more
important questions to answer while working on your MVP.

If you are accustomed to test driven development then you will know (or at
least you will be able to reason about it) when to apply it and when not to.
It is just another development practice. But I do not think that you will fail
if you do TDD or if you do not. But you do risk failing if you emphasize the
wrong things such us should we use X or Y development practice. Use whatever
works for you and your team (if you are not alone).

To give you a more concrete answer: follow common sense and your team's
suggestions. Avoid making a development practice one of your priorities.

------
LnxPrgr3
Personally I wouldn't, but my opinion doesn't matter.

Does TDD help you? If so, use it! If not, don't! The whole point of an MVP is
to ship something ASAP. Do whatever helps you do that.

Personally I've found TDD to be a tremendous waste of time for most things,
but I know some people swear by it. It's your product. Git 'er done in
whatever way you've learned to get things done.

Edited to add: People talk a lot about TDD leading to more maintainable, less
coupled code. There's more than one way to skin a cat—not using TDD doesn't
necessarily doom your codebase, and using TDD won't necessarily save it. Like
I said, do whatever works for you.

------
paulbjensen
I think it's ok to go without tests at the start of your project, but at some
point you have to transition to using TDD, otherwise the ability to refactor
the codebase or change existing features becomes difficult and time-consuming.

To give a rough idea, I spent ~ 2 1/2 months working on my app (Dashku.com)
before I wrote a single test for it. Then when a company approached and asked
to use it with a special requirement, I realised it needed tests in order to
be deemed production-worthy. Now the app has tests (both unit and
integration), and I'm now resuming adding features to the product.

------
frio
It really depends on the degree to which you do it. I'd argue that just having
one or two very basic acceptance tests is probably all you want when you're
testing the waters with an MVP. The most all-encompassing test you can make;
and make it from a client-side perspective (so: JS or Python or something
exercising your HTTP API).

For instance, if you were making a library app: test that a logged in user can
add a book to it. That's as simple as:

    
    
        class AcceptanceTest(TestCase):
            def testThatAUserCanAddABook(self):
                r = requests.post("/books/", data={"title": "My New Book"}, auth=HTTPBasicAuth('user', 'pass'))
                self.assertEquals(r.ok, True)
    

Then, do the absolute bare, hacky minimum to get that test passing: implement
a server which simply returns 200 to that URL.

As you're working, I've found that having that singular, overarching
acceptance test will speed you up massively; there's no need to arduously step
through your UI, or run through various httpie commands; the test covers you
as you refactor to implement your user authentication, basic API, and backing
database.

So, yes: TDD can be a big time investment, and can slow down your launch if
you fully break your work down into unit tests. But having simple,
straightforward, all-encompassing acceptance tests can give you a great base
for getting work done.

(Note that after launch, once you start writing more tests, you should
eventually throw away that acceptance test, as more componentised and faster
unit tests replace it :)).

------
christopherslee
The key question here is, are you building a quick and dirty prototype, or are
you building an MVP? MVP doesn't mean a garbage prototype. Depending on what
you're doing, an MVP could still take significant time to build (and your MVP
would take 1 month vs 3 months for the "full product.")

IMHO, if you're worried about supporting and maintaining your app, and
concerned about your users not having a buggy experience, you should TDD it.

------
davidw
You should at least put a testing framework in place, and test some low-
hanging fruit. That'll make it easier to get momentum going for more testing
if things continue, and at least give you some minimal coverage.

~~~
stakent
And test at least fundamental parts of user interaction with the MVP: sign up,
login/logoff, conversion funnel, what else depends of MVP. MVP may have
limited set of features, but implemented ones should work.

This not TDD but rather BDD without unnecessary fluff.

Additionally manual testing is repetitive work. It is better to outsource it
to the computer to make sure that testing is in fact done when needed -
preferably after each code change. By doing testing this way is far easier to
isolate immediate cause of failing test - developer knows what was changed
recently.

~~~
rhizome
And just as an anecdote, I built a prototype for a client that laid fallow for
a couple of years while he worked on some parallel business ideas that I also
built, but when it came time to return to fire up the original idea there had
been major language and framework version upgrades in the meantime. I had to
almost rebuild the app from scratch due to the lack of tests in the MVP that
could tell me what was what. Basic tests would have cut the upgrade time in
half. Not that I mind the billings, but this time and effort gets in the way
of the bizdev.

------
peterhunt
TDD is good for some things, terrible for others.

TDD's great when what you're working on is easy to test, like your data
backend for your application.

It'll vary based on your product of course, but in my experience I've found
the frontend to be the big time sink in development. Using TDD for
contemporary frontend development is, in my opinion, a big time sink since the
cost/benefit ratio of frontend testing today is way too high.

tl;dr use TDD for your backend (data model invariants) and hack your frontend
as quickly as you can (on top of a decent frontend stack so you don't get
_that_ messy).

------
jdlshore
This is a really good and interesting question, and it's one I've been
struggling with myself.

The problem boils down to this: TDD makes your software more maintainable (if
you do it well) and it lowers your cost of development. _However,_ it also
takes significant time and effort to figure out how to test-drive a technology
for the first time. Everybody can TDD a Stack class; TDD'ing a database, or a
web server, or JavaScript [0] is a _lot_ harder.

So the answer seems simple: use TDD for the parts you already know how to TDD.

But it's _not_ so simple! It's much harder to add tests to existing code than
it is to TDD it from scratch. Sometimes, it's flat-out impossible. The expense
is so high, there's a very good chance that you'll _never_ get around to
adding tests to the un-TDD'd code. It will hang around causing bugs,
preventing refactoring, and sapping your agility forever, or until you
rewrite... and a rewrite of any significance will halt your company in its
tracks, so you won't do that.

So the reality is that, anything you don't TDD from the beginning, you'll
probably never be able to TDD. Companies that go down this road find
themselves doing a major rewrite several years down the road, and that's
crippling [1].

There's another wrinkle on top of this: manually testing code and fixing bugs
is _expensive_. Once your codebase gets above a certain size--about six
developer-weeks of effort, let's say--the cost to manually test everything
exceeds the cost to TDD it. (The six weeks number is a guess. Some people
argue it's less than that.)

So the real answer is a bit more nuanced:

1\. _If_ your MVP is truly a throw-away product that will take less than six
weeks to build and deploy and you'll _never_ build on it after that, use TDD
only where it makes you immediately faster.

2\. If your MVP is the basis of a long-lived product, use TDD for the parts
you know how to TDD and _don't do_ the parts you don't know how to TDD. Be
creative about cutting scope. If you must do something you don't know how to
TDD, figure it out and TDD it.

3\. It's okay to be a bit sloppy about TDD'ing the edges of your app that are
easily rewritten or isolated in modules. But be very careful about the core of
your system.

That's my opinion based on 13 years of doing this stuff, including building
five successively-less-minimal MVPs over the last nine months for my JS
screencast. The first three MVPs were zero coding, the fourth was a throw-away
site, and the fifth was TDD'd with aggressive scope cutting to minimize the
number of technologies that had to be TDD'd.

[0] Shameless plug: I have a screencast on TDD'ing JavaScript that I'll be
opening up to the public, on a subscription basis, in a month or so.
<http://www.letscodejavascript.com> .

[1] Rewrites are crippling: See Joel Spolsky's "Things You Should Never Do,
Part I." <http://www.joelonsoftware.com/articles/fog0000000069.html> (There is
no Part II, by the way.)

~~~
sevenadrian
I think this is the comment I agree with the most so far.

I believe TDD is important, especially for a MVP one could argue. Do MVPs
change a lot? Yes, obviously because you are learning and adapting to the
needs of you customers. This is where have good test coverage (thanks to TDD)
is a big asset. You can refactor, change, pivot, all you want, without the
worry that removing this feature, tweaking that feature, or adding this other
feature, won't break something else.

And manual testing is way more expensive then investing in automated testing.

In my opinion, unless it is a very short lived project that you don't plan to
iterate on and isn't very important, you should have good test coverage. And
good test coverage is best achieved with TDD.

This is of course just my 2 coins

------
PaulKeeble
Those that TDD a lot have come to realise what TDD does is help you find a
good design. It teaches you something about the way in which you code and
design and allows you to cheaply play with the alternatives. It also ensures
the code works and ensures so in the future as you undergo changes.

BDD for functional testing ensures that the functionality you test works and
again that before you cut code you have thought about how your functionality
is going to work. These are valuable exercises to do with or without the code
to run.

So when it comes to an MVP to test the idea how well does it have to work and
how well does it have to be designed? Some amount of your system will be
critical and if in the process of changing it you break that functionality you
may have no product. A certain amount of confidence in the core of what you
are building is important, having it automated will allow you to change
quickly. Ensuring 100% code coverage is not important, some bugs when you are
small are probably acceptable.

------
gkarthik
It depends. A MVP is basically used to test if your core functionality is
popular among users and also to collect some useful feedback from users via a
simple system.

I personally never developed a MVP for my first start up, Tune Patrol.

While, while making Tune Patrol, a social music discovery platform, my MVP
would've involved just putting up a few songs and letting users play them.
Like I said, this would've helped in two ways,

1\. Get traction. Users know that you're up and running.

2\. It helps test to see what kind of features a user might want by using a
good feedback system.

Instead I went on to develop a product with a lot of functionality. My first
problem was that I couldn't immediately determine if users liked my understood
the functionality completely. Luckily for me turned out users understood
pretty much everything and now we are developing on it.

You could also read "Lean Startup" by Eric Ries where he stresses upon the
importance of a MVP and quotes a few examples from his own experiences.

------
programminggeek
I would TDD in either case simply because if you end up maintaining it at all,
you're going to wish you had tests. Also, I wrote an architecture that makes
starting from day one using TDD much easier. It's called Obvious:
<http://obvious.retromocha.com>

Once you understand the concepts, you can whip up a simple crud app in a
couple hours. Example app here: <https://github.com/RetroMocha/obvious_status>

------
ollysb
When I'm starting out I find writing a cucumber scenario is a great way to
focus on getting a useful feature done. It's the first thing I add to a
project. The general setup is done as part of getting that first scenario to
pass. After that I do whatever is fastest. I use cucumber scenarios as tracer
bullets for new areas of functionality but if I'm refining perhaps I won't
bother. As always though, it depends...

------
vlokshin
front end, no. It changes too much. Back end, maybe. Only for the core logic
that (probably) won't change.

Basic rule: don't let work create more work if the benefit's not worth it. You
probably knew that already, so just stick to it.

------
mitchi
I initially thought the question was "Do we ask our MVP to use TDD ?" -.-

------
jmount
"In fact I usually like to TYZCMZQ when I'm not VJBDTLJX-ing!"

~~~
jmount
Okay now the less sarcastic- but equally harsh answer. If you think test
driven development is not good you should not use it. If you think test driven
development is good and you don't use it on your minimally viable product:
then you are signaling that even you don't find the product interesting enough
to even automatically exercise.

~~~
ZeroMinx
I disagree. I like TDD, but if you're quickly building a MVP that has
beforehand been decided(1) to be thrown away once you've gotten funding, just
hack it like you've never hacked something before. You're aiming to prove a
concept, to visually display your idea.

(1 - do make sure the non-technical people in the group really really really
understand this)

~~~
jmount
Interesting. I still come out on the other side. If the product isn't being
used much and might be the basis of pivot/re-factor, then tests become even
more valuable. Better to catch the bug in a test than in your first demo.

~~~
ZeroMinx
I think it very much depends on what your MVP is, what your goal with it is.

My stance here is where the goal of the MVP is to get investment money. You're
not showing your MVP to random users. You want to build something that "works"
quickly, then get money to build it "properly". You have a good idea and you
want to attack a certain market; but, you don't want to spend year(s) building
a product and hope you can grow slowly. Your aim is to prove to investors
that; you're a good team, you have a solid concept, with money you will gather
a team and build this product.

In this case, I think you can(/should?!) skimp on the unit tests. The code
will be thrown away after a few months of its inception.

There are of course many different MVP situations, this is just one small
avenue.

------
volandovengo
short answer: no.

The purpose of a MVP is to prove the idea using the minimal amount of effort.
TDD takes about 50% more time.

------
martinced
Seen that TDD refer to "Test Driven Development" I was nearly sure that the
question asked about "MVP" / Model View Presentation.

I've implemented lots of different of UIs using MVC type 1 and 2, HMVC / PAC
etc. and, as a developer, I don't have single clue as to what "MVP" means
here. Maybe some context to remove the ambiguity would be good in laconic
questions like this one, especially seen that the other acronym is used.

Google for "TDD test driven development MVC" and you'll see that I'm 100%
correct on this one ; )

~~~
sevenadrian
I believe by MVP he means "minimum viable product"
<http://en.wikipedia.org/wiki/Minimum_viable_product>

