
Test Driven Development? You've got to be kidding me... - peteretep
http://www.writemoretests.com/2011/09/test-driven-development-give-me-break.html
======
Luyt
Discussions about TDD always make me think of Ron Jeffries' attempt to develop
a sudoku-solver using TDD.

That attempt is discussed in Ravi's article:
[http://ravimohan.blogspot.com/2007/04/learning-from-
sudoku-s...](http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-
solvers.html)

Peter Norvig wrote a sudoku solver, not by using TDD, but using old-fashioned
engineering: <http://norvig.com/sudoku.html>

Ron Jeffries' attempts:

<http://xprogramming.com/articles/sudokumusings/>

<http://xprogramming.com/articles/oksudoku/>

<http://xprogramming.com/articles/sudoku2>

<http://xprogramming.com/articles/sudoku4>

<http://xprogramming.com/articles/sudoku5>

And, as dessert (Ron is very frank about his failures):

<http://xprogramming.com/articles/roroncemore/>

 _"This is surely the most ignominious debacle of a project listed on my site,
even though others have also not shipped. (Sudoku did not ship and will not.
[...])"_

~~~
gruseom
"Old-fashioned engineering vs. TDD" is misleading. The difference is that
Norvig knew in advance how to solve the problem - constraint propagation -
while Jeffries presumably did not.

Suppose Norvig had used TDD while writing his solver. He would have used
constraint propagation and come up with a good solution that way too.
Similarly, any other technique would have yielded just as poor a result for
Jeffries. Knowing things in advance is not a technique. Norvig made this point
in Coders At Work:

 _I think test-driven design is great. I do that a lot more than I used to do.
But you can test all you want and if you don’t know how to approach the
problem, you’re not going to get a solution._

The important question is: how can you benefit from existing techniques (like
constraint propagation) that reduce your hard problem to an easy one _if you
don't know about them to begin with_? It is a genuine conundrum. Norvig gives
a very non-technical answer in Coders At Work: general education and
intuition. To that list I suppose one could add: asking around. What you don't
know about, other people may.

~~~
JoachimSchipper
TDD seems to encourage you to dive straight into implementation. For certain
problems - e.g. a Sudoku solver - it's _much_ more effective to think through
the entire algorithm before starting to write code, and I think that the above
debacle supports this argument.

You don't exactly need to know any formal theory of constraint propagation to
solve this, after all - I wrote a Sudoku solver many years ago based on the
algorithm "place a random valid value in the cell with the least valid values;
cross off any now-impossible values in the same row/column/block; backtrack if
there are no valid moves" which is _not_ impossible to come up with on the
spot. (I did.)

~~~
gruseom
I agree that there is a serious criticism here, and that sufficiently deep
thinking is underrated.

TDD advocates tend to assume that you can always iterate your way to a
solution. But what you get by iteration is sensitive to how you start.
Technically, yes, you can evolve any program A into any other program B, but
in practice no: the class of programs that A will evolve into is sharply
constrained by A. I think this is true no matter how small A is. If that's
correct, then initial conditions are a lot more important than it's
fashionable to think they are.

Still, this isn't a weakness of TDD per se, but of iterative approaches in
general, and it's something that advocates for iterative development of
software (and other things) haven't yet taken into account. That's
understandable, because the advent of iterative approaches was so necessary
and has proven so valuable in other ways. These things come in historical
waves.

A point about the Sudoku "debacle", though. The posts by Ron Jeffries are
indicative of something other than TDD. They're indicative of mucking around
in public. The difference isn't that other people don't make embarrassing
mistakes; it's that they hide them. Why would Jeffries exhibit his so
blatantly? If he were just a bad programmer or a zealot or a dishonest guru,
obviously he'd have suppressed them. Not one of those types is ever too dumb
to do _that_. (Typically, they're quite good at it. Maybe that's where their
intelligence goes!) So something else is going on, and I found it unfair that
no one who wrote about the Sudoku "debacle" ever asked what it might be.

My guess is that it's the original XP culture. These guys practice a let-it-
all-hang-out style in which they highlight their mistakes and affect being
stupider than they really are. Kent Beck affects this "I'm an idiot" style in
the original TDD book. I say "affect" because they're not idiots, and I find
the tone annoying. But I can see why they do it. It's an educational tactic to
say "see, I make dumb mistakes too". They advocate a way of making software
that embraces dumb mistakes as part of the process and encourages people to
get over the fear of looking like an idiot. Underlying that is a psychological
view of software development that can be traced back to Weinberg's "egoless
programming".

I may be way off base because I haven't read the posts. I tried once and
quickly lost interest. But I do think I recognize the culture.

~~~
plinkplonk
"I agree that there is a serious criticism here, and that sufficiently deep
thinking is underrated."

Exactly. The criticism of TDD is (or should be, imo) about the "Driven" part,
not so much the "Tests" part. Using conformance to an increasing number of
tests as a hill climbing metric and a substitute for deep thinking(sometimes
expressed as the "TDD is not about testing, it is about design") gets you
stuck on local minima, a point Peter Seibel delineates clearly in his blog
post on the subject .

I do disagree with you somewhat in that I think Ron Jeffries ( and _most_ of
the Agile evangelist/conference speaker/methodology-book-author types for that
matter) _are_ dishonest gurus who couldn't code their way out of a paper bag,
but reasonable people can disagree here.

------
raganwald
Having responded to the tone elsewhere, I’ll respond to the content here :-)

The article presents a straw-man in the form of someone who believes TDD is
the One True Way, and then attacks this by suggesting that the purpose of
tests is—more or less—to prevent regressions. Given that this is an important
thing, and since TDD isn’t really about that, TDD is clearly not the One True
Methodology.

Quite honestly, that makes perfect sense to me. Love it or hate it, TDD is a
design technique, not a regression prevention technique. Much of TDD is the
creation of tests that validate an _implementation_ , not a _requirement_.
Thus, changing the implementation breaks the tests and you have to fix them.
In that sense, the tests TDD produces are useful after the fact in the same
way that Design By Contract’s contracts are useful after the fact. And if
Design By Contract wasn’t somebody’s trade mark, I would honestly say that TDD
is a way of practising DBC when you don’t have a language like Eiffel handy.

Is Design By Contract a useful technique? I think so. Is it the only technique
to use? No. Are implementation tests the only tests needed in a project? No.
Are the useful? Yes. Do they impose a maintenance overhead? Absolutely. Are
there other paths to success? Absolutely.

So in summary, if I take the article at its face value of railing against TDD
being the ONLY methodology, I agree. It isn’t. It isn’t even the only testing
methodology. However, the TDD baby is staying right here while I toss out the
fanaticism bath water. I believe that TDD is a useful tool and that one way to
think about it is as a form of Design by Contract at the implementation level.

~~~
Woost
Straw man? Where I work about half the developers believe that TDD is the One
True Way. Unfortunately, this includes the CTO, who wants to force every
developer to practice TDD. (along with other 'agile' methods)

When I've worked on something in the early stages of design with someone who
has used TDD (for example, if someone junior asks for help designing their
project), I end up having to throw out or rewrite major parts of the tests,
and explain to them why that test is useless. Usually the problem is one of
two things: the functionality changed because their design changed (maybe a
subclass where there wasn't before, or the data structure goes from an array
with constants to a hash/map) or the test was actually useless and was testing
that a library worked.

Are there places where you'd want to do TDD? I'm sure there are, though I
don't use it. Do I think less of programmers who use TDD? No. Do I think less
of programmers who try to convince me to use TDD? It depends how. If they say
"it produces better code" or some such bull, then yes: Simply writing tests
first does not help me produce better code, nor have I seen it help other
people in my company produce better code. If they don't know what they're
doing, or aren't familiar with something, writing tests first just means
they're writing broken tests, and increases the overhead for fixing it ("I
can't change that, the test will break!")

Personally I prefer to write code and then write tests for the parts of it
which I think are critical. I also prefer to write tests that ensure the code
itself is solid, not "does it return what I expect when I expect it to". By
that I mean write tests that feed the method bad data, nulls, edge cases, etc.
The way I've seen TDD described/practiced is that tests are written to say
"this is how I expect this method to perform under normal circumstances"

As you said, it does seem like TDD tries to be DBC. However, it also seems
like the people selling TDD sell it as the One True Way, not as another form
of DBC with all the benefits/drawbacks of DBC.

~~~
raganwald
_Straw man? Where I work about half the developers believe that TDD is the One
True Way. Unfortunately, this includes the CTO, who wants to force every
developer to practice TDD. (along with other 'agile' methods)_

I’m sorry that you are frustrated with your work environment, however we seem
to have a misunderstanding about the form of the OP’s original argument.

If you read it as:

(a) There exist TDD OneTrueMethodologists, and (b) TDD is not the One True
Methodology

Therefore:

(c) The OneTrueMethodologists are _wrong_.

Then this argument sets up One True Methodologists and attacks their belief
that TDD is the One True Methodology. I consider this a well-formed argument,
even though obviously half of your company disagrees with its conclusion
because they disagree with (b).

The _other_ way to read the OP is:

(a) There exist TDD OneTrueMethodologists, and (b) TDD is not the One True
Methodology

Therefore: (c) Using TDD is criminal.

This argument attacks TDD by showing that TDD OneTrueMethodologists are wrong.
I do not consider this a well-formed argument against TDD, because I do not
believe that using TDD is synonymous with believing that TDD is the One True
Methodology. There are also some other small issues, such as the question of
whether the _only_ tests in a project are those produced by TDD.

I have seen projects that use TDD part of the time, and use TDD as well as
other types of automated tests. An argument against OneTrueMethodologists that
believe TDD is the only way and that other tests are not useful or that other
design practices are secondary to TDD is not really an argument against using
TDD in a wider context.

I personally read the argument as taking the second form. If you read it as
taking the first form, I can understand your objection to the term “strawman."

~~~
Woost
I read it as the first form. I was trying to offer at least one example of why
I do not think TDD is the OTM. I read the criminal part of the OP as
hyperbole. The article does say there are uses for TDD, (not sure how to do
quotes on HN)

"Writing tests first as a tool to be deployed where it works is "Developer
Driven Testing" - focusing on making the developer more productive by choosing
the right tool for the job. Generalizing a bunch of testing rules and saying
This Is The One True Way Even When It Isn't - that's not right."

------
fadzlan
I am hoping for a more detailed discussions on which case writing test first
_hinders_ productivity.

Being dogmatic and using One True Approach for everything is not good, but if
we were to advance ourselves, we need to be able to define what is good and
what is bad specifically. Then, fruitful discussion can follows.

Blanket accusation like this would not help no one and is just crying for
attention, IMO.

~~~
peteretep
Hey, thanks for your comments. Perhaps I should have made more of that in the
article. Specifically, I find TDD to be completely unsuitable for any process
where you're not 100% wedded to what the result needs to look like at the
outset.

Writing tests first is great for - for example - writing a regression test for
a bug you've found. Or for adding a small piece of well-defined functionality
to an existing interface.

But that's writing tests first in a specific instance, not Test-Driven
Development, which is ALWAYS writing tests first.

Writing tests first falls down as soon as your ideas on what needs to be
implemented may change as implementation progresses. You find yourself
reluctant to change your interface or implementation because dang-it, the test
said it should work that way, and now you don't want to change the test. Or
you wrote a test for a simple piece of sub-functionality, and then it turns
out that relied on an architecture you don't want to use and ...

And if you're not a senior dev with a bucket load of experience, you're
unlikely at that point to want to go back and change things.

~~~
rssvihla
I hope you can appreciate TDD is not terrible for all developers and that
possibly there is something useful in there for some people. As you imply
there are many paths to quality software. You risk being as guilty as those
you criticize if you try and claim others are "less than" that can produce
decent code in a different way than you believe is possible.

TDD definitely has been _very_ positive for my software and there really is no
comparison between what I did before and what I've done since as far as bug
rates, customer satisfaction, and time to completion. I'm sure you'd agree
that those are positive outcomes.

I appreciate you've had lousy examples and that maybe even the majority of TDD
practiced is a lousy example. However, the examples you give and what you're
describing from these consultants is not remotely the way I've done TDD for
the last 5 years. I work extremely hard on refining the concept of testing and
when and where to apply different approaches.

As for "you're unlikely at that point to want to go back and change things". I
make way more changes in my code and tests before TDD than without. Granted I
did spend a year learning how to write tests that hit the right boundaries and
wouldn't break everything when I did change architecture strongly. I think any
developer needs to dive deep into HOW TO TEST anyway test first or after.

Also, I don't know why anyone would have a hard time deleting code that no
longer applied. There is always source control if you really feel like you
need it again.

as for "and then it turns out that you relied on an architecture that you
don't want to use"

Many TDD practitioners have this concept called "spikes" which is code that
you write without tests to get a good idea of how that particular algorithm
will work for you and what approach you want to take. However, its throw away
code that's is often very procedural and is more just thinking through an
issue. This minimizes some of the shifting architecture pain you're referring
to.

------
raganwald
There is something ironic about an article that begins: _When I hear someone
start advocating Test-Driven Development as the One True Programming
Methodology, that's a red flag, and I start to assume you're either a shitty
(or inexperienced) programmer,_ and carries on to say that _The whole concept
of Test-Driven Development is hocus, and embracing it as your philosophy,
criminal._

To summarize, the author seems to have the One True Programmer Evaluation
Methodology, and is able to determine that someone is shitty, inexperienced,
or a criminal from their answer to a single question. How is this different
from someone who says that they can determine whether your project is shitty
from the answer to the question “Do you use TDD?”

My feeling is that the author has a great deal in common with people who say
that X is the One True Programming Methodology, for _any_ value of X,
including TDD.

~~~
peteretep
I'll be honest and say the article started with the most inane troll about
people who run their own mail servers and have beards, and was quickly changed
when I realized people weren't reading beyond that ;-) Clearly my sense of
humour needs work!

~~~
Ejc3
Well, I for one thought it funny (read your post via zite at first), and the
rest of your article was a nice take on the TDD philosophy and it's adherents.
It should merely be another tool in one's tool chest. Maybe a less
confrontational headline would have produced a more measured response.

------
justin_vanw
Why does the author pick out TDD here? All he says is that "People that say
TDD is perfect and the best technique in all situations are wrong."

Once you establish that 'there are no silver bullets', you don't have to go
around to every bullet and say 'that bullet isn't silver, either!'

~~~
peteretep
Because I teach people how to write tests effectively, and run in to this TDD
silver bullet bullshit over and over :-)

------
flink
"Testing is a tool for helping you, not for using to engage in a 'more pious
than thou' dick-swinging my Cucumber is bigger than yours idiocy. Testing is
about giving you the developer useful and quick feedback about if you're on
the right path, and if you've broken something, and for warning people who
come after you if they've broken something. It's not an arcane methodology
that somehow has some magical 'making your code better' side-effect..."

I liked this part; it made me laugh. I also get red flags whenever I hear "X
is the one true way to do Y" statements. Personally, I don't think that there
is much in life that is so black and white that there could be "one true way"
to do anything.

My understanding of the original agile methodologies was that each
team/company needed to do what worked for them. Sure, you could follow some
book to the letter and hire some certified SCRUM master but odds are, some
deviations from the written plan are needed for the best results.

Either way, I'm generally a fan of TDD, test automation and metrics ... where
they make sense and where they aren't going to be abused. Use tools and
methodologies that make sense for the current situation, not just because some
book/person tells you that you'll fail unless you use them.

------
jameskilton
Nothing but flame-bait here. The OP obviously doesn't understand Test Driven
Development or why people swear by it.

It's first and foremost a design tool. Secondary to that it tests
functionality of your code.

TDD leads to better designed software. Period. No it's not the _only_ way to
design software, nor should it be the only tool used when designing the
software you're writing, but it _will_ show problems with any design you've
got and help you fix them.

~~~
cbs
>It's first and foremost a design tool.

Care to expand on that? Other than forcing low coupling, I don't really see
what TDD does for the design of the code, especially because it emphasizes
unit testing, and those are pretty small in the scope of overall design. And
even the low coupling is debatable, a good test framework is going to have
enough hooks to be able to isolate the unit under test to the point that the
surrounding design doesn't really matter to the test.

~~~
rssvihla
I can put in 2 cents here. It makes you think about your API up front. Can you
do that others ways (write your documentation first, etc)? Of course so do the
approach that works for you. TDD just happens to be my preferred approach.

As for "especially because it emphasizes unit testing" that doesn't mean it
frowns on integration or acceptance testing. In fact most practitioners I know
write high level end to end tests for the core functionality.

------
phamilton
"Even if you write only some tests first, if you want to do it meaningfully,
then you either need to zoom down in to tiny bits of functionality first in
order to be able to write those tests, or you write a test that requires most
of the software to be finished, or you cheat and fudge it."

I think this guy doesn't understand the concept of mocking and stubbing in
your tests.

TDD works great in bottom up development ("zooming down to tiny bits of
functionality"), but it also works great in top down via stubbing.

I feel like his complaint isn't against TDD, but about badly written tests. I
think we can all agree: Crappy code bad. Nice code good. Both in tests and in
production.

------
phzbOx
Still, starting with testing helps so much to create modular design because it
forces you to write clear/clean interfaces between them.

My workflow is usually:

    
    
      1. Write the tests and design the module/class/whatever in the same time.
      2. Code it. (Repeat)
    

Although, I rarely write _unit tests_ in TDD; only tests providing a high
level design view (Such as functional or integration tests).

~~~
peteretep
And perhaps I didn't make my points clearly enough in the article; I was
really trying to differentiate between the Test-Driven Development dogma as a
panacea for all development problems ever, and developers writing tests first
where it makes sense to do that.

~~~
cbs
>And perhaps I didn't make my points clearly enough in the article

You made them more than clear. People just want to argue the headline. Good
post BTW I'm subscribing to your RSS feed.

------
desireco42
:) Flame-bait. Seems that Hacker News became a place for such things. Blog
where this article was written was made so it can post there while hiding it's
identity. If you want to make fun of 'agile consultants' go ahead, but stick
with it with your name.

~~~
peteretep
"Posted by Peter Sergeant at 02:30"

~~~
desireco42
Yeah, so you don't have a blog, yet you have opinion :). You can down-vote all
you want, only thing your post shows is that you are struggling with test
driven development. You should learn it well, and then you will be in position
to say what part you don't think is useful. And to be constructive, I would
suggest instead of trying with unit tests, maybe go with integration testing
first and just do that initially, you might enjoy it more.

~~~
gambler
This is awfully generic. Almost a template.

"You can down-vote all you want, only thing your post shows is that you are
struggling with _insert development practice here_ development. You should
learn it well, and then you will be in position to say what part you don't
think is useful."

The blog post, on the other hand, is not generic. I've had similar
experiences: in many cases TDD makes trivial coding issues influence overall
application design (in a bad way) instead of application design driving those
trivial decisions. I can even tell you when this happens. It happens when the
complexity in the application comes mainly from structuring large chunks of
mostly trivial functionality. I've had pretty positive TDD experience with
other type of applications - ones where complexity comes mainly from some
data-processing algorithms, while the overall structure of the app is fairly
simple. (Think web app vs language parser.)

