
Why I don’t use TDD or BDD - jerguismi
http://blog.kangasbros.fi/?p=30
======
elbenshira
Is this article satire? His "reasons" are completely invalid.

>>> “Examples” of unit tests suck donkey balls every time.

So lack of good examples imply that the whole system is useless? Yes, an
example of "assert True" is useless, but just download any of the thousands of
open source projects that use unit tests and look at the code.

>>> They seem to be a great way to spend some extra time doing time consuming,
not very hard work.

It's not that time consuming, really. And it being easy work is a _good_
thing. A lot of benefit for easy work!

>>> You have to think about the “evil” business side too.

Running away from your problems is never a good idea. Face the "evil"
(whatever that means) side like a man.

>>> Unit tests=more code

OK, there is a difference between useless code and good code. Unit tests
doesn't make your system any more complicated or harder to read. In fact, it
makes your system more stable and _easier_ to read.

>>> That is mostly because I’m a lazy-ass bastard.

Ah, so we get to the bottom of the problem.

~~~
dget
>>> Face the "evil" (whatever that means) side like a man.

What? It's not a battle or a problem. Sometimes you need to get something out,
or you're writing a little throwaway bit of code. Sometimes that involves a
tradeoff in future maintainability/pain. It shouldn't justify never testing,
but I think it's a valid reason to not religiously follow TDD.

~~~
kscaldef
But the article isn't about not religiously following TDD. It's about why "I
don’t write any unit tests or specs". Testing isn't discontinuously one or the
other. There's a huge, continuous spectrum in between the two extremes.

~~~
dalke
He said that integration tests cover the tests he needs and that unit tests
don't add much, while being time consuming. And he never said one word about
"spec".

~~~
kscaldef
The article has been edited since I wrote my comment. That was an exact quote
from the article at the time. (And I'm pretty sure there was no mention of
integration testing at that point, either.)

~~~
dalke
Ahh, the joys of web page impermanence.

------
derefr
The only real way there has ever been to make sure the computer is following
your instructions, is to tell those instructions to it twice. (Non-
inferential) static typing is about telling the computer the input and output
types of your functions twice. Unit tests are about telling the computer the
algorithm (or an equivalent, such as a direct mapping function from a subset
of example inputs to pre-calculated outputs), twice. These are both just
protoforms of Reliable Computing—the name for the actual software
_engineering_ that NASA and friends do, where they make the whole _system_
twice (or more) and then run those systems in parallel to ensure they're
always "voting" the same way.

But sometimes you don't need _engineering_ ; sometimes you just need a dirty
hack. Hacking is basically the opposite of engineering: it espouses DSLs and
macros that encode everything exactly once, it prides elegance and other
aesthetic factors of code above engineering tolerances. And, of course, it's a
lot more fun to do—so it gets people interested in your code, which can
actually be the best thing, even more so than engineering it. Linux is a big
hack, and because of that all sorts of people enjoy working with it. _why was
an eminent hacker, and because of that his code (and tutorials) are still used
when getting people interested in coding. None of this code is _guaranteed_ to
work, unlike code that has been engineered—but that's okay, in 99% of cases.

~~~
tomstuart
To elaborate: you tell the computer a specification ("what to do"), you tell
the computer an implementation ("how to do it"), and then you get the computer
to check that the implementation conforms to the specification.

------
dlsspy
Summary: ``I don't get it.''

The parts of my code bases that are the most trouble and the hardest to work
with and the parts that have the least testing have the greatest overlap.

I have some bad tests that fail sporadically, but I also have some tests that
fail that indicate something is actually wrong and will show up in a
production environment in a really, really bad way some day (speaking from
experience).

If you're _why, I think you've got a good case for why not to write unit
tests. Unfortunately, nobody is _why

------
petercooper
I'm ambivalent about testing. I've no fear of not writing tests for something
small or scrappy. But.. when I've persevered, weeks into a project they've
saved my ass more than once and I've been able to recognize situations where a
bug would have previously forced me to spend hours debugging or to release
broken software.

TDD isn't a one size fits all technique. But just because it takes time,
doesn't always yield immediate benefits, and can leave a bad taste in the
mouth at first, it doesn't mean you wouldn't ultimately benefit (even _beer_
shares these negatives ;-)).

The great thing, though, is that there are so many ways to practice testing.
If you don't want to go with full on "test first" TDD practices, even rolling
out a handful of automatic integration tests could cover your most onerous
painpoints (this is my strategy on old projects that have no tests -
something's better than nothing). Though TDD's proponents foam at the mouth
about _always_ testing, it's not a binary option - you can wade in the shallow
end and get into deeper waters over time as it begins to click.

------
marknutter
Here's my take on testing:

I'm the kind of guy who likes to get things working as fast as possible. When
I'm excited about an app or feature idea, I can't be bothered with writing
tests along the way, much less _before_ I write my code. I get much more
satisfaction out of writing code, running it in the browser, and testing it
the old fashioned way. I build apps and features _quickly_ and this usually
makes me (or the client I'm building it for) very happy in the process.

I do, however, understand the value of testing. Unlike the OP I _have_ seen
real world cases where testing has been a godsend. I often feel guilty for not
writing tests, even if it's just a toy app I'm developing for my own
curiosity.

To balance these opposing tendencies of mine, I have taken a ATDD approach to
testing, or "Adaptive Test Driven Development". That is, write tests for the
most critical components of the site, such as anything dealing with money or
sensitive information, and write tests for the parts of the app that get used
the most, both by me when I'm developing them, and the users once it's
deployed.

When I've gone to the browser for the 5th time to test a feature I'm
developing, it's a pretty good sign that an automated test would be more
efficient than my manual testing, so I set aside the time to write it. If I
know there's a part of the app that errors will be completely detrimental to,
I will write tests for that, if only for my own psyche.

To be completely honest, TDD/BDD and some of the rhetoric surrounding it seems
a bit ivory-towerish - a way for the templars of hacking to distinguish
themselves from the proletariat developers. If I had been told TDD is the only
way I could develop a rails app when I first started doing it I probably would
have given up pretty early on. So if TDD is getting in the way of your being
productive, it kind of defeats the purpose of it even existing.

The important thing is to strike a balance that works for you, and keep an
open mind about other developer's habits and tendencies.

------
heresy
Use automated tests for any software where the probability of you working on
it in the future is > 0.

Any testing is better than none.

Good testing is better than bad.

Good tests;

* Are relatively simple to write if the design is decent

* Make clear what is being tested

* Do not break if something unrelated changes

* Run quickly

Understand the difference between unit testing, and integration testing.
Chances are, you're doing the latter, thinking you're doing the former, and
wondering why it's so much work.

Lastly, know when TDD and BDD is appropriate.

For me, it is around the 25% mark; earlier, and I spend too much time fixing
the tests as the design goalposts change, later, and I spend too much time
fixing bugs that could have been prevented by tests.

------
jules
It is very much dependent on the kind of code you write. If you write UI code
then automated testing might not help you much. But for algorithmic code you
can usually write short tests that test very comprehensively and catch nearly
all bugs.

For example if you are writing a sorting function you write some code to
generate a random array, sort it, and test whether it is sorted.

    
    
        n = random int;
        A = int[n];
        for(i in 0..n) A[i] = random int;
        sort(A);
        for(i in 0..n-1) assert(A[i] <= A[i+1]);
    

Running this kind of test saves you a heck of a lot of time manually testing
scenarios, and it tests much more comprehensively too. Tools like QuickCheck
make this style of testing even easier by providing ways to randomly generate
things and if it finds a bug it tries to get a small example that produces
buggy behavior.

I do not see much value in the kind of testing a lot of people seem to do
though, for example in a web application putting something in the database and
then testing that it's there. If the test is about as complicated or more
complicated than the thing it's testing that is a good sign the test is not
very useful.

------
steveklabnik
The entire point of writing software is to automate processes that can be done
manually, but are too time-consuming and/or hard to do so. Testing is a great
example: yes, I could run through my entire application and hand-test it every
time I make a change, or I could automate the process and save myself a ton of
time and work.

Viewpoints like these are the reason we have a long way to go before software
is truly reliable.

~~~
kenjackson
The issue isn't about testing, it's about unit testing. And really more
specifically about BDD/TDD.

And I must admit that I've struggled with TDD myself. And I've generally hit a
couple of issues:

1) Bug rates in test code for some reason seem to be higher than that in
production code. Not sure why. But the end result is a LOT more test bugs than
product bugs. I've sat in meetings where people wanted to block shipping the
product because of test bugs. No evidence there were any blocking issues in
production code.

2) The opportunity cost of TDD style unit tests are rather high. Given the
choice I think I'd rather a more in-depth design and code review.

While I think unit tests are valuable, I take a more sparse notion of them.
Based on code review and design, unit test the code that you have worries
about. Where you think there's a greater chance of a seemingly innocuous
change breaking the code.

I've seen little value, in the projects I've been a part of, with the BDD/TDD
approach. It does seem like a lot of busy work and not a lot thought put into
where the value is. And there's actually pretty good industry data (and I
suspect open source data as well) that its actually not hard to determine what
parts of a program are going to be the buggiest in production. I'd really love
to see a test approach that is density driven...

But with that said, if you're writing a 20,000 line web app, TDD/BDD is
probably fine. You can generally kick out the code and get full test coverage
in very short order.

~~~
steveklabnik
One caveat: I come from the Ruby world, which has top-notch testing support.
So maybe it's horrible in other-platform-land.

I think that it is slow going, at first. Testing is a different skill than
regular coding, and at first it goes slow. But now, I spend much less time
writing tests because I already know how to properly write them.

It's hard to properly evaluate the amount of time tests save you because it's
an up-front investment. But I can't count the number of times I've wanted to
change something fairly large in my codebase, and did so fearlessly, because
then all my tests broke and I was able to quickly fix things where they were
broken. In one recent project I decided to completely re-do the authentication
system, and it only took me about an hour, because I already had an exact list
of every place that needed attention, due to having great test coverage.

> It does seem like a lot of busy work and not a lot thought put into where
> the value is.

I see it the other way: if code has to be written, it should also be tested.
If it's valuable enough to have spend the time to implement, it's worth making
sure that it works correctly.

~~~
kenjackson
"But I can't count the number of times I've wanted to change something fairly
large in my codebase, and did so fearlessly, because then all my tests broke
and I was able to quickly fix things where they were broken "

To me that is a bit scary. Tests don't prove the absence of bugs. They can
only prove the presence of them.

I never make big changes, even in the most tested pieces of code that have
unit, integration, UI, and manual testing without some trepidation. Because
the fact of the matter is that these code paths get the reputation they get
because a customer comes back and says this doesn't work and we're like,
"oh... that's how you use it?"

"I see it the other way: if code has to be written, it should also be tested.
If it's valuable enough to have spend the time to implement, it's worth making
sure that it works correctly."

I don't disagree. It should be tested. I think our disagreement is if TDD/BDD
style unit testing is the best way to do so.

~~~
steveklabnik
> Tests don't prove the absence of bugs. They can only prove the presence of
> them.

Absolutely, and having a list of where they were (now) present was a real
productivity boon.

It's true, I think we're arguing over something very small. I don't think
automated testing is the end-all be-all. But I do think it's much much better
than not doing it.

~~~
kenjackson
I do want to be clear. That I'm not arguing against "automated testing". It's
really against a style of unit test development that is all the rage: TDD/BDD.
The devs I've worked who practice it, is through the use something like
Cucumber to drive the development of the application with tons of unit tests.

I'm a huge advocate of automated testing. And again, just in my experience,
the productivity and perceived code quality I saw coming out of the BDD/TDD
side of the house didn't seem to be a step in the right direction.

------
helmut_hed
I write unit tests so that I can refactor _fearlessly_. How does the author
handle situations where he can either 1) put in a hack to fix a bug or handle
a feature request, or 2) restructure the code to make it fit the modified
problem description better? If you have unit tests you can do the latter, and
feel good about what you have. If you don't have a testing infrastructure, you
must do the first, because you have no confidence about how the behavior of
the code might change and you have to be paranoid.

~~~
dalke
I don't understand how anyone can be fearless during refactoring. Suppose I
refactor string search from the simple version (worst case time O(n*k)) to
Knuth-Morris-Pratt (worst cast time O(n+k)). The API is unchanged, so this is
an excellent example of a refactoring. But the KMP implementation has so many
more edge cases where I might make a mistake that I wouldn't be comfortable
using only the tests uses to drive the simple version.

I would be happier with code coverage-based tests, or better, branch coverage-
based tests. That would help tell me which parts of my new implementations
need test cases, although it wouldn't be sufficient.

Would you really be fearless for this type of refactoring? And how does
following TDD practices help out? It seems to make the fallacy that
refactoring is trivial and therefore doesn't need additional testing.

------
yish
On small projects where one has complete ownership and knowledge of the code,
one can get by without testing. When you have a large project with tens to
hundreds of developers, automated testing gives the developers the confidence
and comfort to make large or deep changes with an accurate measure as to the
overall stability of the product, even the areas where an individual developer
may have little domain knowledge of.

~~~
cageface
Agreed. I submitted a couple of patches to Rails recently and I would have
been far more reluctant to try to make fixes if I hadn't been able to verify
them with a fairly extensive test suite.

People get too dogmatic about testing and particular testing methodologies but
testing in general is a very powerful and useful tool when used wisely.

------
gnuvince
> I just personally haven’t found any use for them in any of my projects or my
> clients’ projects.

I wonder how that person makes sure that what they deliver actually works.
Testing by hand can only get you so far. And when the client calls a year
later with some change requests, tests are a great way to know that you
haven't fucked up something that used to work because you forgot a small
implementation detail.

------
lzw
I think that sometimes people lose sight of the fact that other people are
different. You may not care for sports cars while someone else might. Doesn't
make one of you superior to the other. Neither one of you can say that the
other is wrong and should have made the choice you did.

It is the same thing with many programming methedologies, including unit
tests. Maybe unit tests work well for large companies, and I would think they
might be ideal for parts of certain types of server applications.

But I have tried the methodology, and found that it slows me down and produces
lower quality code. For me, it produces a lot of false positives as the some
number of unit tests become out of date each sprint, and the significantly
increased overhead slows down development significantly which means less
quality functionality per unit time gets developed. I've tested this theory
with a real product written %100 solely by me, and found that the actual
results in production where there were no unit tests and no design documents
(but lots of design "notes") resulted in a shocking result: Not a single
exception over a large deployment among thousands of customers and many months
and only one reported bug which at the end of the day is actually working
correctly but the design decision was not clearly thought out and the software
does something the user doesn't expect in a corner case.

Here I see people talking about refusing to hire people who don't believe in
Unit tests, and everywhere else on the planet I see people talking about Unit
tests as if they are a religion, and as if nobody could write competent code
without using them.

But realize this, every programmer is different, and the path to the highest
quality code per unit time may not be via unit tests.

------
aneth
This guy has everything backwards. I sort of hate writing tests, but still
write them first most of the time. In the end, they catch so many bugs and
save so much time, I would never go back to hacking code and point and click
testing. The only way manual testing works is if you have an over-financed
company with a huge QA department.

------
ruby_rocker
boring ...

------
spooneybarger
I like it when people write blogs like this. One question to determine if it
was bad satire or not and often times, the interview is over. Makes hiring so
much easier.

