(a) in the "driven" part, not the "test" part. Tests are (in general) a good thing. However, using a series of tests to drive your design (aka "TDD is not a testing method,it is a design method" idea) often gives you an illusion of progress as you hill climb using conformance to an increasing number of tests as a progress heuristic and end up on top of a local maximum (as for example in the TDD sudoku episode).
(b)in conflating TDD with one or more of (1) testing, (2) automated testing (3) automated regression test suites (4) developers adding more tests to the automated regression test suite as they develop more features, refactor, debug etc.
You can have (1) to (4) without either (5)writing tests first (aka "don't write a line of code without having written a test covering it") or (6)driving your design with tests. The last two ideas are the real distinguishing features of TDD and are of debatable merit. None of (1) through (4) are novel ideas. (5) and (6) are where differences of opinion happen.
Even if you choose to use TDD, it is good to be aware it is just one tool in your toolbox and not necessarily the default tool to reach for.
(c) in the zealotry of some of its evangelists who insist that TDD is some kind of moral imperative and is the only "correct" way of developing software and anyone who doesn't follow that path or make respectful obeisance to it is "unprofessional","dodgy" etc. This is often accompanied by conflating TDD with more generic notions like "automated tests" etc as above.
For example, Rich Hickey, the author of Clojure, said recently at the Strange Loop conference "We say, “I can make a change because I have tests.” Who does that? Who drives their car around banging into the guard rails!?"
(and that is all he said. One sentence in a keynote presentation)
For this Hickey was taken to task by a TDD advocate, Brian Marick, for not being "respectful" enough to TDD and for his "tone" in daring to mildly criticize it as a development practice. After some tweets complaining about Rich Hickey's tone driving away people from Clojure etc he wrote
"The dodgy attitudes come from the Clojure core, especially Rich Hickey himself, I’m sad to say."
This kind of repeated whining and harassment over a few days made the normally unflappable Hickey (who asked for references to his "disrespect" etc, to the sound of crickets) lose his temper and say (on his twitter stream)
"If people get offended when their tools/methods are criticized as being inadequate for certain purposes, they need to relax.",
and "Testing is not a strawman. It's an activity, it has benefits, but is not a panacea. No 'community' should be threatened by that fact"
and later "Accusing people who merely disagree with you of being snarky, intolerant, dismissive etc is both wrong and destructive."
and much later after being subjected to a barrage of tweets criticizing his tone and 'lack of respect' for TDD, "If launching an ad hominem attack is the product of a lot of thought, it is time for you to move on. Good riddance."
This comment should not be [dead] (if you don't want to give/dock me karma for irahul's comment, down-/upvote my reply to this comment):
irahul 7 hours ago | link [dead]
> For example, Rich Hickey, the author of Clojure, said recently at the Strange Loop conference "We say, “I can make a change because I have tests.” Who does that? Who drives their car around banging into the guard rails!?"
Rich has spoken about it other time with an interview with Fogus:
Hickey: I never spoke out ‘against’ TDD. What I have said is, life is short and there are only a finite number of hours in a day. So, we have to make choices about how we spend our time. If we spend it writing tests, that is time we are not spending doing something else. Each of us needs to assess how best to spend our time in order to maximize our results, both in quantity and quality. If people think that spending fifty percent of their time writing tests maximizes their results—okay for them. I’m sure that’s not true for me—I’d rather spend that time thinking about my problem. I’m certain that, for me, this produces better solutions, with fewer defects, than any other use of my time. A bad design with a complete test suite is still a bad design.
He said something on the similar lines about development on CLR:
Fogus: Clojure was once in parallel development on both the JVM and the CLR, why did you eventually decide to focus in on the former? Hickey: I got tired of doing everything twice, and wanted instead to do twice as much.
His explanation on both fronts boil down to he doesn't find it(TDD and CLR/JVM parallel development) a worthy investment of time, given there are only so many hours in a day.
I don't understand why TDD advocates get all worked up when someone says TDD doesn't work from them. Well, if TDD is silver bullet of software development, they should be delighted that the ignorant singletons fail to see it and they have an edge over the fools.
These reactions remind of this:
“You are never dedicated to something you have complete confidence in. (No one is fanatically shouting that the sun is going to rise tomorrow. They know it's going to rise tomorrow.) When people are fanatically dedicated to political or religious faiths or any other kinds of dogmas or goals, it's always because these dogmas or goals are in doubt.” ― Robert M. Pirsig, Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values
> the zealotry of some of its evangelists who insist that TDD is some kind of moral imperative and is the only "correct" way of developing software and anyone who doesn't follow that path or make respectful obeisance to it is "unprofessional","dodgy" etc.
Well, it is (the only correct way of developing software). If you're writing software without a test, even if only a mental one, you're just goofing off. And if it is mental, write it down and let the computer check it for you.
It's like not writing specs. (Not necessarily the executable type.) If you don't know what you're trying to do how do you know when you're done?
As for what people who confuse specs with acceptance tests think, I wouldn't judge a product/methodology by those who don't use it well.
I don't get why the fact that TDD drives good design is a bad thing. When you're not testing your code, it's easy to start taking shortcuts and coupling modules that should have been kept separated. TDD forces you to think about "modular design", which is just great. Of course, if you're trying lots of solutions to find "the right one", don't write tests. Just write hacky prototypes as fast as you can.
"(b)in conflating TDD with one or more of (1) testing, (2) automated testing (3) automated regression test suites (4) developers adding more tests to the automated regression test suite as they develop more features, refactor, debug etc.
You can have (1) to (4) without either (5)writing tests first (aka "don't write a line..."
I get here that you're being sarcastic; but there's a difference between using TDD effectively to get the job done and using it "because it's cool" or "because people say it's nice". You don't have to write tests for every trivial things, or write tests for your tests of your tests. Lots of time, writing high level tests testing for the interfaces is a good time/quality ratio.. there's nothing wrong with that.
About c). I do agree. However, have you ever jumped into a project without tests? With tests? Which one do you prefer? Tests don't prove that the library is working perfectly; they only prove that what's tested is working. Personally, jumping into a new project that have tests give the confidence to change things and know I won't break anything. Of course, that's not always the case but I think you know what I mean.
And about Hickeys.. I haven't heard the speech and don't know about the context. But changing stuff in your language is something; but again, starting to change stuff in someone else' is different.
Just because you're not doing TDD doesn't mean you're not testing your code. I don't need to be forced to think about modular code, because I (try to) think that way voluntarily.
One of the most frustrating things (to me) about BDD/TDD is that I write a test knowing it will fail (a good thing), then I implement just enough to pass that test. Even though I know that I'll be rewriting that code again very soon to implement just enough more to pass some other test. It's needless context switching when I have a general idea of how I want the code structured before I start writing anyway.
With more experience, you should learn to avoid many of the pitfalls of not writing/designing modularly enough in the first place and at that point writing your tests first becomes a productivity crutch.
You forgot a critical step. Refactoring. Refactoring is a critical part of TDD, one which many people seem to forget.
> It's needless context switching when I have a general idea of how I want the code structured before I start writing anyway.
That's good. You are allowed to have that. In fact, your first test is implementing initial parts of that code. What TDD is force you to document that in a formal manner. It also highlights where things become cumbersome. If it's hard to write a test, your code is probably far too complex for what it's trying to do.
Finally, knowing what you want is different then having what you need. TDD gives the opportunity to focus on getting a finalized code base faster by not having to implement things that are unnecessary.
But really, when you describe TDD as write failing test, then write code to pass the test, and leave it at that, you've left out an essential pieces of the steps that is just as important as writing the test and writing the code. It's akin to me describing TDD as writing code and then refactoring, and leaving out testing.
Unit tests are not formal documentation. Nor are they generally adequate documentation. At best they will explain what the code does, but never how or why.
No, it isn't formal documentation. However, unlike other forms of documentation, it can be proven to be correct. If you want to know how the element being tested is to be used, unit tests are a great way of discovering that. It also tells you what it's supposed to do. Finally, it can tell you all this quickly and efficiently.
So, essentially, it gives you what the code does, as well as how to do it using the code. It doesn't tell you how it does this beneath the API, or why it does what it does, but it doesn't have to.
Your unit test beliefs are giving you a heavy bias.
Unit tests cannot be proven to be correct (not without some other actual formal proof). Running a suite of unit tests proves nothing except that the tests pass. The unit tests can be buggy. The code they test can also still be buggy. Indeed the code is buggy if it's nontrivial, unless you are asserting that unit tests end the very existence of bugs. Unit tests add more confidence about the state of the code, especially with respect to regressions, but they do not prove anything.
As for unit tests telling you how to use the code, I suppose they do to some extent. Your code should probably be clear enough without this, though. If I have to read your unit tests to know how to use your class, then you have failed at writing self-documenting code, and you've also failed at writing API documentation.
I would also say that the how and why are often extremely important. Anyone maintaining your code (i.e. Anyone who cares about your unit tests) needs to understand the how. Anyone using your code probably needs to understand the why. If your calendar unit tests indicate that certain days have 25 hours, but fail to explain that these are due to daylight savings time, that's a pretty important missing why.
> Running a suite of unit tests proves nothing except that the tests pass.
Which, when compared to documentation, is light years ahead in terms or proof.
> Your code should probably be clear enough without this, though. If I have to read your unit tests to know how to use your class, then you have failed at writing self-documenting code, and you've also failed at writing API documentation.
sigh
Well, apparently, if my API is clean enough, then you shouldn't need API documentation. Right?
Regardless, a clean API can be self documenting, but having tests demonstrating all the forms and intents of the classes can help with precisely what to do. As for code being clear: What does code have to do with an API? The whole point is to avoid actually having to look at the implementation of the API.
> I would also say that the how and why are often extremely important. Anyone maintaining your code (i.e. Anyone who cares about your unit tests) needs to understand the how. Anyone using your code probably needs to understand the why. If your calendar unit tests indicate that certain days have 25 hours, but fail to explain that these are due to daylight savings time, that's a pretty important missing why.
These are two different issues entirely. One has nothing to do with the other. Regardless, documenting 25 hours doesn't change the fact that changing it requires testing. It's as simple as that.
You seem to be playing straw man with your first argument, you're just confused with your second, and targeting something that has nothing to do with what we are discussing.
It's like me bashing git because it doesn't compile your code.
> Which, when compared to documentation, is light years ahead in terms or proof.
In three replies you've gone from calling unit tests formal documentation to saying that they are not documentation at all. The fact that you don't see the problem with this is evidence of the bias I mentioned.
> Well, apparently, if my API is clean enough, then you shouldn't need API documentation. Right?
Sure. In an ideal world, the API would be so obvious that further documentation is redundant. That's often not realistic, though, so we have additional documentation to shore up the API. (In modern code, the bulk of this is actually in comments inside the API interface.)
> Regardless, a clean API can be self documenting, but having tests demonstrating all the forms and intents of the classes can help with precisely what to do. As for code being clear: What does code have to do with an API? The whole point is to avoid actually having to look at the implementation of the API.
So instead of looking at the API itself, it's more appropriate to look at unit tests? It's better to look at 'testWiggleWidgetThrowsOnNull()' than to just look at the API and see that 'wiggleWidget()' throws on null? Or better yet, look at the documentation that should state that null is not a valid argument?
Unit tests are not good documentation. Good documentation is intended for humans to read. Unit tests are intended to test code, not to convey information. They generally contain a ton of noise in the form of boilerplate and mock objects.
> These are two different issues entirely. One has nothing to do with the other. Regardless, documenting 25 hours doesn't change the fact that changing it requires testing. It's as simple as that.
Of course. I've not argued against testing. I don't believe unit tests are always appropriate, though, and I think the argument that they are documentation is really weak.
> You seem to be playing straw man with your first argument, you're just confused with your second, and targeting something that has nothing to do with what we are discussing.
I'm not sure exactly what you're referring to as my "first" and "second" argument. It's absolutely not a strawman to say that unit tests don't prove code correct. It's a fallacious claim on your part to say that they do. A unit test passing only proves that the code does what the unit test expects. It doesn't prove that the unit test expects the right thing, nor does it prove that the code is being tested where it actually matters.
If the second argument is about documentation, I stand by my assertion that unit tests are not good documentation. In fact, I would say that they are terrible documentation, for the reasons I outlined above.
> It's like me bashing git because it doesn't compile your code.
Which would be a pretty reasonable response to someone claiming that git is a great compiler.
My point is that it does not necessarily drive good design at all. TDD can lead to crappy code just as easily as any other methodology, it is not a silver bullet that will 'in concert with other stuff' always lead to good design.
It may lead to software that performs correctly but that is not the same things as well designed software.
For instance, you might get software that performs terribly, but still correct.
Actually, I'll disagree. TDD revolves around 3 steps. Unfortunately, people forget the 3rd, which is refactoring. By it's very nature, the 3 step process leads to good code. The problem is when people start to skip that ever important 3rd step.
Countless times I've seen so called TDD evolve into merely writing a test, and then writing the code to pass the test, and then ignoring the 3rd step of the process.
Regardless, you are misrepresenting what is being said:
> it is not a silver bullet that will 'in concert with other stuff' always lead to good design.
And no one is suggesting that. You are inferring it. You are being religious in your dislike of TDD zealotry to the point that you are assuming that suggesting that TDD drives good design means using TDD "with other stuff" always leads to good design.
Rather, TDD drives good design means simply that by properly practicing TDD, you are more likely to result in good design precisely because of how TDD works. That doesn't mean you can't sabotage yourself along the way. However, what I've found is that bad design from TDD is generally more difficult to achieve. If you are finding tests difficult to write, your generally going to find your design is bad. I see this all the time, from programmers who are really smart. We'd prefer to believe in our own brilliance then admit we were proven wrong by a mere methodology.
Side note: Their is a lot of zealotry in this thread. It's mostly those against TDD (you'll notice they use the word religious a lot). I'll admit that some can make the troubling leap that TDD is a silver bullet. But, if your instinct is to argue against TDD by claiming it's not a silver bullet, you're not any better. More importantly, why does TDD need to be a silver bullet for it to be worth while? If their is no silver bullet, then no methodology is worth practicing?
(a) in the "driven" part, not the "test" part. Tests are (in general) a good thing. However, using a series of tests to drive your design (aka "TDD is not a testing method,it is a design method" idea) often gives you an illusion of progress as you hill climb using conformance to an increasing number of tests as a progress heuristic and end up on top of a local maximum (as for example in the TDD sudoku episode).
(b)in conflating TDD with one or more of (1) testing, (2) automated testing (3) automated regression test suites (4) developers adding more tests to the automated regression test suite as they develop more features, refactor, debug etc.
You can have (1) to (4) without either (5)writing tests first (aka "don't write a line of code without having written a test covering it") or (6)driving your design with tests. The last two ideas are the real distinguishing features of TDD and are of debatable merit. None of (1) through (4) are novel ideas. (5) and (6) are where differences of opinion happen.
Even if you choose to use TDD, it is good to be aware it is just one tool in your toolbox and not necessarily the default tool to reach for.
(c) in the zealotry of some of its evangelists who insist that TDD is some kind of moral imperative and is the only "correct" way of developing software and anyone who doesn't follow that path or make respectful obeisance to it is "unprofessional","dodgy" etc. This is often accompanied by conflating TDD with more generic notions like "automated tests" etc as above.
For example, Rich Hickey, the author of Clojure, said recently at the Strange Loop conference "We say, “I can make a change because I have tests.” Who does that? Who drives their car around banging into the guard rails!?"
(and that is all he said. One sentence in a keynote presentation)
For this Hickey was taken to task by a TDD advocate, Brian Marick, for not being "respectful" enough to TDD and for his "tone" in daring to mildly criticize it as a development practice. After some tweets complaining about Rich Hickey's tone driving away people from Clojure etc he wrote
http://www.exampler.com/blog/2011/09/29/my-clojure-problem
"The dodgy attitudes come from the Clojure core, especially Rich Hickey himself, I’m sad to say."
This kind of repeated whining and harassment over a few days made the normally unflappable Hickey (who asked for references to his "disrespect" etc, to the sound of crickets) lose his temper and say (on his twitter stream)
"If people get offended when their tools/methods are criticized as being inadequate for certain purposes, they need to relax.",
and "Testing is not a strawman. It's an activity, it has benefits, but is not a panacea. No 'community' should be threatened by that fact"
and later "Accusing people who merely disagree with you of being snarky, intolerant, dismissive etc is both wrong and destructive."
and much later after being subjected to a barrage of tweets criticizing his tone and 'lack of respect' for TDD, "If launching an ad hominem attack is the product of a lot of thought, it is time for you to move on. Good riddance."
postscript: the best criticism of TDD I've seen is at http://www.dalkescientific.com/writings/diary/archive/2009/1... . The responses at http://dalkescientific.blogspot.com/2009/12/problems-with-td... are (mildly) interesting as well.