Reading such a long article with such a moot point is a waste of time...
More seriously, the point of the article, from a skim, appears to be that "writing too many unit tests is a waste of time". Well, duh. Save 15 minutes of your life and pass on this one, methinks.
Hi, I'm the original author. That wasn't the point I was trying to make at all - despite the attention grabbing headline it's really talking about the value of functional and integration tests.
In retrospect I wish I'd used a different title and skipped the first few paragraphs - I'd intended it to be a bit of humorous indirection making it sound like the post was going to be against automated testing whereas it is really very much in favour of it, but unfortunately people have taken it the wrong way.
The internet is filled with authors who do that trick. It's a bait-and-switch, and I know I don't like reading it.
People didn't "take it the wrong way." You did not clearly express yourself. If you want to talk about the value of functional and integration tests, then talk about that.
Hi Scott - I admit it was a cheap trick. This is the first time I've had something like this posted to an audience so I've had one or two things to learn!
I'd appreciate it if anyone interested in the post would skip the first few paragraphs and start at "Benefits of automated functional tests" since that was what I was intending to convey.
I think a better idea is to just edit your essay directly. Maybe even re-submit the edited version to HN - you may find people are more receptive to discuss the content you wanted to convey.
I wouldn't necessarily discard the trick altogether. However, I'd say that based on my experiences/intuitions, it only works for short posts, and you have to clarify the point of the post by the time you get to the second paragraph. The trick was simply not the right thing for such a long article.
Packaging really matters where blog posts are concerned... the hook needs to be crafted well for each article, or people won't bite.
I'd take scott's advice and just repost... many people will probably read it then, since the subject is relevant.
I've changed the title on the post and added an Update to the top of it. I'd rather not resubmit to hacker news since I think it's taken up enough attention already. Thanks to you both for the suggestions.
Too bad you didn't have the stones to stand by your original claim. Most unit testing is a waste of time. A crazy orthodoxy has arisen around it: never question the value of massive unit testing, just do it! Test-driven development is cumbersome and frequently useless. If you're really doing trapeze acrobatics you need to work with a net, but most of us are just walking down the street, in which case rigging up a bunch of elaborate safety netting is just overkill.
I'd downmod you only for the last line of your post...
But back on topic. There is some research supporting unittesting (McConnell writes about it in Code Complete) it doesn't find even half of the existing bugs though, and it should be supported by other testing methods.
The main-big-very-nice advantage of unittesting is that any bug that is _does_ find is easily found and fixed.
I thought it was an interesting article. If you want to skip the long read, a better summary would be "you can save a lot of code and hassle by writing a few smart functional tests instead of a ton of unit tests."
I continue to assert that most unit tests are the same type of drudge work that good programmers try to avoid in other scenarios (data entry, text editing, email templates, deploy scripts, etc).
"I think sometimes we take for granted how fortunate we are as developers to be able to make such a direct impact on problems instead of being wholly dependent on others."
long article... moot point... unit & functional testing is important. My friend made an awesome analogy to the real world application here: http://app.arat.us/blog/?p=159
Good, but overly long article. I have reached the similar conclusions in favor of functional tests over the last decade or so.
Personally, I have worked seriously used DBC, TDD, and automated scenario testing, and hybrids in that order of learning.
My conclusions about the sweet-spot (and this may vary depending on your area):
1. Automated functional testing in the large, and DBC in the small is a winning combo.
2. Special techniques for graphics intensive programming: I get my scenario tests to show what's happening on the screen, and I can slow 'em down and step through 'em
3. I never got a lot of dependency injection / mocks (like the author) -- not sure if I don't get 'em or application area is wrong
4. You need special techniques for intensive algorithmic code where there's a combinatorial explosion of cases and line coverage won't do; e.g. randomly generated test cases + compute intensive sanity checks
Bottom-line: Automated scenario tests exercise the code; DBC pin-points illegal states.
DBC = Design by Contract = pre-conditions + post-conditions + invariants. In practice understanding the method (this takes work) and writing explicit pre-conditions gives ~70-80% of the benefit. To do invariants well needs language support.
Is there anyone who have really invested time to learn proper unit testing and still thinks it is not one of the best programming inventions ever? Seriously, there are lots of naive posts lately on HN...
Would you by chance also be able to post us toward a blog post that gives some convincing examples of why unit tests / TDD are so great. I have read many advocacy posts that basically repeat the same mantra of how awesome it is, and I have read ones that use simplistic examples, but I honestly don't see how they can be so compelling. Or, to be more precise, how their value exceeds their cost.
Whereas the automated functional testing advocated by the OP makes a tremendous amount of sense to me intuitively.
I don't particularly find unit testing to be effective in programming. While naivety is often a dominating factor, it does not have to be so. First, you should note that you might have a different notion of proper than other people. Second, I don't think you can rank programming inventions. Undoubtly, unit testing is useful. The question one should ask is rather: is it effective? In this question we also lace in the factor of time spent.
Note that the most important programmer commodity is time. Time spent on a unit test means less time to write the program. Now, what ought to tip the scale is that later changes are easier to get right so in the long run, one should win time by writing down unit tests.
But personally, I think that there are other practices which are more effective: Static typing, assertions and automated test harnesses for the whole program. I'll choose to spend my time on these rather than unit testing.
I like _Working Effectively with Legacy Code_ by Michael Feathers. It's a collection of techniques for bootstrapping tests into an existing code base - carefully, incrementally restructuring the code to add enough tests to safely perform deeper structural changes. At the same time, it's ultimately a book about improving legacy codebases (testing is just a major technique), so it focuses on using testing when it's most useful.
(Also, maybe it's just me, but a book that starts from, "Help! I just inherited a C++ project with ten years of rot and half the comments are in Norwegian! How do I even start adding tests to this without breaking it further?" seems a bit more practical than one that starts by applying unit tests to example code designed for convenient testing. It's seldom that easy.)
It definitely helps to start with an environment where tests are easy to write, so that you can focus on the principles of testing without having to learn a lot of mechanics.
Python's built-in "unittest" module is an example of that. While it's not perfect, you can quickly form good habits by using it.
And it is not limited to Python. For instance, I've written unittest-compatible classes that basically run other programs as test cases. Either way, you're forced to think about things like "how do I automatically detect that this failed?" and "is the purpose of this test clear?".
I personally just sort of figured it out by doing BDD on a few small projects to get a feel for how things go. I'm not 100% "driven" now, but overall, it's pretty high. I tend to do Cucumber tests more than straight up unit testing, as I find them more valuble personally...
Doing is the best way to learn most things in the software world.
I think it depends a lot on context. There are some kinds of code where unit testing is just about the only way to test it at all and makes a huge amount of sense. There are other situations where unit testing verges on being a complete waste of time. One of those is that which the author describes, where you are writing what is essentially glue code between several large, complex components. Your code is simple while the components are complex. Mocking the complex components in order to test the simple code usually ends up having very low value to effort ratio. Instead, a functional or integration test that verifies that all the components work together correctly is simpler to make and more useful in terms of the results it produces.
i've got to heavily agree with you on this. i feel more in control of what's going on in my code... and yeah, often confused by peoples massive opposition to it.
generally speaking if someone's angry about something, they probably don't understand it. otherwise... why not ignore it til it goes away?
I simply can't imagine developing without unit tests now. I very rarely (never) get things spot on the first time and the unit tests help me maintain pride and position before most of my big merges.
Sometimes I do wonder if I rely on them too much, but they aren't a waste of time.
I do agree with the point that it is possible to spend a lot of time fixing "broken" tests. The lesson that should often be learned is that certain tests are not adding enough value to warrant the constant maintenance. (Code alone requires plenty of maintenance as it is.)
One thing that works extremely well is to have a test environment. In other words, spend the time to replicate your entire production flow 100% (but in isolation), so that you can "deploy" changes and see if anything blows up. This not only catches stupid deployment bugs, but it also allows you to avoid some unit testing by simply relying on customers themselves: have them log into the isolated environment and do exactly what they would have done in production, and you'll know if it works.
One of the really great things about software development is that reality, rather than a committee, gets the final word. I think that unit testing is a great idea, in theory and in practice, but I do think that the unit testing may get to the point where some practitioners who are not careful will spend far longer to complete a project with no discernible improvement in stability or maintainability than someone who uses unit testing more judiciously. I have used the end-to-end automated functional test approach before, and it did enable me to find problems pretty quickly. However, I would have died a slow and painful mental death without unit tests in some of the tricker sections of my code.
That said, yeah, "unit testing is a waste of time" is link-bait-ish.
More seriously, the point of the article, from a skim, appears to be that "writing too many unit tests is a waste of time". Well, duh. Save 15 minutes of your life and pass on this one, methinks.