
Why Most Unit Testing Is Waste (2014) [pdf] - dumindunuwan
https://wikileaks.org/ciav7p1/cms/files/Why-Most-Unit-Testing-is-Waste.pdf
======
blatherard
This paper has been public for years at [http://rbcs-us.com/documents/Why-
Most-Unit-Testing-is-Waste....](http://rbcs-us.com/documents/Why-Most-Unit-
Testing-is-Waste.pdf)

~~~
rhizome
Previous discussion 6mos ago:
[https://news.ycombinator.com/item?id=12666454](https://news.ycombinator.com/item?id=12666454)

~~~
WalterGR
There have been a ton. For more, click the "past" link.

------
chillacy
> If you want to reduce your test mass, the number one thing you should do is
> look at the tests that have never failed in a year and consider throwing
> them away

And that's how you get regressions. I get the rationale that these tests cover
code which probably never changes, and tests are most useful for code that
changes a lot. But if the guy who wrote the code left the company a year ago
and then you have to change this code... it's going to be a lot easier if
there are good tests.

~~~
jernfrost
He did cover that. He said specifically that regressions tests are a good
thing, but that they are typically not done at the unit level.

~~~
sv123
He means that's how you get regressions of the same bugs popping up in the
future, not regression tests.

------
kevinmhickey
I'm a bit skeptical of a paper written by a consulting company that
specializes in QA services and training. Understanding a codebase well enough
to write useful unit tests is very time consuming and difficult. Teaching devs
the same requires domain, development and process skills. Other forms of
testing (e.g. end-to-end, browser based, and manual) are much easier for an
outside organization to apply based on rules in a BRD or user manual. It
stands to reason that a company would write a white paper that nudges
customers to a methodology that favors them.

~~~
jernfrost
He has some points though. Unit testing has become a bit of a religion, where
people almost write tests for the hell of it regardless of their value. Now I
do usually test e.g. algorithms, but I find it rather wasteful to test dumb
classes at the unit level as well as UI stuff.

I've certainly felt what he pointed out about tests becoming a real hassle to
maintain while not necessarily being very valuable to keep.

------
gillianlish
the programming people i admire most just randomly seem to be the same people
who never write angry rants about programming topics.

took me 20 years to notice this.

------
xwvvvvwx
_When I was programming on a daily basis, I did make code for testability
purposes but I hardly wrote any unit tests. However I was renowned for my code
quality and my nearly bug free software. I like to investigate WHY did this
work for me?_

This made me giggle. It's like magic. don't write tests, don't find bugs....

~~~
mempko
Actually he provides much better alternatives. Design by Contract combined
with system level tests will catch more bugs than you can shake an editor at.

------
majewsky
Link is down for me. Can someone give a tl;dr?

~~~
jaccarmac
The TL;DR: I did a quick read, no evaluation here, just what's in the paper.

1\. Unit tests only work in "functional" code composed of easy-to-reason-about
components that support other similar components. The OO storm introduced a
style of code which is impossible to inspect unless it is running.

2\. Because Turing machines you need as many bits of test code as bits of real
code.

3\. Code coverage requirements can only lead to tests for their own sake, i.e.
wasted lines of code.

4\. If there are many more lines of tests than production code, your
developers are quite possibly paranoid about what they're writing, and that's
bad.

5\. Tests that have not failed in a long time don't convey information, they
only provide noise. Optimize your test suite to provide information.

6\. The reward of seeing a green bar in an IDE encourages appeasing the test
gods instead of designing a good system.

7\. Assertions are better than tests.

8\. The Agile culture has turned unit tests into something worse than what
they could be without it.

~~~
mikekchar
This is such an awesome summary, I'm just going to start here without actually
reading the paper :-). I'll say up front that I'm a TDD/Unit test kind of guy.
I'm also not generally a fan of London School/Outside-In style (but will
obviously use it when circumstances demand it). So I'm probably exactly the
kind of person that should disagree with the above. But I don't really
disagree that much.

One thing I've noticed is that there really is more than one way to skin a
cat. Some people like to have a more linear style in their code. I've found
that such people tend to have good memories and considerable ability at
reasoning. They can see a fairly large piece of code and see quickly where the
errors are. They can remember that one piece of code on the other side of the
project did something one way and code on this side of the project does it
another way.

These people will often look at my code and it looks disjointed to them. They
can't read the code and get a sense of what it is doing. They find themselves
jumping around through hundreds of files just to figure out what's going on.
If I have any ability at all it's really much more in the spatial relations
side. I can visualise networks easily and reason about them.

I can never remember the details about how things are implemented and I'm a
very slow reader (I'm dyslexic). I depend on trust. I want to be able to see a
function and tell in 5 seconds whether or not it is correct. I then rely on
tests to tell me when I have violated some assumptions somewhere -- because I
don't bother to learn and have no capacity for remembering all the assumptions
in the system.

Unit tests and the intense factoring that occurs when doing TDD (at least in a
non-mocking style) favours a style where you admit imperfect knowledge about
the system. It then gives you a framework for reasoning about the system.
People (even on my team) are often surprised that I haven't used a debugger
for at least 10-15 years. Tests are really a misnomer -- they don't verify the
correct execution of the code. What they do is probe the execution of the
code. It tells me what the code is actually doing and documents my assumptions
about what it should be doing. If there is an error, there will be a mismatch
in the two. I may or may not have an explicit test that fails during that
mismatch, but I should be able to reason about it effectively.

It's a different way to think and a different way to work. IMHO, I think it
requires considerably less mental capacity to work in a TDD way. However, on
the other side of the coin it requires considerably more training/experience
to do well. You definitely want to decide upfront which way your team wants to
operate, but I'm not about to say that you're "wrong" if you pick a different
approach than I would.

Anyway, thanks for doing the summary! It was very evocative for me :-)

~~~
FeloniousHam
> These people will often look at my code and it looks disjointed to them.
> They can't read the code and get a sense of what it is doing. They find
> themselves jumping around through hundreds of files just to figure out
> what's going on.

I'd add that unit tests (above the method level) help to "tell the story" of
the code. Coming into a large code base with well-factored code can be
daunting, it's a boon to be able to step through unit tests that exercise even
just typical scenarios.

------
astrodust
There's a lot of out-dated junk in this latest dump, so do take all of this
with a gigantic grain of salt.

~~~
jjawssd
> out-dated junk

You just accurately described half of the Intelligence world accurately

