
Working Effectively with Unit Tests - GarethX
http://blog.fogcreek.com/working-effectively-with-unit-tests-interview-with-jay-fields/
======
colinramsay
I constantly have problems with people posting the same information on unit
tests, again and again. I know how to set up (for example) RSpec, MiniTest,
Cucumber, FactoryGirl. The thing that people struggle with is how to use these
in a larger, practical sense, rather than in just the trivial. I think this
puts a lot of people off testing.

~~~
Tyguy7
Agreed. I have only used TDD once as a front-end web developer, and I've found
that I do struggle with the practicality of it. I spent half my time managing
and maintaining my tests. When I did the math, It seemed that all that extra
time spent was more than It would have taken to simply chase down bugs that
would have arisen in a non-tdd environment. It just seems to take more time
than is practical. Especially in agile startups.

~~~
atilaneves
Perhaps. The difference is that in one case you'll have the machine
automatically tell you when and why your code stops working because of a
regression, and in the other you have no clue until the customer hits it. It's
not only about proving it worked once, it's about proving it with every
commit.

------
struppi
A nice article, and I think I should add the book to my (already too large)
to-read pile.

I totally agree with "DRY might not be great in all cases" \- In tests, it's
often better to make each single test as expressive as possible. I once read
(and forgot who said it):

"Code should be either DRY (don't repeat yourself) or WET (write expressive
tests), but never damp)"

And here's a shameless plug: I recently wrote an article about how tests can
help you achieve simple design, and the next article in the series will be
coming soon:
[http://quickglance.at/en/simple_design/passes_its_tests](http://quickglance.at/en/simple_design/passes_its_tests)

~~~
zak_mc_kracken
> "Code should be either DRY (don't repeat yourself) or WET (write expressive
> tests), but never damp)

I'm getting so tired of people who try to capture the intricacies and
complexities of our profession with witty one liners that have to be taken
like dogma.

Just use your common sense, let your experience guide you and if you're not
sure about something, ask your teammates.

~~~
struppi
Who says that they have to be taken like dogma? Sometimes the one-liners are
just easy to remember. Often they are funny. They can be a great tool to
quickly illustrate a point or start a conversation.

The one liners can never replace a conversation or explanation. They cannot
replace common sense or real knowlege. I would never treat them as laws or
follow them by the letter...

------
collyw
This is one of the most sensible articles I have seen on unit testing. Rather
than religiously insist on tests first or 100% coverage, judge when it is
valuable to have these things and when they are not important. Far too many
articles on testing test the framework / language being used.

------
henrik_w
I like Jay's comments about how tests are wasted if you don't know exactly
what you want, and just try out different things (because then you need to
change/delete the tests when you end up changing your code).

As for common mistakes with unit tests, my top 5 list is:

1\. Testing algorithms together with coordinators.

2\. Mocking too much.

3\. Not using asserts.

4\. Leaving print statements in the tests.

5\. Checking the log statements, not the result.

More details here: 5 Unit Testing Mistakes
[http://henrikwarne.com/2014/02/19/5-unit-testing-
mistakes/](http://henrikwarne.com/2014/02/19/5-unit-testing-mistakes/)

~~~
Iftheshoefits
Here is another one: failing to understand that tests introduce additional
maintenance and design overhead.

Quis custodiet ipsos custodes?

Tests are additional code that is tightly coupled to the target code that is
meant to be used in production. This additional code requires design,
debugging, and maintenance just like the target code. It is also tightly
coupled to the target codebase: a change in either necessitates at least an
inspection to determine what, if any, changes must then be made in the other.

This all adds overhead to the whole process, and too often I see casual, glib
statements implying or even asserting that the overhead is nonexistent or else
pays for itself. It isn't free and it is especially not free the more dogmatic
the test proponents and tests themselves are.

~~~
woah
Your complaints are actually the whole reason tests exist. Of course you know
that your code works when you write it. You then write tests that are tightly
coupled to the behavior that you require from the code to run it through its
paces as you work on it.

Speaking of coupling, tests prevent tight coupling between different pieces of
your codebase. Since there are then at least 2 consumers of any piece of code,
the implementation does not get coupled to a particular one as easily.

Also, if you would like to work with a module or microservice based approach,
tests are simply necessary to work. If you're storing different pieces of the
codebase in different repos, you will need to have tests to work on them at
all.

Basically, if you are "testing" your production code with other pieces of your
production code (I hope you don't do this), it will become tightly coupled.
Test your code with tests.

~~~
Iftheshoefits
I'm not sure we disagree about the purpose and utility of tests. I certainly
was not complaining about tests. If anything, I was complaining that too
often, in my experience at least, it seems to be presented as an undeniable
truism that, generically, 'tests' are always a benefit, and usually the more
tests the better.

My point is that tests (at least, unit/integration tests of the variety
usually mentioned in TDD) are more code that has to be maintained (and
designed, and debugged). They incur overhead and as such impinge on
development resources. As a result tests introduce the need for considering
tradeoffs between developing tests and developing the software product itself.
Simply writing tests for the sake of having tests to test code is naive; at
least as naive as assuming that once working a given piece of code works
forever, but it is a mistake that I've witnessed on more than one occasion.

------
abuehrle
I write lots of unit tests. One thing I'm struggling with now is the
organization of a large number of tests. It's hard to know what I have tests
for without going into my tests directory and reading comments. Even then, I
feel like I could do more to have a "big picture" overview of my testing
strategy rather than seeing a bunch of one-off tests. I haven't quite figured
out what that should look like.

I think the IDE could do more in some cases. For example, when unit testing a
method, I'd like to be able to tag a reference to the method I'm testing.
Then, when I'm in my code, my IDE could show me the docstrings of tests I have
for that method.

It's the difference between "hello world" and software design principles used
to build large scale systems. I see lots of "hello world" for testing, but not
much of the other side. Does anyone have suggestions?

~~~
ereyes01
The way I personally solve this problem is by organizing my code and my tests
in a similar structure. Split application code up into reasonable files, and
split the tests up the same way. Depending on your language/test framework, it
also helps to put the test right next to the code... i.e. in Go, you can have
component.go and component_test.go in the same directory.

I do agree though that it would be nice to see an IDE have some smarts about
tests...

------
zak_mc_kracken
Terrible audio. Next time you interview someone, ask them to install Audacity
and to run it during the interview. Then have them send you the audio file via
Google Drive.

------
dkrich
I have to say that the trade-off argument brought up between DRY and
readability is pretty lame. If you have a junior developer who can't make
sense of the test because you put the test data in a separate class or are
using a setup method, then teach him or her how to trace the call to the setup
method and then read through it.

If we follow this logic then all code should follow the same methodology of
readability because tracing through method calls is too cumbersome. Obviously
that isn't the case for most code, so why would it be true for unit tests?

~~~
woah
Part of the benefit of tests is that they are a form of documentation.

~~~
dkrich
I don't think having readable tests and DRY are necessarily mutually-
exclusive. If you name methods and variables with lengthy but very descriptive
names your tests can still be readable. In fact the more you modularize your
setup methods the more readable they can become. For example:

TestData.CreatePerson();

TestData.AddSsnToPerson();

TestData.PersonModule.checkWhetherPersonHasValidSsn();

...is pretty readable despite being broken up. It may or may not be true that
if you just went through the setup here it may be more readable, but the
benefit to this approach is that you only have to write a method to add an SSN
to a person object one time and then everyone can use it without needing to
understand the structure of different objects. Notwithstanding readability, I
would argue that DRY makes _writing_ tests easier than cramming a bunch of
setters into every test case. Once somebody understands what a setup method
does, in every place that it is used in other tests the tester can skip over
that part and move on to testing their specific use case.

------
encoderer
Much of what passes for testing is a near-criminal waste of corporate
resources. I've seen so much rubbish test code.

~~~
jakub_g
Tests are a great thing when everyone involved understands their value. In my
former team we had a big number of unit & integration tests that saved my ass
many times.

On the other hand, when the team has a KPI "we need a big test coverage!" and
people are lazy or simply bad developers, you end up with a code like

    
    
      try {
        runTheWholeApp()
      } catch (e) {}
      assertTrue(itRan)

~~~
ZeroGravitas
Mutation Testing can test your tests in these situations.

Well actually if the organization is that dysfunctional it's probably game
over, but Mutation Testing can catch you when you do this accidentally.

------
chatmasta
Does anyone have any resources for best practices unit testing web apps
written with frameworks like Django, Rails, etc? Whenever I try to apply TDD
to a framework, I find myself testing the functionality of the framework
(which should already be tested), rather than my app itself. How do you decide
what to test when you are effectively cobbling together framework concepts? Is
this the time for focus on integration tests over unit tests?

------
vowelless
I wish the audio quality was a little better. Is there any post process work
that can be done to improve the quality?

~~~
GarethX
Sadly that's with post-processing improvements. Without the guests recording
their own audio and then syncing it after (which is something of a big ask),
we're reliant on the quality of the connection. Something to work on for sure.
I'm hopeful solutions like [http://podclear.com](http://podclear.com) can
help.

------
platz
For tests we should use DAMP, Descriptive And Meaningful Phrases, instead of
full DRY

------
Juraci
Rant about "unity". The idea of unity is vague. Almost every person has a
different definition of unity. Do not think about unity think about
granularity, or even better facets of behavior. The amount of integration can
scale to infinity. You can have infinity layers of integration between unity
and end2end tests. The attempt to classify granularity is a waste of time.

------
lukasm
You may want to invest a few bucks in a good mic.

~~~
GarethX
Yeah, sorry about that. Derrick has one, but was moving house at the time of
recording and it was packed away. So future ones should be using that, at
least on the host side.

