
Zen and the Art of Unit Testing - Liriel
http://marcin-chwedczuk.github.io/zen-and-the-art-of-unit-testing
======
royjacobs
I see this type of mock-heavy testing in lots of places. I used to do the same
thing too, but I found that it basically just tightly couples your test to
your implementation details. The whole purpose of unit tests is to be able to
refactor things with confidence that your tests will still pass. If you so
closely tie your test to your implementation you lose that.

In the example, the tests now really care about how data is being retrieved
from the database. Instead, I'd use an in-memory database (or even a
Dockerized one). That would then also test that the right queries are being
done, but you could still refactor the internals of UserServiceImpl (terrible
name btw) and your tests wouldn't fail.

~~~
lubonay
I tend to do mock-heavy testing in Java projects, and it has burned me several
times. You write mock tests along your architectural boundaries, but if those
need to change due to a deeper refactoring, all your tests need to get
refactored as well.

Mock testing essentially tests expected side effects. A more powerful concept
is the use of pure functions wherever possible, so that your tests compare
input/output pairs, instead of long, difficult to maintain and sometimes non-
exhaustive lists of expected side effects.

Does anybody know how one can replace Services, Controllers, Presenters and
other such imperative mediator objects with a more functional approach? I'm
just speculating, but that should make test maintenance easier.

~~~
adamkl
Take a look at Gary Bernhardt's talk "Boundaries"[1]. It touches on this very
topic and was quite the eye-opener for me (having experienced all the same
issues you've described).

[1][https://www.destroyallsoftware.com/talks/boundaries](https://www.destroyallsoftware.com/talks/boundaries)

~~~
lubonay
I think this may hit the nail on the head, gonna check it out later. Thanks!

------
sidlls
I must be one of the few who think this sort of unit-testing is necessary to a
certain extent, but in practice seems rather to lend itself to causing
terrible code smells all over the place: things like small one-liner functions
for conditionals and basic arithmetic that are designed with mock- and test-
ability in mind (rather than whether or not they make the code maintainable or
correct), factoring code specifically to increase "code coverage" of the
tests, etc.

I think integration and other coarse-grained tests are far more useful more
often, and using asserts (which will cause these coarse-grained tests to fail,
too) is a better pattern than "unit test all the things".

~~~
JamesBarney
I've heard these change to design tradeoffs that occur to increase
testability, test induced design damage _.

_ Enterprise craftsman I believe

~~~
sidlls
This sort of rigid adherence to a particular paradigm isn't restricted to
enterprise developers. I've seen this sort of thing in practice (or attempted)
from colleagues in data pipelines for machine learning and other very much
not-enterprise settings.

~~~
adrianratnapala
I find such religious devotion rules common among scientists who have to code
even though they are not programmers by trade. Religious overcommenting etc.
Whatever rule they happened to read in their FORTRAN-for-beginners book.

ML people are probably more knowledgeable, but their job is also often more
like science than traditional software engineering.

~~~
sidlls
That doesn't match my experience (as a former academic scientist who moved
into software development). The governing rule for us was always "publish or
perish." Programming was a means (exploring a problem) to an end (advancing
knowledge and publishing the results). I, at least, never came across the
jargon for testing, maintenance, etc. until I moved out of academia and into
this industry.

~~~
adrianratnapala
Oh I agree, scientists are far less inundated by software "best practices"
than professionals. It's just that those which they have heard of, they often
stick to religiously.

Personally I prefer the results of these scientists to much of what I see from
real software engineers. It might be wreck, but it is only 1000 lines of wreck
-- and not a sophisticated wreck created with powerful modern tools.

------
stuartd
Misleading title - there is absolutely no Zen in this article. The nearest it
gets is a quote from Confucius, who wasn't a Buddhist.

~~~
ChefDenominator
I wish people would stop using that title. They seem to think it is a clever
click-grabber, which is probably true for most people, but for those even
remotely familiar with the book, _Zen and the Art of Motorcycle Maintenance:
An Inquiry into Values_ , opening up the article and finding no application of
Pirsig's Metaphysics of Quality is disappointing.

~~~
abiox
there's plenty of such things... "X considered harmful" seemed to have an
uptick in usage for a while.

~~~
alberto_ol
also "what we talk about whwn we talk about...", "the unreasonable
effectiveness of ..."

------
latch
I've been writing tests for a long time, and I went through a phase where I
depended heavily on DI. While it's still useful in some conditions, I rarely
make use of it.

First, DI is simply the least subtle for of IoC available. When all else
fails, you can always rely on DI. But, this is something languages should be
looking to help developers with. Elixir, for example, supports IoC as part of
the language, and is much more elegant than DI [1]. When monkey patching is
available, this is also a possible solution (easy to abuse, yes, but suitable
in simple cases). Functions as first class values can also help (instead of
passing a clock around, you can have a SystemClock.Time func that you can
override in your tests, which is similar to what Date.now() is in javascript).

But, perhaps more importantly, and as much as I tried to deny it, integration
tests are absolutely and totally worth the trouble. If you unit test two parts
independently, you run the very real risk of having them pass under test, but
fail once deployed. I've seen more production downtime caused by incorrect
assumptions between services than anything else.

Also, lately, I've been writing more and more fuzz tests. I'm probably not
very good at it yet, but I think for the couple of projects we did it, it's
been a worthwhile effort (moreso when we started...they barely catch anything
now since we're all coding much more defensively).

[1] [http://openmymind.net/Dependency-Injection-In-
Elixir/](http://openmymind.net/Dependency-Injection-In-Elixir/)

~~~
UK-AL
Nobody denies integration tests are good. You just don't want your unit tests
to be integration tests.

Unit tests are meant to be fast, and quick to isolate the failing code.

~~~
snovv_crash
If you make your so-called-'unit' tests too closely coupled to you don't
actually work faster though, because you constantly have to refactor them to
match the internal implementation.

Your testing feedback might be quicker, but quickly getting useless
information (ie. your implementation changed) is just as bad as slower tests,
because you need to fix the test, recompile and again before you get useful
information.

Also not all 'integration tests' are slow. If you can test using SQlite in
memory, instead of pointing at a PG instance that needs to be provisioned, for
instance, you can get a lot of your end-to-end testing running really quickly.

~~~
bradmwalker
Try CREATE TABLE UNLOGGED with postgres. The diversity of RDBMS and SQL
dialects defeats the point of integration testing the persistence layer with
SQLite.

~~~
snovv_crash
Yes, but a large portion of code that needs testing is simple inserting,
queries and joins, and SQLite can do this without any problems.

You can push complex or db-specific work to slower test groups, but not
everything requires it.

~~~
abiox
even this is less clear - for some dbs it's normal to use a sequence for
rowid's (vs some sort of autoincrement) and i don't think sqlite offers all
the same behavior there.

------
mybrid
I found the art of testing is more tied to the reality on the ground as
opposed to the testing methodology.

1\. Do you have a a QA department doing manual or even automated testing? 2\.
Do you have performance testing? 3\. Is the code base large or small? 4\.
Transactions versus analytics? 5\. Social media? Some code doesn't need
testing. Facebook doesn't test their code.

Each new project requires a flexibility to understand the reality of time and
resources available for testing, the type of application and performance
considerations. If one works at a bank or on transactional software then
testing needs to verify exactness. If you are working on analytical software
then what's another anomaly between friends, errors in your errors, who cares.
If you are working on social media then just pump and dump like NodeJS.

That is the art of testing to me. A mind set of one size fits all with respect
to the of value unit testing, integration testing, regression testing or
manual testing is the opposite of art: it's ideology.

------
tener
> Without DI you are forced to write slow and difficult to maintain
> integration tests instead of unit tests.

Integration tests are worthwhile even if you do have unit tests. They serve
different purposes.

~~~
UK-AL
Yes, but you keep them separate from unit tests. Unit tests should be fast to
run, for quick feedback.

------
jrpelkonen
> String newPasswordHash = cryptoService.sha1(newPassword);

Ouch. It's 2017. I know it's a blog about unit testing, but using SHA-1, let
alone unsalted SHA-1, for password hashing even if for illustration purposes
is dangerous.

~~~
mdaniel
At the risk of spitting into the wind, so is using String in Java for
passwords; the actual password-centric interfaces in the JVM use char[]
because one can zero-out those, but cannot zero-out immutable Strings

------
macca321
It's a shame that the functional requirements, which were neatly stated in the
spec, weren't preserved in the test code.

Rather than "arrange", "act", "assert", I've taken to writing the Gherkin spec
inline with the test code, e.g.

    
    
      @Test
      public void onlyExistingEmailsCanResetTheirPasswords() {
       //Given an email address which is not in the database 
          String email =  "unknown@example.com";
          when(userRepository.findByEmailAddress("unknown@example.com"))
    	   .thenReturn(null);
          
       //When a password reset is requested for that email
       UserServiceImpl userService = ...
       userService.startResetPasswordProcess("unknown@example.com");
    
       //Then no password reset notification should be sent
       verify(notificationService, never()).sendResetPasswordNotification(any());
       //And the user should be given a success message so they can't determine if the email address is already present
       verify(userInterface, once()).sendMessage("You have been sent Please check your email.");
      }
    

This way you preserve the intent for the programmers that will come after you,
but you don't need a complicated Gherkin runner

------
mybrid
Testing cost money.

I find it very unusual to talk with developers who regularly modify their
testing practice based on time and money.

The most expensive is to do it all: unit testing, integration testing,
component testing, system testing, end-to-end testing, stochastic testing,
performance testing.

The art of testing in no small part comes down to the art of money. How much
bang-for-the buck does any testing provide given limited resources? I answer
that question.

------
mybrid
Understand the workload.

Take the workload difference between in house versus public usage.

If one releases software into the public domain then one has no reasonable
expectation as to the kinds of data driving your code. For this kind of
general purpose, heavily reused software then code coverage with, unit testing
and mocking makes the most sense is my experience.

However, one can also write code for a very specific, limited usage. In those
cases I find regression testing with regression data works best. They are not
paying me to find bugs to satisfy code coverage.

I find understanding the workload is a significant factor in determining the
kinds of testing I use.

------
_pmf_
> startResetPasswordProcess_givenEmailOfExistingUser_sendsNotificationToUser

It's sad that this is the state of the art when unit testing (but I have not
found any viable alternative when the premise is that I have to use JUnit).

