
Why I don't mock - mpweiher
http://blog.metaobject.com/2014/05/why-i-don-mock.html
======
ChrisGaudreau
Mocking is the primary problem with TDD as it exists right now. "Isolated
testing" (or isolated TDD) is advocated by people like Uncle Bob. Many
developers listen to Uncle Bob and end up believing that unit testing without
complete isolation is not "real" unit testing.

Isolated unit testing is supposed to improve your designs. But when you mock
everything, you end up with tests that know everything about your
implementation (white box testing). The result is that refactoring breaks your
tests. Your tests no longer document how to use the system; they are merely a
mirror of its implementation.

Kent Beck's central point in his book is that TDD mitigates fear, allows
refactoring, and gives you immediate feedback. Extensive mocking makes
developers fearful to refactor (thus hurting design) and reduces the quality
of feedback. Not to mention it makes your tests hard to refactor too.

Martin Fowler explains the difference between "classical" and "mockist"
testing:

[http://martinfowler.com/articles/mocksArentStubs.html#Classi...](http://martinfowler.com/articles/mocksArentStubs.html#ClassicalAndMockistTesting)

~~~
CMCDragonkai
I spent more time building my mocks instead of building my tests. Sometimes a
file might have 90% mocks and 10% tests. Or maybe that's just my mocking
framework...

~~~
glenjamin
The idea is that if you end up with code like this, it shows you have too many
collaborators and should refactor them away.

~~~
oelmekki
Exactly.

An other reason could be it's a main entry point for code execution, a class
that only purpose is the glue others.

In that case, having a lot of mocks is a code smell pointing that you should
probably test this on the integration level rather than on the unit one (and
thus, discard mocks and use "real" everything).

(but beware not to be too much self indulgent on what is a gluing class, most
of the time, they are not and it's indeed a god class problem).

------
emsy
I found the article disappointing. Instead of giving reasonable arguments
against mocking, the author offers an example where mocking was used wrong and
therefore propagates that it shouldn't be used at all.

The problem is not mocking. I found it to be absolutely helpful if used right
(and right often means sparsely). The problem is this "all or nothing"
mentality that many programmers have. Either they mock the hell out of the
codebase or they don't use it at all.

I don't use mock objects, unless it's message based business logic, often
located at the topmost layer of abstraction.

------
mpdehaan2
So I do a lot of systems programming type development.

In these cases, whether you're PXE booting a server or talking to 500
different APIs that all range between the latest database or cloud API,
flashing firmware on a RAID controller (shudder), whatever, there's a point of
diminishing returns. There's also most of the bugs that you won't be able to
simulate, because they come from external factors.

The magic lies in automated integration tests.

QA automation is _AMAZING_ stuff. Set up real servers and real test
environments when you can. Test real database upgrades.

Mock testing can have it's place, but often you find yourself making internal
API contracts more rigid than you want, which can limit refactoring.

~~~
humanrebar
How do you test your system's response to error conditions? Do you have hooks
in your integrated system to force errors (throw exceptions, violate
contracts, etc.)?

~~~
mpdehaan2
In some cases, yes.

You can do things like set up scenarios you know won't work, such as
connecting to machines that don't exist or sending the wrong parameters.

The main thing is if the external inputs can't be made to "violate a
contract", the internals aren't the problem.

Ultimately, it depends on what you are building. Simple CRUD web applications
are different from systems development type applications in many cases. Many
of these classes _can_ be more easily mocked out.

Often the breaking change you'll see is an upstream API or SaaS endpoint
changing in ways you didn't expect (or the difference between a Linux utility
on the latest Ubuntu, an older RHEL, and Solaris), so it's often better to
concentrate on testing the real thing.

And these tests are expensive, so an automated test matrix that deploys a VM
fleet with lots of permutations is usually necessary.

------
InclinedPlane
A big danger with mocks is just shoving the problems under the rug rather than
fixing them. Mocks represent assumptions, typically untested assumptions.
There is generally nothing enforcing a mock's behaviors to be in accordance
with whatever it is mocking, so that leaves it as a free parameter which can
easily be incorrect if someone makes an error (which obviously can and does
happen).

~~~
tomstuart
This isn’t a danger, it’s a benefit: your tests of one unit (whatever you
consider a “unit” to be) are isolated from the behaviour of the rest of the
system. For example, if you break one part of your implementation, you get one
failing unit test, not a thousand.

Checking these assumptions is what integration and acceptance tests are for.

------
trustfundbaby
As always theres a middle ground between the extremes of MOCK-ALL-TEH-THINGS
or mock nothing!!

Since reading Sandi Metz's very fantastic "Practical Object-oriented design in
ruby" it clear to me that mocking is great for checking that objects other the
one under test receive messages sent by the methods being tested (makes tests
easier to setup and write usually). If I find that I'm doing too much mocking,
then its usually because the method I'm using is doing way too much. FWIW.

------
giulianob
I really want to see people's complex projects and example code. There have
been way too much opinion on this topic lately without any concrete evidence
to show the strengths/weaknesses of using or not using mocks.

~~~
pling
I can only give you an anecdote as our product is closed source. Mocks went
like this, in a bell curve fashion:

1\. Mocks are great, we'll use them everywhere.

2\. Mocks are preventing our tests from being refactored.

3\. Mocks are preventing our code form being refactored.

4\. Mocks are preventing progress on the platform.

5\. Code is being modified and tests and mocks are being removed to keep
productivity up.

6\. There are no mocks and few tests.

7\. Tests increase again as defects are found. No mocks required.

This is over the space of 8 years and our problem was the amount of coupling
they caused.

~~~
emsy
Mocks don't cause coupling. Programmers do.

~~~
revscat
Cute, but even as terse as this comment is its sentiment is incorrect.

Mocks, almost by their very nature, encourage coupling. Ask yourself the
question "What tends to get mocked? " Answer: Objects which are difficult
and/or heavy to instantiate, but which are outside of your direct control and
cannot be changed, e.g.: database connections, network sockets, etc.

So when you want to write tests around this you are almost forced to mock them
out. So you instantiate those classes (as mocks) in your test class, so that
the code under test. There are now two references to that class: one in the
test, and one in the code being tested.

This is coupling. QED.

~~~
tomstuart
> Ask yourself the question "What tends to get mocked?"

Answer: every collaborator of the object under test, so that only the tested
object is actually having its implementation exercised. All other objects that
it needs to talk to — database connections, network sockets, or whatever — are
replaced with mocks, and the test subject’s interactions with those mocks are
verified, so that a) we have confidence that the tested object is interacting
with its collaborators in the expected way, and b) the test itself doesn’t
depend at all on the actual behaviour of those collaborators’ implementations,
which behaviour should be explicitly exercised elsewhere.

That’s the opposite of coupling. I don’t understand where your answer comes
from or what it means.

~~~
mpdehaan2
I view this as definitely enforcing coupling.

The issue is that you should want to make it very easy to change the API
signature of internal components.

Mock objects can tend to reinforce internal coding choices when things that
are not a public interface, or a service boundary, are mocked out.

If you're mocking at this level, there is a potential for refactoring of
intra-class API contracts to become much harder to change.

Thus, it would be better IMHO to mock out only meaningful service boundaries,
and concentrate testing at the public contracts.

You can still get very strong coverage, but you're just thinking about a
different level of inputs.

------
meric
I don't use mocks much, except with regard to the "now" method. There's some
code that relies on the current time, I can use mock to mock many values of
'current time' instead of being forced to only test the case where the current
time is when I run the unit test. This is especially important when dealing
with code involving time zone and day light savings.

~~~
mpweiher
My experience with time is that whenever you build a system that is time-
dependent in any meaningful way, you want to have "synthetic time" that you
control, which you only feed with real time in a single, controllable place.

Otherwise you get nasty effects, such as time moving on while your system is
processing etc.

~~~
meric
I'm guessing you mean like:

    
    
        def has_expired(now=None):
            #.. implementation
    

instead of:

    
    
        def has_expired():
            #.. implementation
    

But at some point you need to call that method somewhere else...

    
    
        def get_request_handler(request):
            #...code
            now = timezone.now()
            if has_expired(now=now):
               raise Error403.
    

And `get_request_handler` still needs to be tested, which I 'stub' the API for
getting the current time in the correct timezone within the test for the
request. (Taking into account the other comment), using a library called
'mock'.
([https://pypi.python.org/pypi/mock](https://pypi.python.org/pypi/mock))

------
programminggeek
I think OOP is exposing serious flaws in how humans write software and solve
problems. I'm not sure, but it seems there is an impedance mismatch in how we
write code, and how we'd like to test code, but there is a reluctance or
outright defiance toward the idea of writing testable code.

The FP community doesn't seem to have a problem so much with testing simply
because they mostly write pure functions that are data in -> data out and that
is the easiest thing in the world to test.

It occurs to me that most developers aren't writing testable OO code, and thus
we have a never-ending argument about the most clever way to test (or not
test) code that is not easily tested in isolation.

If we never get around to writing testable code, we aren't going to have
systems that are easy to test. If we don't have easy to test systems, testing
is going to suck and people won't do it.

I don't know that FP is the answer, but just asking people to write better OO
systems isn't working. The problem certainly is more fundamental than simply
TDD or mocking or whatever language or tool we are using.

The problem is in how we look at solving problems in code and how that relates
to system maintenance over time. It's a people problem, not a tooling one.

~~~
keeperofdakeys
I don't think there is enough of an FP community doing large, heavily tested
systems to allow accurate generalisations like that. I can think of plenty "We
used FP, and everything went better than expected!" articles, but you probably
don't hear from the people who tried, but came across problems.

~~~
programminggeek
I agree, and I don't know that FP is the solution.

I just know that the OO design approach isn't working super well with the
people who are using it, and we now have a couple decades where design hasn't
progressed significantly on the average project.

So, how to we change the human behavior that is leading to less than ideal
outcomes?

------
stinos
While I think I get where the OP is going, this seems just too easy to claim
without any real-life code. So let's try this then. The last time I used
mocks: we're interfacing with motor control hardware.

In code we have interfaces for axis configuration, axis movement and so on.
All implemented for different controllers. On top of that there's a layer
defining specific movements. Say you want to go from point a to b while
avoiding c, there's a piece of code that calculates a route (so it needs axes
configurations for max velocity/range/...) and another piece that applies this
route to a set of axes (so it needs axes to drive).

Apart from dragging around a bunch of motors everywhere I code, how am I
supposed to properly test those two pieces without any mocks? Telling me the
design is all wrong to start with is fine, but then you also have to come up
with a better one of course :). Telling me that route calculation shouldn't
depend directly on axis configuration but should use it's own configuration
type and I should then add a conversion between the two is also no solution as
it a) introduces an extra layer and b) still requires a mock to test the
conversion, d'uh :)

------
mattdeboard
There's a time and place for various tools. Mocking isn't all good, it's not
all bad. Sometimes it's a good tool for accomplishing an immediate goal.
Sometimes it's a crutch that encourages poor design. Keep an open mind and
make evidence-based decisions. Only Siths deal in absolutes.

~~~
crashedsnow
"Do or do not. There is no try". Absolutely.

------
ytechie
I don't understand how you can be 100% against mocking. When I want to
visualize testing, I often find it useful to think about how it's applied in
the real world. For example, when building a car, the parts are tested in
isolation (for the most part). Individual parts are tested - size, shape,
stress, etc. Groups of parts are also tested in isolation. For example, the
engine is actually connected to a mock drivetrain, mock computer, mock user
inputs, etc. I see that as useful, and I think the same goes for software. It
can simply be too costly in terms of time, bandwidth, etc to test against real
resources.

------
pswenson
Good points... but what happens when you walk into a legacy code base with few
tests that wasn't designed to be tested and is tightly coupled to a component
such as a database?

------
webhat
I dislike the "that's not supposed to throw an exception" developer statement,
if something is not supposed to throw an exception under normal operating you
can almost guarantee that it will throw exceptions in the demo. So one of the
few cases where I use lots of mock is in Exception Handling.

Some exceptions are almost impossible to generate consistently in tests, or
can't be simulated easily next to test that are supposed to pass when that
exception state doesn't occur - think API failures, databases being yanked
mid-query, etc.

------
nope_42
I had a random idea the other day to create a program that monitors/logs
system level calls and lets you replay them for a program. This would
effectively let you use your program as normal, log any actual pieces that use
external resources, and then automatically use the saved results as part of
your test framework by overriding those system calls at the appropriate time.

Is anyone aware of a solution like this? I certainly don't have time to create
it but I would definitely find it useful.

~~~
AlexanderDhoore
We have something like this at the company I work for. We have a "simulation
layer" which wraps ALL third party code. This lets us simulate the failure of
file system calls, network drivers... And we use it to write our tests.

------
SideburnsOfDoom
TFA says "For example, when building SportStats v2 at the BBC we thought we
needed a database for persistence. But we didn't build it in until we needed
it, and we didn't mock it out either. We waited until the code told us that we
now needed a database. It never did"

So they seem to be saying "if you write code upfront before you need it, you
might not need it, therefor mocking is bad"

IMHO, that is not a valid conclusion to draw.

~~~
the_af
I agree with you, that seems to be what the author is saying, and his
conclusion doesn't follow. Also, I'm not sure if he really means mocking or
any kind of test doubles.

I thought everyone agreed that writing code upfront before you need it is
always a bad idea, _especially_ specific implementations like a database.

------
camus2
So, a lot of ideas are thrown here without any real use cases.People say
"mocks are bad,whitebox testing is bad".But how can I unit test anything
without mocks?

let's take javascript.I cant generate mocks from my code,i have to write
them.I have this data access object that takes a connnection.

    
    
        var dao=new Dao(connection);
        dao.getById(1,function(err,res){ console.log(res) });
    

Tell me how do I unit test getById in isolation without mocking
connection,without doing whitebox testing,without knowing what actually
happens in my getById.

the obvious thing here is to spy on a hypothetic connection.query, fake a db
call and return a predictable result.I cant do anything without a mock.

------
lmm
Mocks shouldn't be the first resort; they're there as a way to allow you to
test code that you can't improve right now. I wouldn't want to use them on new
code, but as a stepping-stone for testing and improving a large existing
codebase they have their place.

~~~
tomstuart
That’s not what they’re for! Mocks allow you to design the interface of a new
object by implementing that object’s collaborators before you implement the
object itself. This is a very high-fidelity source of information about what
the interface should look like; anything else is just (educated?) guessing.

~~~
mattdeboard
That may not be what they're "for" (wtf does that even mean?) but they're a
damn good use of mocks. You can't fix everything right now.

~~~
tomstuart
> That may not be what they're "for" (wtf does that even mean?)

It means that “[mocks are] a way to allow you to test code that you can't
improve right now” and “[mocks are] a stepping-stone for testing and improving
a large existing codebase” is a mischaracterisation of the role and benefits
of mocks.

If using them like that is useful for you then that’s great, but it’s
misleading to say that that’s why “they’re there”.

Steve Freeman & Nat Pryce’s book [0], for example, is an excellent
illustration of what mocks are for.

[0] [http://www.growing-object-oriented-software.com/](http://www.growing-
object-oriented-software.com/)

------
john2x
I'm new to mocking (to serious unit testing, really), is mocking functions of
3rd-party libraries and/or web API's recommended? It sounds like a good idea,
especially for functions that need to hit the network for a web API, but maybe
I'm missing something?

~~~
michaelmior
I would say it's generally a bad idea to mock 3rd party libraries. It's code
that's not under your control and testing with a mock which may not match the
actual behaviour could cause you trouble. In the case of network requests,
something like VCR[1] helps a lot.

[1] [https://github.com/vcr/vcr](https://github.com/vcr/vcr)

~~~
SideburnsOfDoom
I would disagree. If you want to unit-test your code that depends on those 3rd
party libraries, then your choices are to:

1) Use the real thing. This can be slower and a lot less reliable.

2) Capture some typical output of the 3rd party api, and make a mock that
emits it.

Which technique is useful? Both are. But the second one gives you fast-running
fine-grained repeatable unit tests so therefore is your first line of testing.

So you discover in an integration test (#1) that the live api has a behaviour
that the mock doesn't, maybe an occasional malformed response due to an error
at the other end. Great, capture it and add it to the mock. Now you can
reproduce it at will and write fast tests on how you handle it.

~~~
derefr
3) Define a Gateway interface (e.g. DataStoreGateway) that presents exactly
the API you want to consume; write one implementation of it that speaks to
your third-party API (e.g. ODBCDataStoreGateway), and another that doesn't do
much (e.g. MemoryDataStoreGateway).

Note that a trivial gateway-implementation, like MemoryDataStoreGateway, isn't
a mock! It's a fully-functional component that works perfectly well from its
consumers' perspectives. But, unlike the version that consumes a third-party
component, it doesn't have nearly any failure modes that would confuse your
tests. It just does what it does, simply, and lets your tests test what you're
trying to test.

And note that when you stop mocking, and limit yourself to switching out
fully-fledged gateway implementations, "dependency injection" stops being this
huge pain with Factories and heavily-parameterized initializers. Instead, you
can just make your objects discover their collaborators through a service
registry. Your unit tests just register the trivial implementations in the
service registry on setup().

\---

† Don't get a bad taste in your mouth thinking about Spring here. A simpler,
healthier example is Erlang's "global" module
([http://erlang.org/doc/man/global.html](http://erlang.org/doc/man/global.html)).

~~~
SideburnsOfDoom
> But, unlike the version that consumes a third-party component, it doesn't
> have nearly any failure modes that would confuse your tests.

That's all fine until you need to test handling the third-party component's
failure modes.

> Note that a trivial gateway-implementation, like MemoryDataStoreGateway,
> isn't a mock!

Probably not. Though it depends at which point you insert the "trivial"
components. If you're using a trivial component to unit test a collaborating
class that isn't at the edges of the system, then that's a mock and it's
helpful. It's not the _only_ helpful technique or even the usual one, but
"mocks are never helpful, always avoid them" is silly thing to say.

A mock store for one unit test is far more lightweight than a
MemoryDataStoreGateway, and allows an easy way to simulate server errors, so
sometimes it is a better thing to code.

> This is what "dependency injection" is really about, actually: not inserting
> mock-objects ... but rather allowing you to plug in ... versions of your
> collaborators.

It can be about both. They are not exclusive.

> Instead, you can just make your objects discover their collaborators through
> a service registry

You may be happy with every object reaching out to a global singleton. But I'm
going to stay well away. Thanks but no thanks.

~~~
derefr
When you're testing the third-party component's failure modes, the tests
you're writing are _functional tests_ belonging to the nontrivial gateway
implementation (e.g. ODBCDataStoreGateway).

The important thing about the gateway _interface_ is that it's an _error
boundary_ : you don't need to think about "server errors" when dealing with a
DataStoreGateway; DataStoreGateways don't _have_ server errors. The ODBC
object the ODBCDataStoreGateway holds might hand it a server error, but it
won't _propagate_ that back to you. It'll just put out a plain†
DataStoreGateway::InsertError or somesuch.

† The exception should probably hold a copy of the implementation-exception
that triggered it, but this is just for the sake of debugging the
implementation. The client should never unwrap the internal exception.

~~~
SideburnsOfDoom
> you don't need to think about "server errors" when dealing with a
> DataStoreGateway; DataStoreGateways don't have server errors.

I'm not sure why you even make that distinction. More than once I've dealt
with servers that occasionally time out and fail; (say once a week or so). I
wish to test what I show on my UI under those conditions, and what part of the
text and status codes returned from the remote server that _represent_ (or
whatever) its failure I will choose to display.

Doing so without using a mock seems pointlessly obtuse and roundabout when I
can make a mock to insert whatever error response I want at any interesting
point in my code.

~~~
derefr
Split that problem into two problems:

1\. Your collaborator cannot be reached.

2\. Your client has a network connectivity problem.

I've never seen a client who needs to know #1. All they care about is that
there was a DataStoreGateway::TransientInsertError, and that
TransientInsertErrors mean retrying and notifying the UI that their work
hasn't been committed yet.

#2 isn't a problem the DataStoreGateway should be responsible for, because it
will affect almost everything in the system in one way or another, and you
don't want to have to write that code repeatedly. It's more likely a kind of
Alarm--the kind of thing that gets pushed out from wherever it is to a global
alarm handler, which then does something like putting the UI into a different
root view-controller state, e.g. darkening it and covering it with a modal
saying "lost connection."

~~~
SideburnsOfDoom
Actually no, it was not either of those cases, it was what I said. The APIS
that I'm working with aren't that much like a DataStoreGateway.

Your design points may be valid, but they don't really talk about mocks; why
bend over backwards to avoid testing using a mock?

------
krazydad
Was hoping to read an admonition against excessive snark and bullying in
Hacker News comments. Ah well.

------
clubhi
Mocking helps with feature isolation. If a test failed you can be sure it was
because of the use case of the test.

Mocking helps when you are calling into code that you can't control. E.g, when
calling into an API that is broken and sometimes returns incorrect results.

I don't write very many tests anymore. I haven't mocked an object in years. I
also have a lot of bugs in my code.

~~~
mpweiher
_Mocking helps with feature isolation._

By enabling isolation in the test, mocking reduces the pressure to design more
isolated code.

 _Mocking helps when you are calling into code that you can 't control._

Pervasive mocking helps isolate you from the pressure to remove (deep)
dependencies on code that you can't control. Redesign your code so it only
interacts with such code at the edges. At those places, create APIs that are
independent of the code you can't control, and adapt that code to fit your
APIs.

~~~
pswenson
"Redesign your code so it only interacts with such code at the edges."

yeah, and what if you don't have time to do this? sometimes mocks are
appropriate because of schedules and other priorities. Much better to mock
than to not write tests because mocking is not pure.

~~~
eropple
Quit? Before it makes you quit?

If your schedules and other priorities are against fixing broken code, I'd be
leery of your organization's long-term health. If you're in for the long haul
--and at my current gig I certainly am--then eliminating bad code (and any
code as tightly coupled as what you describe) is the only sane option. Because
it'll kill you eventually.

