
Mockito 2.1.0 - raphw
https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2
======
ThomasRooney
I spent my undergraduate thesis on automating the production of mock objects
and unit tests using program tracing via the JDK's debug APIs [0]. I'd thought
it might be useful to have a tool which, requiring no user input, would output
when expected contracts of behaviour (in the context of two objects
interactions with each other at production) had changed. I wrote my tool to
generate JUnit/Mockito tests. It was almost bulletproof in producing working
tests for arbitrary Java bytecode (see the link for examples).

At the end of it I was utterly disillusioned that the vast majority of tests
with mocks were actually useful to an end developer. It felt too fragile and
made refactoring work much more of a chore. Since then I've tested via
constructing independent slices of implementation and pushing sample
input/output in. In my experience, it has being vastly easier to maintain such
a test suite.

I've meant to go back and open source my tool for a while, but found other
parts of life getting in the way. If anyone's interested, would love to chat
about it (might help me find motivation to continue the work).

[0] [http://www.doc.ic.ac.uk/teaching/distinguished-
projects/2015...](http://www.doc.ic.ac.uk/teaching/distinguished-
projects/2015/t.rooney.pdf)

~~~
larsthorup
I have recently ventured into auto-mocking, see
[http://zealake.com/2015/01/05/unit-test-your-service-
integra...](http://zealake.com/2015/01/05/unit-test-your-service-integration-
layer/), which allows me to run super fast unit-tests, that are really doing
end-to-end testing, giving me 100's of end-to-end tests per second. Is that
the same line of thought as your "independent slices of implementation and
pushing sample input/output in"?

~~~
ThomasRooney
That's a really interesting idea. It looks like you're capturing and proxying
the latest backend request/response patterns used in its test suite into a
frontend test requirement so that you can run them both independently, and
thus much faster, whilst still keeping most of the assurances of an end-to-end
test. I'm going to have to think about applying that approach to my own
projects.

What I actually meant is slightly more literal. The backend of my latest web
project is 100% web client invocable functions in AWS Lambda. The frontend is
a static site built in redux with sagas. The view is a dumb layer mapping
state to html and dispatching redux actions which are picked up by the sagas.
Each saga only invokes one lambda function, though does send/receive signals
with other sagas. Responses break down into state changes which map to a
single react view.

By taking this path all implementation can be visualised as discrete slices of
logic. Moving signals `up` and `down` the slice requires request/response
patterns between the components. To test, I run a script against the developer
staging area which sends API calls to each function and diffs the result with
expected response back. It is fragile to any change which affects API
responses, but is dumb enough that it gives me confidence to flip the switch
turning staging environment into production. Its super easy to maintain and
update, though I suspect it'd be difficult to apply the approach to more
complex application domains (this is a virtual telephone number service).

------
brianwawok
And in most cases you will regret the web of mocks you have made :)

~~~
uw_rob
Do you have any examples of where mocking has let you down? I found creating
unit tests with mocks and DI extremely easy and satisfying. At least for me,
they also provide a pretty convincing argument that the tested code is
correct.

Also, they are basically required if you want to do any testing with external
services. I'm not sure how else you could create unit tests for DBs etc.

~~~
brianwawok
Yes. Every project I have ever used with a web of mocks, especially around
final things made refactoring near impossible. At some point the codebase got
so brittle, it fell apart.

I find it funny that in most discussions around mocking, a database is the
given example. 100% of the time when I need a database for "unit tests", I use
a database. Not a mock. I don't want to know that when I do select blar from
flar I get back the value 7, and hardcode it. I want to actually load the data
and see it happen. Because if I don't, how do I know that my code will
actually work in production?

Sometimes that involves sqllite (easy but not preferred because I don't use it
in production). Sometimes it just involves running a local database that gets
built up for a unit test run. Sometimes something like docker, sometimes not.
Sometimes a shared "dev" database. Lots of options. But literally every
project I have been involved with that used a REAL database instead of mocking
it out, has worked better and been more stable and easier to change.

Now for actual external services.. say you call some Netflix API. Yes don't
call that for real in a unit test. But you can do that with either:

a) Basic mocking, because you created a very simple API. You don't just call
some random driver directly with your code, right? You have an interface in
front of it. Then you can mock it with no final or static override crap going
on.

b) If your API is clean enough, you can just code up a very simple dummy
implementation. Lets you set the return value for methods, then the next call
it gives them back.

I don't feel that strongly between A and B above. But I feel very very very
strongly about people mocking databases or, even worse, every random call
between classes in their own codebase, or someone elses codebase.

~~~
ng12
> I find it funny that in most discussions around mocking, a database is the
> given example. 100% of the time when I need a database for "unit tests", I
> use a database. Not a mock. I don't want to know that when I do select blar
> from flar I get back the value 7, and hardcode it. I want to actually load
> the data and see it happen. Because if I don't, how do I know that my code
> will actually work in production?

That is not a unit test. What if your database is corrupted? What if your mock
Netflix API is buggy? The second you do anything like that you're no longer
unit testing. The whole point is you're testing a single method/class in
isolation -- Mockito gives you the power to assume external structures (the
database) work as expected.

If you don't like unit testing, sure. But unit testing is what Mockito is all
about.

~~~
brianwawok
What if my DB is corrupted - Why did it get corrupted? Sounds like you have a
good failure to chase down. It would be even worse if your prod DB got
corrupt!

For most use cases, a given piece needs 1 test around it. "The test". If you
can test your logic + view + db all in 1 test, you get a pretty huge
efficiency gain.. over sitting there writing "unit tests" where you mock what
a select * from a database returns. Life is too short to do that.

~~~
uw_rob
Having just a single set of tests has a lot of draw backs compared to
splitting up your tests.

Consider this, we have a simple CRUD app that uses a datastore.

Since you want everyone to be able to run the test suite locally, setting up a
local datastore manually isn't an option. You need a script/function that will
setup the database.

Because you want your tests to be as deterministic as possible, and you don't
want the order that tests are executed in to change the results, you need to
setup the database before every test, and then clean it up again after the
test.

However, setting up the datastore and cleaning it up again can take
significant amount of time. Even if it only takes 5 seconds, when you multiply
it out for each test your test suite can now take 100x longer to run.

This means you have now limited yourself to being able to run tests after you
have completed large changes, and not incrementally.

Also when a test breaks, you now need to determine if it is broken because
your DB isn't configured correctly, or because something is wrong with your
code.

You can help mitigate these problems by creating 2 sets of tests. Unit tests
and integration test. Unit tests use mocks, and run quickly. Integration tests
will create a environment that mimics what you have in prod, and will take a
while to complete.

I think you also overestimate how long it takes to write mock tests.

Hopefully this provides some insight into why I think unit + integration tests
are the way to go.

~~~
brianwawok
> Since you want everyone to be able to run the test suite locally, setting up
> a local datastore manually isn't an option. You need a script/function that
> will setup the database.

Since we want our environment to be reproducible and stored in git, this is a
requirement anyway. If your schema is not versioned in Git, and cannot
automatically be applied in all environments, including prod, you are doing it
wrong.

> Because you want your tests to be as deterministic as possible, and you
> don't want the order that tests are executed in to change the results, you
> need to setup the database before every test, and then clean it up again
> after the test.

No not really. Lots of ways to solve this. For example DJango will run a test
in a transaction, ensure it passes, then roll back the transaction. Order thus
does not matter, the DB is cleaned up for free.

> However, setting up the datastore and cleaning it up again can take
> significant amount of time. Even if it only takes 5 seconds, when you
> multiply it out for each test your test suite can now take 100x longer to
> run. This means you have now limited yourself to being able to run tests
> after you have completed large changes, and not incrementally.

So the old wisdom on this is that unit tests should be able to run in 5
minutes or less. Just a silly Django crud app with a parallel test runner, you
can do something like 50k-100k tests in 5 minutes (Most tests take about
1/10th of a second, with several going on in parallel). Past the 50-100k test
mark it gets harder to do locally, but is trivial to do in CI (for example,
there are several CI as a service services that split your tests between
containers, so you can run any number of parallel containers).

> Also when a test breaks, you now need to determine if it is broken because
> your DB isn't configured correctly, or because something is wrong with your
> code.

Because database state and schema is stored in git, and applied
programmatically, we already ruled out the DB. So if a test breaks, it is the
code.

> You can help mitigate these problems by creating 2 sets of tests. Unit tests
> and integration test. Unit tests use mocks, and run quickly. Integration
> tests will create a environment that mimics what you have in prod, and will
> take a while to complete.

To me, you had a problem (Database not consistent), and rather solve it in a
sane way (store schema in source control, apply automatically) - You solved it
in a very roundabout way (worry about which class of a tests a given test is
in, write two types of tests, still fight bad schema in prod issues because
you didn't solve the root problem (schema not in source control) )

------
carrja99
After journeying through many languages since my long time with Java that
ended in 2011 I have to say I have yet found a mock object framework in other
languages that match the expressiveness that Mockito did. Python's mock has
the least memorable API I have ever worked with. :-(

~~~
dom0
This is because using Mocks in a language like Python is a bit awkward and
unnatural in itself... if you look at the way tests are usually written, there
is just little need for it. That's also the reason why only recently
unittest.mock was added in the first place.

~~~
samcal
Can you elaborate on this? What is the correct way to test external services
that may have side effects in Python?

~~~
bebop
I think maybe because there is no type checking in Python. This makes it more
trivial to have objects that quack like a duck, without the need to explicitly
create a class that conforms to an interface.

~~~
LgWoodenBadger
FWIW, you don't do that with Mockito either, in Java land.

Mockito will happily use the existing type definition of your dependency,
provided that the class declaration is not final (i.e. can't be extended).

~~~
raphw
This is actually not a requirement anymore. Mockito 2 can mock final types and
methods.

------
jaypaulynice
There's been a lot of discussions on Mockists vs Classicists. See Martin
Fowler's article:
[http://martinfowler.com/articles/mocksArentStubs.html](http://martinfowler.com/articles/mocksArentStubs.html).
In my experience, projects that use mocks extensively usually have the most
bugs. If you also have integration tests then mocks are ok. But then it's more
code to maintain.

Given a choice I would always do "integration test". Fact is you can test a
single class/method, reality is that no class/method exists alone.

~~~
lightbendover
We recently implemented an extensive test suite that can be run with either
mocks for service dependencies or in a live configuration, depending on config
settings. Before, we simply had integration tests that relied on _very_
specific data across disparate services and running it in CI was a nightmare
(to the point that we would typically only run the full suite in a QA
environment shortly before a production deploy).

Whereas our integration testing has been responsible for catching hundreds of
obscure bugs over the past year, most of the bugs found by our mocks have been
bugs in the mocks themselves and I expect that issue to compound due to the
brittle nature of mocks in general. Mocks make every change some multiple more
expensive in both initial cost and maintenance.

------
compay
This project is the standard case I use when telling people not to choose
project names in a language you do not speak. It means "booger" in Spanish.

~~~
euyyn
A little one, though.

------
geraldtrue
Thank you Rafael, Tim, Marcin #1 & Marcin #2! Great team and great project!

