
Slow Tests Are the Symptom, Not the Cause - orend
http://re-factor.com/blog/2013/09/27/slow-tests-are-the-symptom-not-the-cause/
======
lmm
In my experience when you do this you end up with anaemic tests that are
basically useless. As the page comes close to admitting, all you do is write
the same code twice. The result is a lot of wasted effort on unit tests that
never fail.

In an app designed like this the integration tests are the useful ones,
because they're more contentful and test assumptions that are more likely to
be wrong. The article flatly asserts that code written in this extremely
decoupled style will require fewer integration tests, but I see no evidence
for this. And you still need to figure out how to make them run fast.

~~~
orend
The _unit_ test for the service object might be anemic, but it's far from
useless IMO. The unit test helps you design the object under test and it is
written first if you are practicing TDD.

The argument for having fewer integration tests goes like this: if it's hard
to write isolated unit tests for some classes you end up testing many more
execution paths in integrations tests.

~~~
lmm
> The _unit_ test for the service object might be anemic, but it's far from
> useless IMO. The unit test helps you design the object under test and it is
> written first if you are practicing TDD.

But it doesn't help when your test is solving the same problem. If your test
says "this object calls collaborator1 and collaborator2, then passes both
results to collaborator3" then that's exactly the same design problem you'd
have if you were to just write the object straight-up.

This is why people talk about BDD rather than simply TDD. It's not about
having tests, it's about identifying the user-facing behaviour that you want,
testing that, and then writing your objects to conform to it. This really does
help your design, because it lets you start with the appropriate interface on
the user side and work down from there, and the resulting object boundaries
can be better than the ones you'd have thought of if you started with design.
But if you restrict your test to a single class and mock its collaborators
then that implies you've already decided what your class boundaries are; in
effect you've already done the most important part of the design.

>The argument for having fewer integration tests goes like this: if it's hard
to write isolated unit tests for some classes you end up testing many more
execution paths in integrations tests.

True enough - if there are things you can't test in a unit test then you need
to test them in an integration test. So if you have a complex piece of logic
that you want to run several tests of, it's best if you can test that piece of
logic in isolation - this is exactly what unit tests are good at.

But you still need to test all your integration pathways. All too often a
class doesn't contain any logic as such, particularly when you design the way
the article suggests - it's just an integration between its collaborators. For
such a class there's no point in unit testing, because the only thing the
class actually does is integration.

------
programminggeek
Pluggable systems (loose coupling) are the easiest way to create a system that
is cheap and easy to maintain. I've found that a lot of Rails developers in
particular don't see the benefits of this and they really, really like tightly
coupled systems because of the convenience of it all.

Benefits like fast, easy testing, easier maintenance, swappable external
things like databases, queues, external api's are apparently not enough of a
benefit to change away from rails.

I wrote a pattern
([http://obvious.retromocha.com](http://obvious.retromocha.com)) that makes
pluggable systems easy based off of ideas from Uncle Bob, Alistair Cockburn,
and Kent Beck which I think makes writing pluggable systems a lot easier, but
as with all similar structures like Hexagonal Architecture and the like, Rails
devs seem to not see the value in it until their app is a complete mess, at
which point they would rather add more rails code than fix the underlying
problems.

For a lot of developers, now that rails is popular, rails IS the architecture
and things that go off the rails are either a waste of energy or too
difficult.

Rails is what Kent Beck calls a connected architecture and as his incredibly
scientific graph describes, the cost of change spikes dramatically over time
compared to a modular architecture:
[http://www.threeriversinstitute.org/blog/wp-
content/uploads/...](http://www.threeriversinstitute.org/blog/wp-
content/uploads/2009/08/connectedAndModularDesign.jpg)

Until developers adopt modular or at least more service oriented designs,
nobody should be surprised by slow tests, features taking too long to ship,
the cost to ship code to rise dramatically over time. It's mostly our own
fault.

~~~
lkrubner
Where I work, we just had a long internal debate about the usefulness of
monolithic frameworks such as Rails and Symfony. I argued against the
frameworks, but much of the team is committed to using them. The convenience
of having everything in one place apparently is important to a lot of people,
despite all the problems that are caused by that. I posted the debate here:

[http://www.smashcompany.com/technology/an-architecture-of-
sm...](http://www.smashcompany.com/technology/an-architecture-of-small-apps)

~~~
steveklabnik
It's not actually Rails that's the problem, often: it's a cultural issue. You
can build a Rails app that has a very thin 'rails' layer and has a super well
decoupled domain model layer.

~~~
huntedsnark
Absolutely, the Rails app I'm working on right now is just like this. Still
feels like Rails but the complex logic and interactions can run without it.

~~~
lucaspiller
I would be interested to more details on how your architect things - we've
been in the process of moving the business logic of our app into service
objects as 2000 line models have become a bit unmanageable :D

------
bradhe
Was mostly with the author right up until he shows an example of the worst
possible unit test. Testing the implementation and not the intent.

This feels like how Spring must have been born. Trying to think of an
alternative, and I think I would be happy with the fat controller example
originally given. It's simple enough and straight forward.

Perhaps it's just bad examples all the way around...

~~~
lectrick
I think this was only a poor example because the method under test was a
"director" style method (not sure what the name of it would actually be) that
simply forwards a bunch of messages/commands to other objects and doesn't do
much actual logic/computation.

Tests for these "director" style methods must by nature assert that all the
right messages were forwarded to the right objects under the right
circumstances, so yes, you do end up with tests that look a lot like what
they're testing.

Agreed that ideally you are only testing ins and outs though, but I think this
case was exceptional.

~~~
pkolaczk
It is better to not have a test at all than to have a test that looks a lot
like what it is testing. I never uderstood that crazy push for 100% unit test
coverage that some are advocating. Unit tests are just one tool among many.
Use with care and where appropriate.

------
schmrz
I don't know (and don't care) if this makes your unit tests faster, you
shouldn't write code like that. He took something that should be simple and
made it really complicated. The execution time of your tests shouldn't dictate
how you will design and write your application. Especially if it makes things
harder to maintain and understand.

Edit: I love tests and they should be fast so that you can run them every time
before pushing your changes but as I said they shouldn't dictate how you
design and write your application.

~~~
orend
The point here is not that the tests execution time should dictate the design.
On the contrary, the idea of this post was to stress the importance of good
design, and that fast tests are just a side effect of it. Now, you might
disagree with me on what constitutes good design but that's a different issue.

~~~
schmrz
Don't get me wrong, I agree with the idea that you should look at the root
cause of why your tests are slow instead of trying to "fix" the tests. I just
feel that what you demonstrated is bad design. So you ended up with bad design
and fast tests.

------
VeejayRampay
Isn't part of the problem the fact that Ruby is ridiculously slow? I'm not
even trolling, I am a Ruby programmer, I've used the language for about 5
years now and seeing friends of mine run their 200+ files test suite in under
half a second (C programmers), I can't help but think about how nice that must
be to be able to run your tests when you want to without thinking of the
consequences (and yes I understand comparing C and Ruby is ludicrous, just
pointing to the fact that Ruby is still extremely slow and would very much
benefit from a good speed bump eventually).

Cause slow tests as a symptom pretty much means that if you're not an
excellent programmer then tough sh*t, testing will be slow and painful, which
ultimately drives people away from those good practices.

Also, I find ironic that Rails (as a framework that is supposed to empower
people and make them efficient) is, according to many comments in this thread
and "tech pundits" conducive to the kind of tight coupling and dependency that
will lead to bad design choices and slow tests.

~~~
pan69
>> and seeing friends of mine run their 200+ files test suite in under half a
second (C programmers)

But they need to spend half hour compiling before they can run their tests.

[tongue in cheek obviously]

------
huntedsnark
The slowest part of most tests suites are the integration and acceptance
tests. Regardless of how decoupled my actual domain logic is, these will
usually be the bottleneck, so I can't really agree with slow tests always
being a symptom of some underlying architecture problem.

~~~
regularfry
Yes, but you don't run those all the time. It's the unit tests that have to be
fast, and if they aren't, that's a problem.

------
garethadams
Not a fan of the way the link jumps to the Disqus comments rather than just
being at the top of the page

~~~
orend
sorry about that. Can't seem to edit this now thuogh.

------
yxhuvud
Personally I would have

1: Added a users resource beneath the mailing list, ie

    
    
      resources :users
      resources :mailing_lists do 
        resources :users
      end
    

add_foo is almost always a sign that a new nested controller should be made,
and there is almost always a need for a delete as well - in this case users
should be able to remove themselves from a mailing list.

2: Put the add method on the mailing list and not on the user, because that is
where I'd expect the least logic to be.

because:

3: I'd put the actual mailing list mailer logic in a separate class for the
configuration of it.

------
matlock
We've published two blogposts how we sped up our test and boot times
considerably. Maybe helpful to some:

[http://blog.codeship.io/2013/08/21/faster-test-suite-boot-
ti...](http://blog.codeship.io/2013/08/21/faster-test-suite-boot-times-with-
ruby-on-rails.html)

[http://blog.codeship.io/2012/11/15/speeding-up-our-test-
suit...](http://blog.codeship.io/2012/11/15/speeding-up-our-test-suite.html)

------
gingerlime
after reading sandi metz POODR[1], I tend to prefer the approach on the post
too. However, I also must say that the time it takes to execute unit tests,
even with factories, db access etc is still an order of magnitude _faster_
than running things like feature tests with capybara. This is by far the
biggest drain on test time (and resources), and these are also the tests most
difficult to write and maintain. Way too often, when we decide to change some
functionality, there's a whole battle with tweaking capybara tests to work
again.

Perhaps I'm showing my lack of knowledge or experience with capybara, or
integration testing, but that's one aspect I wish there were some better tools
/ approaches. Having to dig through css/xpath elements to click on or perform
interaction is still a major PITA. If anybody has some suggestions to share,
I'd love to learn more!

[1][http://www.poodr.com/](http://www.poodr.com/)

~~~
regularfry
I think the key here is not to try to aim for anything like full coverage with
capybara, just enough to be effective. Martin Fowler has some words on this
here:
[http://martinfowler.com/bliki/TestPyramid.html](http://martinfowler.com/bliki/TestPyramid.html)

~~~
gingerlime
Thanks for the link. We're pretty much using this pyramid approach
intuitively, but just as a single data point - _one_ feature test takes around
15% of the execution of our entire test suite (with around 850 tests). This
one test is quite comprehensive and touches the core functionality, but
nevertheless suffers from all the negative aspects Martin Fowler mentions.

Were we to improve our unit tests to run even faster, as the post suggests,
the difference in execution time will become much more pronounced. several
orders of magnitude.

I think feature/UI testing is still an interesting space for better tools, but
of course the problem is very difficult to solve.

------
mguterl
It's worth stating that this is the MailingListsController and there is no
mention of a MailingList. It seems to me like some functionality in the
"before" example could live in the MailingList class.

