
A Comparison of Assertion Libraries - nfrankel
https://blog.frankel.ch/comparison-assertion-libraries/
======
zaphar
I have a possibly unpopular opinion that assertion libraries that depend on
throwing exceptions are bad API design for unit testing. Mainly because most
test functions end up with multiple assertions in them and exceptions abort
the test function early.

Many times understanding what causes a unit test failure is easier to debug if
you can see the rest of the unit tests all at once.

I much prefer testing frameworks that use simple true/false tests and not
exceptions like Go's unit testing library or Perl's TAP library.

~~~
barrkel
In my experience test failures occur vastly less frequently than the rate at
which tests are written, and when they do fail, it's usually due to a
refactoring where lots of tests fail with only a tiny handful of failure
modes.

If test setup is involved, adding multiple assertions is much cheaper (both in
code and test suite execution time) than having a test per assertion. If
assertion failures don't fail fast, later assertion failures are frequently
just noise. For example, often the size of a list or map is asserted on, and
then the entries checked; if the size fails, there may be spurious null
reference or index out of bounds failures.

And due to far fewer failure modes than broken tests, it's often the case that
once one test has been fixed, a similar fix needs to be applied to most of the
other tests that failed. In a dynamically typed language, this is usually due
to a refactoring that changed the API in some way; a method renamed,
parameters added or removed, that kind of thing.

(I am increasingly convinced of my long-held opinion that unit tests are
mostly useful at the leaves of the call graph, and the pinch points in data
flow (which are usually fairly integration flavoured), and not very valuable
everywhere else. Integration tests that seldom need to be rewritten after a
major refactoring are worth their weight in gold, though. Major bonus points
of they can be run quickly.)

~~~
zaphar
That is usually true, and it's a lot easier to find all of the failures first
if you can see all of them at once instead of fixing one rerunning the test
suite fixing another, and continuing the cycle until they all pass.

