
Falsifiability and Unit Testing - codingismycraft
http://codingismycraft.com/index.php/2016/05/25/falsifiability-and-unit-testing/
======
dalke
I disagree with the given metaphor of TDD as scientifically falsifiable. TDD
posits the answers first, and builds a theory around it. As new answers/unit
tests are put into place, the theory/program becomes more complicated.

Given enough parameters (or complications), it's possible to fit any set of
known data points.

What's missing, to be falsifiable, is to pose a set of questions/tests which
you expect to succeed if the theory/software is working. These might include
new edge cases, scalability testing, and performance testing that fit better
outside of TDD, which focuses on fast, short tests, and especially outside of
red-green-refactor, since these validations tests are supposed to be green-
green-green and never red.

I, for example, have a set of tests which require loading >4GB of data, to
test that I didn't use a 32 bit offset in places where I should have had 64
bit offset. These tests take minutes to run, and are implemented as system
tests through the command-line API, not unit tests.

> Following this way of delivering software, TDD is one of the practices that
> is usually overlooked, as inexperienced and non technical PMs, like to
> believe that writing comprehensive tests is very time consuming and it does
> not really add lots of business value in the final solution!

To address that point more directly, "testing" is much more than TDD. I
believe after-the-fact tests are more akin to tests for falsifiability than
TDD is.

The ultimate test for many programs is, do users use it? (Or "do people buy
it?") That certain is one way to judge the validity of the program, with a way
which is outside the TDD framework.

~~~
codingismycraft
>> I disagree with the given metaphor of TDD as scientifically falsifiable.
TDD >> posits the answers first, and builds a theory around it. As new
answers/unit >> tests are put into place, the theory/program becomes more
complicated.

The software development process starts with the definition of the problem and
its desired answer. The code is a finite state box that claims that it meets
the expectation of receiving the problem and return the answer while the
purpose of testing is to verify this claim.

>> To address that point more directly, "testing" is much more than TDD. I >>
believe after-the-fact tests are more akin to tests for falsifiability than >>
TDD is.

I agree. Still, these after-the-fact tests belong in the infinite universe of
tests that they were not identified originaly and can now become part of the
testing suite. What distincts a good from a bad tester, is his ability to
minimize the need for these "after-the-fact tests".

~~~
dalke
> The software development process starts with the definition of the problem
> and its desired answer.

That's precisely where the metaphor breaks down. For a physical theory, we
don't have a desired answer.

The desired answer that motion in the universe is made of circles does not
somehow turn ellipses into a set of epicycles.

