Hacker News new | past | comments | ask | show | jobs | submit login

The answer depends a lot on your terminology.

I would say that unit tests should always be as repeatable as possible, which means no randomization. But I would ALSO say that you should prefer property tests over unit tests whenever it's feasible to do so, and fuzzing with random input is one of the defining features of property tests.

Others consider property tests to be a special class of unit test, which, given that pretty much all tests these days are written and run using a framework with "unit" in the name, is also reasonable terminology.

As for why to use property tests: In a nutshell, because achieving sufficient coverage with unit tests alone is infeasible. One of the unstated assumptions of a primarily unit test oriented strategy is that devs are so on their game that they can just reliably think up all the boundary cases, all on their own, in order to achieve good test coverage. I'd argue that assuming everyone is just that good isn't too far off from assuming everyone is so good that you really don't need tests at all.

To the irreproducibility argument: Any property test framework will hunt for a minimum failing input and then print it out in the test failure report, so that you can then use to construct a unit test. From there, it's up to the developer to be disciplined, just like it always is. And some particularly clever ones, like Python's Hypothesis, will automatically (locally) remember failing cases and run them first.






> I'd argue that assuming everyone is just that good isn't too far off from assuming everyone is so good that you really don't need tests at all.

I'd argue unit tests aren't made to stop developers from pushing bad code, they're made to prevent future developers from breaking it even further.


I'd agree. But unit tests can still only guard the behaviors that someone thought to cover with a test. Property testing is still your friend here, either all by itself or as a way to find all the cases to memorialize with unit tests.



Applications are open for YC Summer 2020

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: