
Fuzzing vs. Property Testing - chwolfe
https://www.tedinski.com/2018/12/11/fuzzing-and-property-testing.html
======
kcsrk
Of interest is Crowbar[0,1], a testing tool for OCaml that combines Quickcheck
like property testing with AFL fuzz.

[0] [https://github.com/stedolan/crowbar](https://github.com/stedolan/crowbar)

[1] [https://ocaml.org/meetings/ocaml/2017/extended-
abstract__201...](https://ocaml.org/meetings/ocaml/2017/extended-
abstract__2017__stephen-dolan_mindy-preston__testing-with-crowbar.pdf)

------
stevekemp
One of the things that I like about go is that the standard-library really
encourages the use of writing test-cases. Most of the bigger projects have
good coverage, and I try hard to get my own.

But nothing beats the sheer tenacity of running your code through a fuzz-
tester. I've written a simple virtual machine, which interprets bytecode, and
a BASIC interpreter amongst other things recently. Both of these projects
benefited hugely from fuzz-testing, despite having high coverage via manually-
written test-cases.

Fuzz testing is cheap and largely automatic, so if it takes a few hours or a
few days to find an interesting result that's not a problem. I remember the
first time I tested my interpreter when I had ~90% coverage of the code with
my test-cases and it crashed via fuzzing within seconds! Magic!

------
ehsankia
Related, I've been having a lot of success with Hypothesis [0], the Python
property-based testing library.

[0]
[https://hypothesis.readthedocs.io/en/latest/](https://hypothesis.readthedocs.io/en/latest/)

~~~
SloopJon
Hypothesis is my favorite. I also use TestCheck.js [0] for JavaScript and
RapidCheck for C++ [1], but the Hypothesis generators seem the smartest, and
they're easy to customize.

[0] [http://leebyron.com/testcheck-js/api](http://leebyron.com/testcheck-
js/api)

[1]
[https://github.com/emil-e/rapidcheck](https://github.com/emil-e/rapidcheck)

------
dnautics
A good property testing framework has two properties, 1) inputs increase in
complexity over generations, and 2) properties can shrink, enabling you to
find "minimum error conditions", so property tests can help you identify the
source of the error too.

------
carlmr
Is there something like a good proptest in C++?

~~~
patrec
[https://github.com/emil-e/rapidcheck](https://github.com/emil-e/rapidcheck)
is not bad.

