
RamFuzz, a fuzzer for individual method parameters in unit tests - ingve
https://github.com/dekimir/RamFuzz#ramfuzz-combining-unit-tests-fuzzing-and-ai
======
fenollp
You are looking for QuickCheck.

~~~
Mateon1
Unfortunate that this project is only a partial reimplementation of
QuickCheck. I was hoping that this would use fuzzing coverage.

Coverage guided QuickCheck has been on my mind for a while, I believe it would
greatly improve fuzzing experience and allow to easily and thoughroughly
check/fuzz things that aren't parsers/decoders/deserializers or functions that
operate on strings in general.

~~~
nuclx
As proposed by Dan Luu:
[https://danluu.com/testing/](https://danluu.com/testing/)

~~~
dekimir
As I explain in the RamFuzz overview paper [0], Dan Luu's idea is unlikely to
work because fuzzing uses mutation techniques developed for file input, not
individual parameter values. What does seem to work is frameworks like
EvoSuite and eToc, which mutate method-invocation sequences to try and
maximize coverage.

And yes, QuickCheck (or RapidCheck[1], for C++) is what you need if you enjoy
manually specifying your test inputs. RamFuzz is a tool to support research
into automating that and, eventually, the entire process of writing unit
tests.

[0]
[https://github.com/dekimir/RamFuzz/blob/master/sci/ramfuzz.m...](https://github.com/dekimir/RamFuzz/blob/master/sci/ramfuzz.md#related-
work)

[1]
[https://github.com/emil-e/rapidcheck](https://github.com/emil-e/rapidcheck)

~~~
catamorphismic
It doesn't sound like you're talking about the same thing. Dan Luu's idea
sounded more about adding coverage guidance to QuickCheck, which presently
lacks it, not using fuzzing-like mutators.

~~~
dekimir
But how exactly does Dan propose to add coverage "guidance"? Here's what he
says:

 _" Since Zalewski has already done the work of figuring out, empirically,
what heuristics are likely to exercise more code paths, it seems like a waste
to ignore that and just generate totally random values."_

Dan does, in fact, explicitly suggest "using fuzzing-like mutators", as you
put it, in QuickCheck's random value generator. But those mutators work on
continuous file contents, not on a bunch of disparate parameter values
generated at distinct moments in the program. An early version of RamFuzz
tried to apply AFL to the log file recording all generated random values, but
all that yielded was invalid logs that couldn't be replayed. I don't know how
else one would leverage Zalewski's heuristics in a random-value generator,
though I'm obviously open to suggestions.

------
rhizome
This applies to C++, maybe the title should be updated.

~~~
dekimir
The previous submission's title does mention C++ ;)

[https://news.ycombinator.com/item?id=15789033](https://news.ycombinator.com/item?id=15789033)

