
The Smart Fuzzer Revolution [video] - nuclx
https://blog.trailofbits.com/2017/02/16/the-smart-fuzzer-revolution/
======
dguido
I'm the speaker in this video. AMA!

If you're interested in more of the technical details of how a CRS (automatic
bug finding system) works, I recommend watching this presentation from my
colleague Artem Dinaburg.

"Making A Scalable Automated Hacking System"

* [https://www.youtube.com/watch?v=pOuO5m1ljRI](https://www.youtube.com/watch?v=pOuO5m1ljRI)

* [https://github.com/trailofbits/presentations/blob/master/Cyb...](https://github.com/trailofbits/presentations/blob/master/Cyber%20Grand%20Challenge/Dinaburg_INFILTRATE_2016.pdf)

You should also keep your eye on
[https://github.com/trailofbits](https://github.com/trailofbits) \-- we are
releasing the final component of our CRS as open-source in a very short time.
Manticore, our symbolic execution framework, will be up there soon! I'm happy
to give you early access if you get in touch with me on Twitter.

~~~
1ris
I'm dealing with a very old, large, somewhat rotting codebase. it's barley
tested. Is fuzzing something for me to improve code quality, or are tests the
lower hanging fruit?

~~~
dguido
Disclaimer: I have not looked at your codebase, so this should only be taken
as my 2 cents. I might have a different recommendation if I had greater
familiarity with your exact problem.

I'm biased, but I would start with a fuzzer. Fuzzers have two key differences
for your scenario that makes me lean towards them:

 __They provide the will to act __. You mentioned the codebase is old and
rotting, so a normal bug may not rise to enough attention to fix. Finding
security bugs may give greater justification to undertake an effort to start
maintaining it properly.

 __1 fuzzer exercises more than 1 test __. I think you 'll get more bang for
your buck by integrating a fuzzer, whether it is libFuzzer, AFL, Radamsa, or
anything else. Fuzzers are not targeted to a single unit and will, ideally,
find bugs all over the place from a single, simple starting point.

That said, there are good arguments for diving into the codebase and writing
tests too. In fact, writing tests may make your fuzzer more effective.

~~~
dguido
One extra thought:

You only asked fuzzer vs tests, but I'd offer that you should update your
build settings before fuzzing. New compilers have a lot more diagnostics than
they used to. But then I'd set up libFuzzer or AFL.

------
bullsandabears
Does anyone know of a list of categorized and recommended fuzzers for
different purposes, or just more specifically for smart fuzzing web API's and
how to get started with it? Search results for this kind of stuff are hard to
parse cause they're either dated or the use cases are very specific one-offs.

I realize answering my questions with the given broadly-defined tools may be
the required manual expertise they refer to in the presentation. But I'm just
looking for a foothold somewhere at the least.

~~~
dguido
Very short answer: For web testing, you should run Burp Suite in headless
mode. Lots of people use Carbonator to do this:

[https://portswigger.net/bappstore/showbappdetails.aspx?uuid=...](https://portswigger.net/bappstore/showbappdetails.aspx?uuid=e3a26fff8e1d401dade52f3a8d42d06b)

Burp Suite is planning on adding native support for continuous integration...
integration in the second half of 2017.

If you're reading between the lines: there are _very few_ security testing
tools that are built well. So you're asking the wrong question. You don't need
a huge list. There are only a small handful of fuzzers or analysis tools I
would recommend at all, and Burp is it for web testing.

Most projects out there are hobby projects from people trying to learn
something new and ignoring what has already been done. They don't serve a very
useful purpose other than as a learning or teaching tool.

We used tried and true basics for our CRS: Radamsa, KLEE, our own open-source
binary lifter, and a Python symbolic execution framework built around Z3.
Nothing new, or hip, or magic.

~~~
_pdp_
Burp suite does not do a good job at fuzzing APIs - not biased but true. APIs
require more structured fuzzers that expose application level problems - not
like burp's fuzzers which are working on raw HTTP requests which was useful
sometime ago when you had to find bugs in the actual server implementation.
This is not relevant in the web application security space anymore apart from
the fields of research which is exactly what most web shops not interested to
do. You can still use Burp for that but the user needs to do all the heavy
lifting by hand. How does Burp do recursive XML or JSON fuzzing? It doesn't.
You can write a plugin for that but that defeats the purpose of using an off
the shelf tool.

~~~
dguido
Yep, it's maybe a little more complicated than I let on. We went through the
same process you described on a recent engagement and here was the outcome:

[https://github.com/trailofbits/protofuzz](https://github.com/trailofbits/protofuzz)

