
Fuzzing the .NET JIT Compiler - matthewwarren
https://mattwarren.org/2018/08/28/Fuzzing-the-.NET-JIT-Compiler/
======
hobls
> Or in other words, a fuzzer is a program that tries to create source code
> that finds bugs in a compiler.

This is a very narrow definition of a fuzzer. There are a lot of types of
fuzzer that do not generate source code, and are not intended to test
compilers.

~~~
seanmcdirmid
When I was an undergrad in 1997, I created a fuzzer (well, we didn't know to
call it that back then) to insert one byte changes into Java classfiles, and
then used that to test our bytecode verifier along with Sun and Microsoft's.

Suffice it to say, we found lots of bugs, the most famous one being what my
grad advisor called the vacuum bug that could read a web client's environment
variables (among other things):
[https://archive.nytimes.com/www.nytimes.com/library/cyber/un...](https://archive.nytimes.com/www.nytimes.com/library/cyber/under/072397under.html)

~~~
hobls
Very cool! I think fuzzers are SO interesting. One of my favorite examples is
lcamtuf’s post on generating valid JPGs from nothing [1]. Just so awesome.

1: [https://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-
th...](https://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-thin-
air.html)

------
kodablah
I wonder, instead of an AST-generating fuzzer, if they could have used AFL on
an instrumented binary that may find more possibilities since it just works on
paths and bytes without being constrained by what this fuzzer knows are
certain constructs.

~~~
ygra
Can you easily run AFL on .NET (or jitted) code? My understanding was that AFL
works by instrumenting paths taken and kinda needed a gcc-produced binary with
normal Linux debug symbol stuff.

~~~
0xad
What AFL needs is _instrumentation_ and of course the easiest way is to get
that at compilation step, _however_ you are not constrained by anything to get
that part via other means. Check
[https://github.com/ivanfratric/winafl](https://github.com/ivanfratric/winafl)
that uses DynamoRIO.

So, to answer your question -- it wouldn't be easy but it can be done.

------
mehrdada
An obvious next step to consider might be applying EMI techniques:
[https://mehrdad.afshari.me/publications/compiler-
validation-...](https://mehrdad.afshari.me/publications/compiler-validation-
via-equivalence-modulo-inputs.pdf)

~~~
matthewwarren
Thanks for the link, I'd not heard of EMI techniques before, they look
interesting

~~~
matt_d
If you're interested, here's a collection of resources on compilers
correctness (testing & fuzzing, validation, and verification):
[https://github.com/MattPD/cpplinks/blob/master/compilers.cor...](https://github.com/MattPD/cpplinks/blob/master/compilers.correctness.md)

------
0xad
Great article. Kudos.

On a side node, my old project [https://github.com/dyjakan/interpreter-
bugs](https://github.com/dyjakan/interpreter-bugs) along with short
presentation I did on WarCon 2017 [https://github.com/dyjakan/conference-
talks/blob/master/2017...](https://github.com/dyjakan/conference-
talks/blob/master/2017-WarCon-Interpreters-Under-Pressure.pdf) (references
might be interesting for others).

------
rkagerer
Nice use of Beyond Compare for the side-by-side IL snapshots.

~~~
matthewwarren
I'm glad you liked that! (I spent more time that I should've trying to show
that in a meaningful way and I was pretty glad when I remembered about 'Beyond
Compare')

