
Show HN: Differential Fuzzing of Cryptographic Libraries - guidovranken
https://guidovranken.com/2019/05/14/differential-fuzzing-of-cryptographic-libraries/
======
guidovranken
I've posted this once before but it failed to get traction. I'm reposting this
with the permission of HN admins.

In a nutshell: This open-source project provides a framework for fuzzing
cryptographic libraries. It can find both memory bugs and implementation
errors. So far it has found about 35 bugs of varying severity in the stable
releases of major cryptographic libraries. The fuzzers run continually on
Google's OSS-Fuzz platform which helps detect bugs before they end up in
stable releases.

The article describes the technical details of the software. Here you can find
a summary of the bugs it has found:
[https://github.com/guidovranken/cryptofuzz#hall-of-
fame](https://github.com/guidovranken/cryptofuzz#hall-of-fame)

~~~
technion
Thanks for posting this. It looks like some really great tooling. I tried
fuzzing the Argon2 library a while ago and very quickly hit some of the issues
you appear to have addressed.

~~~
TheCrott
What tools/library you used to fuzz Argon2?

------
nullc
In the Bitcoin project we found and reported several bugs in OpenSSL based on
similar harsnesses that compared our code with similar functions in OpenSSL
(where they exist).

It's a pretty powerful technique at least in problem domains where fully
specified behavior is a realistic expectation.

Mutation testing-- where you break the code and confirm that your tests fail--
is also pretty powerful in those same domains.

~~~
nullc
(in particular: mutate the code, if the tests pass, test random cases
differential with the original, if any discrepancies are found then you just
found a missing test case.)

------
CJefferson
Differential fuzzing (and the closely related metamorphic testing) are amazing
ideas, I've used them in three different ways:

1) Compare two programs

2) Generate two inputs for one program which should produce the same output

3) generate two inputs for one program where there is a simple relationship
between the outputs.

In all cases, it has been a great way of testing, and has shuck out some
incredibly subtle bugs I don't think we would have found any other way.

------
mahemm
I use the high level concept pretty regularly in my day-to-day as a security
consultant specializing in cryptography, and this project is a fantastic way
to democratize the use of differential fuzzing. The only negative thought I
have about this is that I didn't think of it first!

------
colorincorrect
dumb question: but why is it so hard to create very-secure crypto libraries? i
recall NASA's code used during the Apollo missions had like an ungodly low
error rate, so surely it's feasible in practice? admittedly, comparing
anything to NASA standards is a very high bar to set, but considering the
stakes involved with cryptography, you'd imagine a similar scrutiny?

~~~
emmelaich
Apollo computers had input that was trusted, and predictable (within ranges).
It only had one job to do. Albeit a big job.

Crypto libs (or any libs) typically can be used in a vast number of ways with
completely untrustworthy input.

------
jwilk
Please don't make screenshots of text.

