
Announcing OSS-Fuzz: Continuous Fuzzing for Open Source Software - tanin
https://testing.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html
======
orf
Awesome, it's found a few bugs with Sqlite3 already:
[https://bugs.chromium.org/p/oss-
fuzz/issues/list?can=1&q=typ...](https://bugs.chromium.org/p/oss-
fuzz/issues/list?can=1&q=type%3DBug-
Security%2CBug+-component%3AInfra+status%3AFixed%2CVerified+sqlite3&sort=-id&colspec=ID+Type+Component+Status+Library+Reported+Owner+Summary&cells=ids)

------
nchammas
On the topic of fuzz testing, Python has an excellent library for property-
based testing called Hypothesis [0] [1]. I don't think it does guided testing
like AFL or libFuzzer (which OSS-Fuzz uses), but it's very powerful
nonetheless.

[0] [http://hypothesis.works/](http://hypothesis.works/)

[1] [https://github.com/HypothesisWorks/hypothesis-
python](https://github.com/HypothesisWorks/hypothesis-python)

~~~
nradov
I developed a fuzzer that generates random values based on ABNF rules, such as
often appear in IETF RFCs. So it can be used for testing RFC implementations.
It's written in Java but can be called from other tools.

[https://github.com/nradov/abnffuzzer](https://github.com/nradov/abnffuzzer)

------
newman314
I'd like to see openssh added to the list of fuzzed projects.

~~~
wyldfire
Adding a new project doesn't seem too tough. Guide:
[https://github.com/google/oss-
fuzz/blob/master/docs/new_proj...](https://github.com/google/oss-
fuzz/blob/master/docs/new_project_guide.md)

~~~
newman314
I'm not against doing the work to create the pull (unless someone else wants
to) but looking at
[https://www.openssh.com/report.html](https://www.openssh.com/report.html),
it's not clear to me if openssh[at]openssh.com or openssh-unix-
dev[at]mindrot.org should be used for something like this.

~~~
greglindahl
Try firing it at yourself to start with, then ask the openssh community once
you've got some useful data?

Edit: Thanks for the downvote! Every time I consider sending in a pile of
automated bug-reports to a project where I'm not already part of the
community, I look at them myself first. If that's bad, I'm happy to be bad.

------
seanwilson
> Recent security stories confirm that errors like buffer overflow and use-
> after-free can have serious, widespread consequences when they occur in
> critical open source software.

This project is awesome and incredibly valuable but what alternatives are
there to making the libraries it checks more secure besides rewriting them in
another language? When languages exist where buffer overflows and use-after-
free are essentially impossible it's a bit depressing that we have to rely on
fuzzing unless fuzzing can find these kinds of bugs with high reliability?

~~~
tanin
> what alternatives are there to making the libraries it checks more secure
> besides rewriting them in another language?

I'm not sure what you mean by rewriting a library.

For example, if a library is written in Ruby, fuzzing might not be suitable.
you shouldn't rewrite it in C, so that you can fuzz it. (I thinkI
misunderstand your point here. Please correct me.)

If we want to discover vulnerabilities in Ruby, we should fuzz the Ruby VM
directly. We shouldn't fuzz a Ruby library.

------
mtgx
> _In order for a project to be accepted to OSS-Fuzz, it needs to have a large
> user base and /or be critical to Global IT infrastructure_

Let's see - Firefox and/or the Tor browser? I imagine Google wouldn't be too
happy about doing free security research for Firefox, but it seems to fit the
bill quite well for the goals and mission of the Core Infrastructure
Initiative organization.

~~~
dgacmu
I think that represents a fundamental misunderstanding of Google's incentives.
(disclaimer: I'm typing this from Google, but it's purely my own opinion.)
(And wishing my experiment would finish running faster.)

Google's biggest revenue stream is ads. It has many others, but that's the big
one. Ads are seen by people who use Google search, and who browse many ad-
supported websites. Google also has a lot of users of its services, e.g.,
gmail, who have accounts. Some also entrust valuable data to Google.

Google therefore has a great deal of incentive to make sure that: (a) Nobody
messes with the ability of general users to browse the Internet safely
(preserving ad revenue); and (b) It's very hard to compromise users computers
and gain access to their Google/gmail/whatever accounts (reducing support
costs and keeping users happy), or destroy their data, or exfiltrate it. Even
if the compromise was the user's computer, it's still a very bad experience.

Project zero is a pretty good example of this incentive structure in action.
(such as fuzzing for windows font bugs:
[https://googleprojectzero.blogspot.com/2016/07/a-year-of-
win...](https://googleprojectzero.blogspot.com/2016/07/a-year-of-windows-
kernel-font-fuzzing-2.html) ).

With fuzzing itself, as DannyBee alluded to below, one of the leads of a lot
of this infrastructure, Kostya Serebryany, is also personally passionate about
seeing it be taken up to make the software of the world better. I've seen this
in action - he convinced me to use libFuzzer to improve TensorFlow's
robustness, and mentioned that he was doing so in part so I'd take that
experience back to Carnegie Mellon and spread the word. :) (and it worked -
[https://github.com/tensorflow/tensorflow/commit/7231d01fcb2c...](https://github.com/tensorflow/tensorflow/commit/7231d01fcb2cd9ef9ffbfea03b724892c8a4026e)
for example).

~~~
ocdtrekkie
One could argue, especially considering some of the disclosure situations
surrounding Project Zero that have occurred, that it exists more to shame
competitors than improve the world. :) To my knowledge, only like once have I
heard the words "Project Zero" in relation to a flaw in Google software.

But as Google definitely does do security testing on competitors' products
such as Windows, it definitely goes to show that Google would be more than
happy to test for security flaws in Firefox. And regardless of Google's
motives, finding security flaws does, in the end, make a more secure Internet
for everybody.

~~~
halbecaf
Project Zero has reported plenty of bugs in Google software:
[https://bugs.chromium.org/p/project-
zero/issues/list?can=1&q...](https://bugs.chromium.org/p/project-
zero/issues/list?can=1&q=vendor%3AGoogle)

~~~
username223
That's encouraging -- I had cynically assumed it was just a naive PR play --
but have they screwed over other Google projects by publishing 0-days at the
end of their arbitrary time window? For example, do they publish zero-days on
widely-deployed versions of Android?

~~~
dgacmu
I believe they follow the same reporting process regardless of vendor, Google
included. Here's one that Google fixed in Chrome about 15 days before the 90
day public release threshold: [https://bugs.chromium.org/p/project-
zero/issues/detail?id=16...](https://bugs.chromium.org/p/project-
zero/issues/detail?id=162&can=1&q=&start=100)

(You'll note the explicit discussion in there about the deadline:

"Chromium issues should be treated the same as any others. So there's a 90-day
deadline (which was not exceeded in this case), ...

Same disclosure warning to the Chrome team was in this bug:
[https://bugs.chromium.org/p/project-
zero/issues/detail?id=51...](https://bugs.chromium.org/p/project-
zero/issues/detail?id=513&can=1&q=&start=400)

And project zero explicitly warned the Android team about the 90 day
disclosure policy in the one bug report I checked:

[https://code.google.com/p/android/issues/detail?id=182510](https://code.google.com/p/android/issues/detail?id=182510)

Edited to add:

Here's one where they disclosed prior to Android fixing the bug:
[https://bugs.chromium.org/p/project-
zero/issues/detail?id=86...](https://bugs.chromium.org/p/project-
zero/issues/detail?id=860&q=)

with the note "deadline exceeded". Unfortunately, the link to the Android bug
is still protected, so we can't learn why AOSP hasn't fixed it yet.

