

RAM Disks and Saving Your SSD from AFL Fuzzing - michaelrash
http://www.cipherdyne.org/blog/2014/12/ram-disks-and-saving-your-ssd-from-afl-fuzzing.html

======
JonathonW
I'm not sure how one would get statistics on this, but I can't help but wonder
how much of those ~2,000 writes/sec are actually getting written to disk.

Basically, the same property that gives AFL fuzzing such high write
amplification (it's repeatedly changing only a few bytes within a single
sector of a file) _should_ also mean that the data in question is mostly
staying in cache-- the OS is going to flush that cache out to disk
periodically, but it's _probably_ not going to be flushing that cache after
every write; that would be pretty slow and silly.

My 840 EVO tracks its total number of LBAs written, and makes it available as
a SMART attribute. Assuming your drive does the same, it might be interesting
to see how quickly that number goes up while fuzzing versus how quickly it
goes up with a more "normal" workload.

~~~
nkurz
_I can 't help but wonder how much of those ~2,000 writes/sec are actually
getting written to disk_

Approximately zero would be my fairly confident guess, although I'm more
familiar with the Linux page cache than OSX. For repeated writes to the same
small file without an explicit sync, I don't think there should be any
difference between a RAM disk and a regular drive. One way to test would be to
measure the number of writes/sec to the the RAM disk, the SSD, and a slow
spinning drive --- my theory says they should be very close to the same. But
I'd love to learn by being shown otherwise.

~~~
michaelrash
Remember from the article that AFL is being run from a VM. So, my guess is
that OS X (host) doesn't know anything about whether what is being written on
the VM is a small file or not. There are two possibilities: 1) either the OS X
activity monitor is actually measuring disk+cache when it says "disk writes"
or 2) disk writes really are happening as fast as shown in the graphs. My bet
is the later. What saves things here is the usage of the RAM disk on the
guest, since at least that is a clean mechanism for really only using RAM
instead of disk - and that carries through from the guest to the host.

~~~
nkurz
You make good points, and I hadn't considered how the VM would affect things.
I'm still doubtful that frequent rewrites of the same file would actually hit
the flash of the SSD, but with much less certainty. If it is writing this
frequently, this seems like a place for an easy optimization of the VM. Or
maybe I'm misunderstanding, and it's writing a series of small files rather
than rewriting the same one?

~~~
michaelrash
On the VM, AFL is rewriting the same small file millions of times, and the
fuzzed program is reading this file from disk (as opposed to stdin). I think
the main question is why does the OS X activity monitor show a massive spike
in disk writes when AFL is started and a corresponding decrease when it is
stopped? This is 100% reproducible, and the magnitude of the change is
unmistakable. It is absolutely clear that the spike in disk writes is due to
AFL activity. Further, this same spike does not happen when running under a
RAM disk on the VM. I'm confident you would see the same result. So, either
the OS X activity monitor has some special way to measure caching in the VM
and that is what it is showing (surely this is unlikely), or the writes really
are hitting the SSD when not running under the RAM disk. If not the later,
what other (realistic) possibility is there?

~~~
nkurz
How about this interpretation (which is only a guess, although I think it's
likely):

Activity Monitor does not distinguish between cache hits and misses, and
counts the number of reads and writes to the host filesystem rather than
summing the requests to the physical devices. Parallels presents the guest
with a virtual filesystem, and the device driver for this filesystem does no
caching of its own, as this would waste memory due to double caching. Instead,
it merely passes requests through to the real filesystem on the host. These
requests (read or write) produce system calls, and this is what is being
reported in Activity Monitor regardless of whether they use the page cache or
physical SSD.

One way to test this would be to use DTrace, which can distinguish between
requests to the filesystem and activity that reaches the device. The scripts
"hfsslower.d" and "iosnoop" are here:
[http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-
scr...](http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scripts-for-
mac-os-x/)

~~~
michaelrash
Ok, that sounds like a reasonable alternative, and Brendan Greeg is certainly
a powerful force in this area. I'll run dtrace and see what happens, but I'm
going to let a current long-running fuzzing run finish first.

------
NKCSS
I'm not sure it's really that big of a problem. If you look at
[http://techreport.com/review/27436/the-ssd-endurance-
experim...](http://techreport.com/review/27436/the-ssd-endurance-experiment-
two-freaking-petabytes) you can see that even consumer drives vastly
outperform the given rates.

~~~
CSDude
They are not regular consumer ssd drives, they are the most expensive ones
(among the consumer ssd drives)

~~~
vegardx
They are regular consumer drives. They are even markedet and priced as that.

They don't even have super capacitors, which more and more consumer drives are
coming equipped with standard these days.

