
Exploiting Byte-Accessibility of SSDs Within a Unified Memory-Storage Hierarchy [pdf] - lainon
https://jianh.web.engr.illinois.edu/papers/flatflash.pdf
======
twotwotwo
I _think_ the sources of the claimed benefits here are 1) the OS can stay out
of some reads entirely, because PCIe devices directly answering requests to
read/write memory is a thing, and 2) for small reads [or writes, as arcticbull
notes below] you don't have to send the whole page between the SSD and the
host system, freeing up bandwidth.

The gains measured in the paper are using emulation, so, like, they didn't
really hack SSD firmware. There's got to be some cost on the controller side
to service these new requests. And they seem to be accounting for this, but of
course taking away some overheads doesn't take away the underlying limits of
the storage medium or controller.

A couple things make something like this more interesting in the future: PCIe
4 doubles bandwidth and is supposed to be coming in some AMD chipsets soon
(and the PCIe 5 spec has been finalized and, once it's really shipping,
provides another doubling), and of course Intel/Micron's new storage medium
has zippier read/writes than Flash.

I wonder whether arrangements like this over PCIe end up a middle ground
between using swap/mmap'ing vs. way pricier setups with nonvolatile DIMMs.

~~~
arcticbull
Some of the benefits claimed here are that in the paging model, even if you
want to write a couple of bytes here or there (the FS metadata issue) you have
to bring an entire page into memory, update it, then write the entire block
back out immediately. They're suggesting directly making byte-level mutations
to the PCIe device which (battery-backed to ensure consistency) tracks those
changes and writes them back out when reasonable.

In terms of SSD durability, by tracking mutations at the controller you can
avoid unnecessary re-writes so where reading may have been (page-in-block,
interpret, page-out-block), reading can now be direct and limited to the range
of interest.

At least that was my first-glance take.

~~~
infogulch
It basically eliminates the "paging" abstraction, and lets the storage
controller, with specific knowledge of the hardware implementation, to
intelligently do the minimum necessary reads/writes with the actual hardware
to express the desired changes. It makes me wonder how locking/race
conditions/consistency is implemented since that's one of the features of the
page abstraction that's not obviously addressed with this tech.

