

New Middleware Technology Quadruples SSD Speed? - daureg
http://techon.nikkeibp.co.jp/english/NEWS_EN/20140522/353388/

======
zurn
This is just some PR wildly exaggerating results from a SSD garbage collection
paper.

The linked graphs show the big improvement in just one of many workloads and
only in the case where the SSD is near-full, and don't show what disk/ftl
they're comparing against. But even then these are their own hand picked
benchmarks. (presumably the details will be published along with the paper at
some point)

The trick itself is that they scramble the LBA addresses of blocks with some
1:1 function. But it raises the question why the FTL doesn't already do this.
Which is a reason to suspect they're benchmarking against some el cheapo SSD
with a dumb FTL.

------
mnw21cam
Sounds basically like the fundamental research on log structured file systems
done back in 1992. [http://www.stanford.edu/~ouster/cgi-
bin/papers/lfs.pdf](http://www.stanford.edu/~ouster/cgi-bin/papers/lfs.pdf)

------
leeoniya
i wonder if this is something similar to USB Supercharger [1]? i personally
used this back in the USB 2.0 days to make running Firefox Portable bearable
from what was already the fastest 4k random flash drive [2]. The write speed
increase was truly astonishing, there was nothing like it on the market. If
you look at their CrystalDiskMark screenshots, the 4k random goes up from 0.01
MB/s to 10MB/s - no joke. On my faster drive the increase was only a mere
100x.

[1] [http://easyco.com/usb-sc.htm](http://easyco.com/usb-sc.htm)

[2] [http://www.amazon.com/Patriot-Xporter-Boost-Flash-
PEF8GUSB/d...](http://www.amazon.com/Patriot-Xporter-Boost-Flash-
PEF8GUSB/dp/B000VE2RP6)

~~~
acqq
It looks quite suspicious. Are you sure they don't just turn on the OS-
provided write caching on the USB drive, which is by default turned off? No
need for any additional program:

[http://www.techsupportalert.com/content/how-optimize-usb-
dri...](http://www.techsupportalert.com/content/how-optimize-usb-drive-
performance-windows-7.htm)

There's nothing on [http://easyco.com/usb-sc.htm](http://easyco.com/usb-
sc.htm) which would assure me they are doing something special.

~~~
leeoniya
pretty sure it was ntfs formatted, so it would have already been enabled. not
only that, they actually have a custom partition that gets created and needs
to be mounted for their stuff to work. it doesnt "just work", so there's more
going on there.

~~~
acqq
If the write cache is on (as you claim it was!) you don't benchmark the times
it takes for some write actually end on the medium, only the time be copied to
the RAM, so the measurements you quote have no sense.

They also mention on their page "when the light stops blinking it's over"
which means exactly this: the software already consider everything copied, but
the movement from the RAM to the device still goes on -- and that is not
measured. Moreover, the OS can combine more writes to the same area to one to
the device so you can actually observe less writes to the device. I still
suspect that software to be a kind of snake oil.

~~~
leeoniya
interesting. i think i still have a license left if you want to test it out,
lemme know.

~~~
acqq
Even without the license, I have already some time ago measured the overhead
of copying a lot of small files with and without OS write cache. They cite 1
GB data with the average file size of 154 KB which gives around 6500 files to
copy, all on FAT32 (see "How fast is SuperCharger?") The overhead of the write
cache turned off for that number of files on the old stick I had (on FAT32)
was around 2 minutes and that, of course, measuring "the blinking stops" in
the "on" case.

------
nkurz
This post is quite vague on the details, but I came across a a more detailed
paper recently that also promised big gains:

[http://www.ece.eng.wayne.edu/~sjiang/pubs/papers/ouyang14-SD...](http://www.ece.eng.wayne.edu/~sjiang/pubs/papers/ouyang14-SDF.pdf)

SDF: Software-Deﬁned Flash for Web-Scale Internet Storage Systems

Working with/as Baidu, they are using an FPGA as a custom flash controller
that exposes the underlying "channels" more directly.

------
zokier
It is stupendous how much effort we are using to make our nice flash disks
look like the old rusty rotary ones instead of (re)writing our software to be
aware of the true characteristics of the underlying hardware.

~~~
JohnBooty

      our software to be aware of the true characteristics of the underlying hardware.
    

Ewww, no.

That's a terrible goal, and a poor understanding of how things currently work.

Traditional spinning disks aren't dumb devices. There is a lot of stuff
happening at the firmware level. Address translation, remapping of bad sectors
to spare sectors, command queuing, etc.

It's complicated stuff, to put it mildly, and you're suggesting that we ought
to duplicate these tasks across every app and/or driver and/or OS?

Even if (to choose a single, arbitrary application) Postgres had some kind of
_really_ specialized code that bypassed the SATA protocol and had direct
access to the physical flash memory cells of a single manufacturer's SSD
firmware, the odds of achieving a performance win are pretty low and the
complexity of Postgres is sure to have increased sharply.

Maybe you could move up a level, and have that kind of specialization at the
driver level and not the application level, but still: yuck. Hardware and
software creators should focus on kicking ass at implementing protocols, not
on ignoring protocols and creating compatibility nightmares.

~~~
zokier
> It's complicated stuff, to put it mildly, and you're suggesting that we
> ought to duplicate these tasks across every app and/or driver and/or OS?

I was thinking the filesystem layer.

> Hardware and software creators should focus on kicking ass at implementing
> protocols, not on ignoring protocols and creating compatibility nightmares.

I didn't mean that protocols should be ignored. I did mean that the protocols
should evolve. Of course the transition period might be bit painful.

Some existing work:

Open NAND Flash Interface Working Group
[http://en.wikipedia.org/wiki/Open_NAND_Flash_Interface_Worki...](http://en.wikipedia.org/wiki/Open_NAND_Flash_Interface_Working_Group)
(note the "Block Abstracted NAND" which is an example of the kind of mid-way
solution I'm thinking of)

UBIFS [http://en.wikipedia.org/wiki/UBIFS](http://en.wikipedia.org/wiki/UBIFS)
this also has layered approach, UBI layer for managing the raw flash and then
on top of it the proper FS.

~~~
JohnBooty
I re-read my initial post and I just wanted to say that my negative tone was
totally out of line. I'm sorry I posted such a condescending and snarky reply.

I think you really do raise an interesting point, and thank you for the links!

------
Sami_Lehtinen
They just increase fragmentation and decrease amount of data written to disk.
These are very traditional trade offs especially with database WAL and LSFS.
They didn't tell if it decreases read speed, because even if SSD 'seek times'
are low, most drives read random blocks from disk much slower than linear
data. Which means that fragmenting data especially in very small blocks might
cause reduce the drive read speed.

------
awestroke
I wonder if this reduces durability as it inhibits the NAND block rewrite
balancing or whatever it's called

------
bobbles
"In a simulation, the research team confirmed that the new technology improves
the writing speed of SSD by up to 300% and reduces power consumption by up to
60% and the number of write/erase cycles by up to 55%, increasing product
life. Because, with the new method, it is not necessary to make any changes to
NAND flash memory, and the method is completed within the middleware, it can
be applied to existing SSDs as it is."

Promising news! Essentially removing fragmentation on the fly instead of just
accepting it as a done deal on SSD

------
callesgg
Is there a way to actually access the NAND chips in a better way? Having the
ssd firmware guessing stuff seams like quite a waste.

------
mpg33
Didn't realize NAND flash memory couldn't write over itself...ie it is
constantly having to use new storage areas

~~~
gnoway
It can write over itself, but does so in pages which are typically larger than
the amount of data you want to write. So you have to find a page to write,
read it, modify the data in memory, erase the page, then write to it. I think
the page size actually varies but wikipedia[1] says 256KB.

You also want to spread out your writes anyway, because each NAND cell has a
limited number of program/erase cycles before it stops accepting writes.

This is why TRIM support was such a big deal a few years ago, since it let the
OS communicate when a filesystem block was no longer in use, simplifying the
above to erase-then-write or in some cases, just write.

[1]
[https://en.wikipedia.org/wiki/Write_amplification#Basic_SSD_...](https://en.wikipedia.org/wiki/Write_amplification#Basic_SSD_operation)

~~~
rasz_pl
You dont need to erase every time. Erased page is all ones (#FFFFF). If data
you modify doesnt need more active bits than it started with you can simply
overwrite. This means you can do things like counting down on the same NAND
page without erasing every single time. Just make sure you only flip 1s to 0s.

------
integricho
They only got FTL wrong, because it's a Faster Than Light engine.

