

Flash Drives Replace Disks at Amazon, Facebook, Dropbox - 127001brewer
http://www.wired.com/wiredenterprise/2012/06/flash-data-centers/all/

======
ajross
I'm not sure I buy this as a good idea quite yet. Flash is, right now, about
8x cheaper than DRAM per storage and 10x more expensive than disks. And the
DRAM latency is, of course, about 1000x better still.

Basically for many workloads, it's now ( _edited: original said "not" -- oops_
) possible to fit entire data sets in RAM, backed by cheaper traditional disks
that essentially just stream write logs.

And for workloads where storage costs actually dominate (I'm thinking
youtube/dropbox type stuff) it seems like access is mostly streaming, where HD
performance is actually quite good.

Obviously SSDs make a ton of sense in the consumer space, where you only get
one drive. But if you could pick your RAM/Flash/Disk mix yourself and tailor
it to your app, I guess I don't see a lot of advantage to putting your money
into the "middle" of that stack.

~~~
jchrisa
At scale, disk throughput more than anything else will dominate. How long do
you want backups to take? If ssds are faster they might mean fewer servers
with more disk throughput per server can give users a snappier experience.

------
PaulHoule
The economic crossover for HDD, SDD and distributed main-memory database
depends on data volume vs. query volume.

SSD and DMRD win when the query volume is high compared to the data volume
since with HDD you'll need to build multiple copies of the system (or
equivalent) to handle the load, whereas SSD and DMRD serve more queries with
less hardware.

On the other hand, in cases where query volume is low, HDD is cheaper. But if
time-to-market matters, you really do want to reduce the turnaround time for
your batch jobs and exploratory queries because in the development of KB's
with a subjective element you'll probably need to throw 10-100 things at the
wall to get the one that sticks.

------
kfury
I can absolutely see Dropbox, Google and Facebook creating a class of SSDful
machines as a middle performance layer between RAMful and Diskful machines,
but price point and storage limits have a long way to go before there's any
wholesale replacement of diskful machines with SSDs.

Also, I'm really surprised there's a datacenter that has a meaningful amount
of Facebook, Mozilla, Dropbox and Google servers in the same DC. Looks like
someone might be trying to bolster their DC image a bit.

~~~
davidandgoliath
re: bolster their DC image -- equinix doesn't need to bolster anything. $166 a
share: EQIX on Nasdaq. They're fairly well renowned for their facilities
around the world.

~~~
18pfsmt
I'm sorry, but share price has almost no meaning without knowing the number of
shares outstanding; and then, it only let's you know that the market currently
values them at ~$8B. It still seems like a non sequitur, as they are still
competing with Raxspace among others. So why wouldn't they need extra press
attention?

------
dredmorbius
How much do persistent HD shortages (especially for higher-volume, high-speed
drives) since the Japanese earthquake/tsunami and Thailand flooding have to do
with moving relative price points of SSD vs. rotational disk?

Also: for app sever storage (and temporary database tables), SSD makes a lot
of sense, and contemporary needs vs. available size (100-200 GB is plenty for
many system installs, if not a factor-of-ten overkill) makes SSD vs.
rotational a no-brainer.

~~~
corin_
I don't think it has had a much of an impact as I personally expected it to
have, though no-doubt it has helped SSDs at least a little.

The biggest effect I've seen is that a bunch of companies (generally big
companies, but not HDD creators) have made a shit-load of profit from
reselling stock they had before the floods because they realised it would make
them more money than using the drives for the purpose they were bought for.

------
patrickgzill
Facebook has an open source Linux module that uses SSDs as read and write
accelerators for regular disks: <https://github.com/facebook/flashcache>

~~~
sciurus
There's also bcache, which may make it into mainline linux one day.

<https://lwn.net/Articles/497024/> <http://bcache.evilpiepirate.org/>

------
uslic001
I have installed 3 in the past month as the prices have come down. I should
have done it sooner but did not realize how much they speed up a computer to
the point where it is hard to use a computer with a old fashioned hard drive
now.

------
qq66
Interesting that Dropbox is making heavy use of SSDs, as I would think that
most of their content is more static long-term than similar data at Facebook,
for example.

~~~
RandallBrown
they're using SSDs for their metadata about the files you store. The actual
files are stored in Amazon S3.

------
cluda01
Do flash drives still have a ceiling for how many times you can re-write a
particular cell? This is fairly common lore and the article didn't mention it.

~~~
InclinedPlane
Yes, but there are two types of flash cell you can use. SLC cells have an
order of magnitude higher write-cycle lifetime than MLC cells, which is why
there's a difference in "enterprise" vs. consumer ssds. In practice for most
uses an SLC based flash drive will be retired before it reaches its durability
limits.

~~~
StavrosK
Eh, I just had an X-25M G2 start to fail the other day, after two years of
desktop use. Intel replaced it within the week for free, but it was hardly
retired.

~~~
rtkwe
Just a question, did you take the swap partition off the SSD? That's one of
the main tips I've seen in improving SSD life-spans.

~~~
StavrosK
I didn't, but I don't know if it was swapping all that much... Maybe that was
it.

------
LaSombra
If that's true, I hope that reduces SSD/Flash prices for consumers.

~~~
tumblen
November 2011 I bought a Samsung 470 128GB for $199.

2 weeks ago I bought a Samsung 830 (faster) 256GB (twice as big) for $220.

The price drop is pretty amazing.

~~~
commandar
Newegg had 120GB 830s for $89 earlier this morning, for a more apples-to-
apples comparison.

------
th0ma5
When I read this I thought they really meant flash in a general way like USB
sticks, heh, not in an underlying technology way. I've read of several people
doing such a thing but it seems to be a unnecessary hack anymore. (see
<http://analogbit.com/node/4> )

------
thomasfl
Maybe we all should chip in on some SSD's for Hacker News?

~~~
kogir
If only it were that simple :) Sadly, HN is not disk IO bound.

------
Dinoguy1000
I really like the correction at the end of the article:

> _Due to a typo, this article originally said that a 300 terabyte hard drive
> sells for about $350._

300 terabytes for $350? In one drive? Sign me up. =D

~~~
Dinoguy1000
What's with the downvote? I was being sarcastic. =/

------
lawnchair_larry
Doesn't dropbox use AWS? Do they have their own drives?

~~~
thomasnno
The article states that Dropbox store the files themselves on AWS, and the
metadata about the files on their own servers, many of which use SSDs.

------
nirvana
Flash is really great in theory, but in my experience with intel and sandforce
controllers, it isn't reliable enough. I've bought 2 flash drives and had 4
failures already.

I've seen both drive controllers get themselves wedged to the point where they
just shut down and go dead. This happens because they aren't doing a straight
1-1 mapping and along with wear leveling (which is necessary) they're trying
to compress (sandforce) and optimize (intel) the way data is stored but their
algorithms are too complex to be fully debugged (as of last year, anyway.)

I've gone back to spinning rust as a result. I'd love to move my servers to
flash, but not until they go back to really straightforward controllers (I
hear the samsung controllers are very reliable, and this is why Apple chose
them for their laptops) and don't do anything more than wear leveling.

For performance in servers, I'm going with a lot of RAM.

Right now the drive failure rates of SSDs are way too high in my experience,
and reading between the lines and user reviews on retailer sites it seems that
my experience is not unusual.

~~~
pivo
Just curious (and I haven't really used any SSDs yet) if your drive failure
experience is with consumer or "Enterprise" SSDs? I'd expect that the
enterprise versions would be more resilient but I don't know.

~~~
corin_
They key aspects for Enterprise drives (both disks and SSDs) are generally 1.)
Performance and 2.) When you buy them they come from seperate batches (i.e. a
single failure doesn't indicate more are likely to follow).

Servers should basically be created to cope with drive failure in such a way
that % of failure isn't an impact on "will this cause problems" but merely on
working out the cost of using them.

~~~
reeses
Actually, the key difference with "enterprise" SSD is that they overprovision
the drive to have a cell reserve. That's why you can buy a 512Go consumer
drive or a 480Go enterprise drive. As cells get wonky, they are 'replaced'
from the reserve.

