
Backblaze Hard Drive Stats for 2016 - ingve
https://www.backblaze.com/blog/hard-drive-benchmark-stats-2016/
======
Twirrim
Backblaze's environment is abnormal. They do very little to dampen vibration
amongst other things, based on their last pod design. That puts drives under
stresses that don't reflect what you're likely to see in your environments.

What you're seeing is not the most reliable, but the most reliable under
_their_ particular set of bad conditions. The reliability under _your_
circumstances might be wildly different.

Their pod approach even results in worse performance: more vibration results
in more effort spent by the drives trying to keep the head positioned in the
right places, increasing latencies and decreasing read performance. The higher
density they push for with their pods actually compounds this. Drives vibrate.
Fans vibrate (chassis, PSU, CPU, the works) . If you don't pad them all
appropriately, the vibration of the components ends up causing the entire rack
to vibrate. By the time you've got several dozen of drives in the rack, the
whole rack will be feeling the effect.

~~~
slewis
At Google my colleague developed a simple test that made vibration effects
very obvious (this was in 2008). In a server with a bunch of disks, do random
reads or writes on all but one disk. On the final disk, do sequential reads.
Back then you could get on the order of 100MB/s sequential read speed on the
outer edge of a hard drive in ideal conditions. Under this test performance
would dive to something like 10MB/s or worse. All of the seeking would cause
vibration, which caused misreads on the disk under test. When a bad read
happens a hard drive has to wait until the next rotation to try again. We
eventually fixed this with better damping.

~~~
raubitsj
I am that colleague. While it great that Backblaze releases this information,
there are many factors that you need to keep in mind when looking at this
data.

With respect to vibration: we found vibration caused by adjacent drives in
some of our earlier drive chassis could cause off-track writes. This will
cause future reads to the data to return uncorrectable read errors. Based on
Backblaze's methodology they will likely call out these drives as failed based
on SMART or RAID/ReedSolomon sync errors.

~~~
swampthinker
HN is amazing.

~~~
Jgrubb
Threads like this are what it's all about :)

------
softawre
Funny how it is this exact blog post, every year, that really sets it in for
me how fast the last year has gone by.

Thanks for that, I guess.

~~~
atYevP
Time..../sigh

~~~
nthitz
More over I can always expect "Yev from Backblaze here". Hey Yev, miss you
from the mashupfm days!

~~~
atYevP
Heeeey ;-) Yea, I look back fondly on them. Had much less to do, less
responsibility - could spend my day frolicking on the internet...no more!

------
user5994461
Related topic: I'd like to see an update on the enterprise vs commodity hard
drives.

A few years ago, the consensus (and data) was that they were mostly the same
thing and it was arguable to pay double for the enterprise one.

But... in recent years, the consumer market have moved toward ever cheaper,
slower and power savings hard drives (e.g. reduced RPM and stop the disk when
unused for 30s).

That calls for a re evaluation of the situation.

~~~
atYevP
> Related topic: I'd like to see an update on the enterprise vs commodity hard
> drives.

Yev from Backblaze here -> we wrote about this in 2013 and have honestly not
bought too many enterprise drives since, simply because they were more
expensive and the benefit was negligible in our usage. Lately though Seagate
has had a nice run of enterprise drives and they're well-priced, so we might
be giving those more of a shot in the coming months!

~~~
pooloo1
How is the benefit negligible? You are running consumer drives that are not
designed to be run at a 100% load 100% of the time. The drives are not meant
to be spun up constantly; in fact, many of them have mechanisms designed for
them to spin down between use and lower power consumption. It's like
complaining about a drag racer not being able to haul a ton of bricks without
breaking the transmission on a regular basis.

~~~
effie
> _The drives are not meant to be spun up constantly;_

I think having the drives spin constantly is the best for lifespan, because
there are less spinups and thus components wear less.

~~~
Already__Taken
There should be no wear at all once it's running, read:
[https://en.wikipedia.org/wiki/Air_bearing](https://en.wikipedia.org/wiki/Air_bearing)

So barring any manufacturing defects that I'm soure would be abundantly clear
early on, it's down to logic or electrical failures.

------
overcast
Always amazed on how far ahead HGST is in reliability. What are they doing
that is 2-6x better in failure rates?

~~~
handruin
I've gone all-in on HGST for personal use because I've seen their results from
Backblaze and also have good luck with them so far. I have 14 of their 4TB NAS
and 22 of the 6TB NAS.

~~~
Symbiote
I'm amazed that personal use requires 188TB of storage.

That's more than my employer's Hadoop cluster...

~~~
avn2109
Maybe he does his own offsite backups with "personal use" HD's in several
physical locations, each in a highly-redundant raid configuration?

Even so it seems like a lot.

~~~
antisthenes
Someone's gotta back up that climate data...

~~~
handruin
Actually...I am doing some of that also.

------
gravypod
I'd love a set of charts made that would plot Hours Runtime vs Failure Rate
and Data Transfer Rate vs Failure Rate.

These tell us little about how good these drives hold up, it tells us more of
the churn cost that backblaze has.

------
ents
Has anyone made a failure rate vs current price chart for all drives included
in these reports to help guide purchasing decisions for normal people? I'd
love to find where the dollar per terabyte sweet spot is, with respect to
reliability.

------
runjake
Are there any reasonably scientific "death matches" like these for 2.5"
spinning hard drives?

Edit: clarification

~~~
DiabloD3
The 2.5" market has gone almost entirely SSD.

I, too, would like to see more regular SSD death matches.

~~~
cm2187
Particularly given that almost no SSD review says a single word about
reliability, which is my principal concern.

~~~
wmf
Reviewers can't really believe anything the vendors say and can't afford to
run hundreds of SSDs for years to gather real data so there's not much they
can say. By the time there is real-world experience with an SSD model it's
probably obsolete anyway.

~~~
cm2187
Yep. But that's a problem...

------
PascLeRasc
Does anyone have any insight into why they don't have any 10TB drives running?

~~~
atYevP
Yev from Backblaze here w/ some insight -> Yes! They are expensive. We are
looking to test some in our environment very soon, but may not have enough to
(we want at least 45) to truly test them. Once they drop further in price and
we can afford to drop them in some of our pods we'll be all over it!

~~~
jasode
Since you have many years of data, do you explicitly factor the observed the
failure rate patterns of particular brands into cost models when buying
drives? (E.g. more anticipated replacement cost factor when buying Seagate
drives.) Or is the failure data purely for curiosity and not fed back into a
spreadsheet? In other words, does every cost projection treat every brand with
the same predefined anticipated failure rate regardless of brand?

~~~
atYevP
> Since you have many years of data, do you explicitly factor the observed the
> failure rate patterns of particular brands into cost models when buying
> drives?

Sometimes. We do take it in to account but if a hard drive that costs less but
fails more comes around and the failure rate in our environment is within some
"tolerance" we'll get that drive, even if we might see some more failure. So
we do use the data to inform our purchasing department of the "future costs"
of drives, but a good deal's a good deal :)

~~~
jasode
I was worried that my question wasn't worded well enough to make sense but
your calculation answer was exactly what I was looking for. Very insightful
and thanks for sharing the thought process.

~~~
atYevP
I got you :D

------
rbirnie
How are you generating the raw data? Can you share your scripts that are used
to generate it?

I'd be very interested in generating raw data against the disks I maintain as
well, might also be able to share it, which might be interesting since its
primarily SSDs.

------
E6300
I love these statistics, but I would just like some clarification on what
exactly constitutes a "failure". Does the drive have to stop responding
completely, or are a few bad sectors enough to consider the drive "bad"?

~~~
atYevP
Yev from Backblaze here -> We wrote a bit about this in our SMART Stats blog
post -> [https://www.backblaze.com/blog/hard-drive-smart-
stats/](https://www.backblaze.com/blog/hard-drive-smart-stats/).

~~~
E6300
Thanks!

------
izzydata
Why do they buy such weird proportions of each type of harddrive? Do they just
use anything they can get their hands on? If the purpose was actually to test
drive failure rates wouldn't you want to buy similar amounts of every drive?

~~~
andrewbinstock
Drive failure rates can be computed across different batch sizes, provide the
batches are big enough to provide meaningful numbers. There's no need to have
the batches be exactly the same size to get useful and accurate data.

~~~
izzydata
Sure, but a drive count of 45 is not very reliable.

~~~
basch
a pod was 45 drives, so everything they do was in pod units. they would want
at lest 45-90 of a drive to have a complete set.

pods are 60 drives per unit now.

------
frik
Who owns the company that produces the HGST 3.5" drives? (the former Hitachi
drives)

Is it Western Digital? Is it Toshiba? someone else? (afaik the HGST 2.5 and
3.5 production lines were sold to different companies -right?)

~~~
currysausage
It's complicated:
[https://news.ycombinator.com/item?id=10057519](https://news.ycombinator.com/item?id=10057519)

~~~
frik
Thanks a lot, it's indeed complicated.

------
cm2187
Does anyone has any experience on the reliability of the Seagate 8TB archive
drives? They are incredibly cheap for cold storage, but a bit concerned by the
Seagate numbers in the Backblaze stats...

~~~
bkruse
Yes, I run about 800 of them now. They have not been as reliable as the 6TB
archive drives. I think this was related to a bad run that seagate had while
manufacturing. Our numbers, overall (with 6,000+ drives now, if I remember
correctly) are much lower than Backblaze's reported failures.

We also classify a drive as failed when it throws ANY SMART error, then it's
diagnosed with Seagate's internal tool (which I believe they are open
sourcing, if they haven't already) and put back into production

Seagate has given us good warranties on these drives as well. Yes a higher
immediate failure rate (first 1 month) with the 8TB drives was annoying, but
they more than made it right replacing the drives.

~~~
cm2187
Thanks!

------
devwastaken
I remember a bunch of people saying that seagate's drives with 15% failure
rates were a 'one time thing'.

------
coralreef
Was not surprised to see Seagate leading the failure rate. Had a portable 2TB
seagate drive fail within weeks. There goes my vacation photos and videos!

~~~
handruin
I had four of the Seagate 3TB drives that are listed in their ratings with
high failure rates. All four of mine died in less than 3-years. They were a
really disappointing purchase.

~~~
aaronmdjones
I had 4 Seagate drives of varying capacity and model numbers die on me within
11 weeks of each other, all were less than 3 years old and one of them was in
use for less than an hour.

Never again.

