
Hard Drive Cost per Gigabyte - ingve
https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/
======
synack
I wrote a thing to pull the prices of all the hard drives on Amazon every 12
hours and sort them by price per GB.
[https://diskprices.com/](https://diskprices.com/)

~~~
RobLach
I'm still wondering why external drives are cheaper than internal ones.

It seems silly that if I wanted to buy an 8TB drive for my desktop I should
buy an external and toss the enclosure.

My only guess is market pressure because of higher demand. Possibly externals
drives are pulled from older stock? Does anyone have another explanation?

~~~
fencepost
Be careful about what drives you do this with - I've seen reports of some of
these drives not having standard SATA headers once they're shucked.

~~~
arthurfm
Are you referring to the 2.5" HDDs with native USB connectors? I've never come
across an external 3.5" HDD like that.

------
timerol
I would be interested in seeing the log of the prices on the vertical axis.
The curve seems to fit the exponential decline in prices that I expect to see.
I think the difference in regimes would be more clear cut that way.

Just picking the end points and the middle, I see: (11 cents, 2009), (5 cents,
2013), (2.8 cents, 2017). So it went from slightly more than halving in cost
in the first 4 years (55% reduction) to slightly less than halving in cost in
the second 4 years (44% reduction). There's definitely a bit of a slowdown
there, but not the huge difference that the article claims.

Of course, the first half includes the Thailand flood, and the second half
doesn't. Googling for some historical data leads me to
[http://www.mkomo.com/cost-per-gigabyte](http://www.mkomo.com/cost-per-
gigabyte), which claims that (from 1980-2010), the cost of a gigabyte has been
halving in about 14 months, which is closer to the decay pre-Thailand flood.

~~~
akrasuski1
That is exactly what I thought - it seems to follow the exponential decay. A
quick and dirty graph with logarithmic scale on price:
[https://imgur.com/a/90a4p](https://imgur.com/a/90a4p)

Pretty much a straight line, except for the Thailand hiccup.

------
gwern
There's a clear regime shift in the slope pre/post Thai floods. It's not a
surprise that the floods set back Kryder's law by years, but does anyone have
any insight in why it would affect the rate of progress and switch hard drives
to a much slower rate of improvement? Surely the floods wouldn't still affect
investment or R&D? Or is the timing merely a coincidence and reflect consumer
demand being saturated, switches to SSD etc?

~~~
briffle
There was also some consolidation at that same time. For example, Western
Digital purchased Hitachi in 2012. At around the same time, Fujitsu, Lacie,
and Samsung also were acquired.

[http://www.itcandor.com/hard-disk-suppliers/](http://www.itcandor.com/hard-
disk-suppliers/)

~~~
chx
Lacie, as far as I am aware, never manufactured their own hard drives. Look
here for example in 2006
[http://forums.macresource.com/read.php?1,85231,85262](http://forums.macresource.com/read.php?1,85231,85262)

> Thank you for your email. We use a few different manufacturers, so we would
> not be able to tell you which drives go to which item (except the 500GB
> drives which are Hitachi). In general we do use Western Digital, Seagate,
> and we did use Maxtor before they were bought by Western Digital. For DVDs,
> we use Samsung.

Wikipedia says , much earlier:

> As a subsidiary of Quantum, La Cie was licensed as the exclusive
> manufacturer of Apple-branded external SCSI hard drives, using Quantum hard
> disks.

~~~
Theodores
This confirms what I remember. A 3D modeller I worked with was deeply upset
when Maxtor took over Quantum as the 2Gb SCSI disks that were carried around
at the time were LaCie, apparently with Quantum disks which were allegedly the
best for replaying video frames. At the time I did not know LaCie was owned by
Quantum. Back then there wasn't the internet to quickly Google brands so you
only had marketing information to go on.

~~~
chx
o_O Maxtor purchased Quantum in October 2000. This is the same month when
Adwords left beta. What the heck are you talk of? By 2000 home broadband is
beginnning to spread, dial up Internet is everyday in most of the so called
developed world.

------
tytso
The key flaw in this analysis is that the author is only using consumer grade
drives from Amazon. And yes, I think what is going on is that consumers don't
really need much more than 4TB. Heck, most consumers have trouble filling 1 or
2TB. It's for the same reason that drive managed SMR drives haven't really
been a big thing, in the consumer market. Who cares about 15-20% more bytes
when you can't use all of the bytes in a 4TB CMR drive?

Companies at the "hyper-scale cloud companies" (e.g., Facebook, Amazon,
Microsoft, Google) can actually use the extra bytes, and they are probably the
ones that will be able to use the 10TB, 12TB, 14TB, 16TB etc. drive that are
on the vendors' roadmap[1].

[1] [http://www.pcworld.com/article/3162084/storage/seagates-
road...](http://www.pcworld.com/article/3162084/storage/seagates-roadmap-
includes-14tb-16tb-hard-drives-within-18-months.html)

It may be that Amazon might not be interested in stocking these drives because
they won't sell enough of them to consumers. And if some of the discussions
proposed in Google's "Disks for Data Centers"[2] paper come to fruition, such
as adding a second disk actuator arm, or changing the height of the disk
drives, there might be a further diversion between drives optimized for the
consumer market and drives optimized for the "hyper-scale cloud market".
(Note: the intent of Google publishing this paper is to hopefully allow
discussions amongst the community of hyper-scale cloud storage customers to
work with the HDD vendors for something that helps the whole high capacity
storage industry.)

[2]
[https://research.google.com/pubs/pub44830.html](https://research.google.com/pubs/pub44830.html)

~~~
gwern
> The key flaw in this analysis is that the author is only using consumer
> grade drives from Amazon

? This is Backblaze, a commercial storage company. They buy in bulk in
thousands of drives, not from Amazon, and the prices are what they paid. As
they say, "The price you might get at Costco or BestBuy, or on Amazon will
most likely be higher." It's the trend and comparisons which matter, not so
much the absolute price.

~~~
tytso
In the article the author asserted that BackBlaze was typically only using
consumer drives. So that makes them very different from the companies in the
"hyper-scale cloud storage" market.

And if the trends the author is talking about consumer disk drives, I would
argue that's a very incomplete picture. It doesn't include the traditional
enterprise users, and it doesn't include hyper-scale storage companies that
are using the bigger disks. Or else why else do you think that Seagate is
putting 14T and larger drives on their roadmap:
[http://www.pcworld.com/article/3162084/storage/seagates-
road...](http://www.pcworld.com/article/3162084/storage/seagates-roadmap-
includes-14tb-16tb-hard-drives-within-18-months.html)

This is very different from what the article was asserting, which was that
disks weren't getting much bigger, and that they aren't getting cheaper per
gigabyte. Even from a trend perspective, it's just not correct. (For example,
it's not taking SMR disks into account; but that's because the consumers will
probably not be interested in SMR drives.)

~~~
brianwski
> BackBlaze was typically only using consumer drives

Disclaimer: I work at Backblaze. If anybody is curious, in our blog posts we
list the exact serial numbers of all of the drives. See here for an example:
[https://www.backblaze.com/blog/wp-
content/uploads/2017/05/bl...](https://www.backblaze.com/blog/wp-
content/uploads/2017/05/blog-hd-stats-q1-2017-table.jpg)

We have been at this for a while, and the distinction between "consumer drive"
and "enterprise drive" is not what most people think it is. But in the end it
simply doesn't matter -> the differences between failure rates of individual
MODELS of drives in one line is larger than any difference in failure rates of
"consumer" vs "enterprise".

~~~
tytso
It's not just "consumer drive" versus "enterprise drive". It's also "consumer
drive" versus "hyper-scale cloud storage drive". Google used to use consumer
drives (and the latter category didn't used to exist) but the world has moved
on, and if you are looking for the lowest storage TCO (again, please see
Google's "Disks for Data Centers" paper), I'm going to suggest it's a lot more
than just using 4T drives.

Hint: you're assuming the only cost is just the raw spindle. You also need to
take into account the cost of power, cooling, some portion of the server that
the disk is attached to, the cost of the tray, etc., etc., etc. This is all in
the "Disks for Data Center" paper, so I'm not giving away any corporate
secrets.

Disclaimer: I work at Google and I am a co-author (but only in a very small
way) of the "Disks for Data Centers" paper. We have also been at this for a
while...

~~~
brianwski
> Hint: you're assuming the only cost is just the raw spindle.

Not me, not sure who you are responding to? I was just correcting the record
that Backblaze ONLY used "consumer" drives when some of our drive serial
numbers are "enterprise" drives, I wasn't commenting on total cost of
ownership.

Backblaze has our own spreadsheet of total cost of ownership, and we are
mostly deploying 8 TByte drives right now because the increased density means
renting fewer cabinets and paying less for electricity. I'll just make the
point that the spreadsheet takes the inputs of WHICH DATACENTER because
different locations pay different amounts for electricity.

> I am a co-author of the "Disks for Data Centers" paper.

I read it when it came out, it is filled with good ideas, concepts,
possibilities, and suggestions. Very worth reading. However, it is all about
spinning drives, which may not be the lowest cost and highest performance in
"X" number of years. There is a lot of disagreement around this point (even
within Backblaze) but I think "X" is "really close" like within two years.
Even if the individual SSD costs more per GByte, like you brought up, the
savings in electricity (our number one datacenter cost) are really good which
might make up for it soon (again, depending on the region they are deployed
in).

But for at least the next two years (and possibly five?), I think we all know
we're deploying spinning drives. :-)

~~~
tytso
Well, the Backblaze blog post only talks about the raw spindle costs, and
strongly implies that 4TB drives were the lowest cost option and ignores the
existence of any disks larger than 8TB. Which is true only if you are talking
about consumer drives and only concerned about raw spindle costs. Otherwise,
it's a highly misleading post.

The electricity cost and the fractional share of the disk tray and server to
attach the HDD are mostly constant no matter whether you use a 4TB, 8TB, 10TB,
or 14TB disk. So if you are only worried about renting fewer cabinets and
paying less for electricity, then if 8TB drives make sense, 14TB drives would
make even more sense, no?

Of course, you have to worry about more than just bytes; you also have to
worry about IOPS. So it starts being about whether you can use nearly all of
the IOPS in every drive. (This is why having a "hot pool" and a "cold pool" of
disks really don't make that much sense. You'll leaving money on the table by
wasting the potential usable IOPS in your cold pool. Instead you can get a
much more TCO positive solution by mixing your hot and cold data so you case
use most of the IOPS and most of the bytes for all of your drives.)

But if you think SSD's are so "cheap", why not use 14TB drives plus SSD's?
Perhaps because using SSD's for storage (as opposed to cache) really isn't
that cheap as people think. Sure, lots of IOPS, but unless the bytes you need
to read are the right ones to be stored in the SSD, the cost per byte for
SSD's is quite astronomical, even taking electricity, server attachment costs,
etc., into account.

As a result, I believe using large numbers of spinning drives will make sense
for a lot longer than you think.

------
Fej
Many people are saying here that the slowing of the rate of change is due to
business consolidation... I would argue that it is probably due to physics
instead. The traditional way of storing more data on a platter, shrinking the
size of each bit, has just about run out. Each bit is now stored on a handful
of atoms. The drive manufacturers are trying new things to get around this
problem (e.g. shingling) but once each bit is down to a few atoms you can't
really go any smaller. Further improvements in disk technology will probably
have to be a bit more radical.

~~~
StillBored
Except oddly enough, you can frequently buy a drive wrapped in a USB enclosure
for 10-15% less than the bare drive...

Plus, the sleep modes on a lot of "consumer" drives significantly slow them
down when used in a desktop environment (as well as creating second long
pauses in RAID arrays with more than 5-6 drives), and the option to disable
that "feature" is slowly disappearing on drives without the "enterprise" tax.

Market segmentation shenanigans like this would never fly in a healthy
competitive market are now very common.

~~~
Fej
That's true. Honestly I'm just hoping that HGST keeps their reputation as
reliable HDDs even though they've been bought by WD. Backblaze always found
that they fail less often.

------
slyrus
And SSD costs still 10x per GB... Hoping that comes down.

~~~
tombrossman
Agreed, I'm having to constantly manage space and move media files to external
drives to keep using a SSD for my primary drive. I bought 30GB, 128GB, and
256GB drives as prices came down and I had hoped 1TB drives would be under
$200 by now. Not even close. What is worse, I built a new desktop and planned
to use a M.2 SSD for even higher speeds. That's a long way off now and will
probably have to wait for the next build.

Truly a first-world problem but damn, where are my cheap SSD's?

~~~
tertius
They're here.

Samsung 960 EVO 500GB at $250. That's insane my friend. I guess it depends how
long you've been in the game.

I remember my 40MB drive being huge after we upgraded from a 12MB drive. It
was truly hard to find enough to fill it up with.

~~~
agumonkey
Once tiny tiny sad thing, they shifted the entry models since. I used to find
40$ average but good brand (intel) ssd; now it's impossible.

~~~
mark-r
That's exactly the opposite of the way I expected things to turn out. Hard
drives with their moving parts have a floor on the price, just because you
can't get all those parts any cheaper; the only way to get the $/TB lower is
to keep making them bigger. SSDs have no such problem, so I expected on the
low end we'd see SSDs undercutting hard drives by a lot. Where are the $20
SSDs?

~~~
tertius
Here:
[https://www.amazon.com/gp/product/B010Q588D4?tag=slicinc-20&...](https://www.amazon.com/gp/product/B010Q588D4?tag=slicinc-20&ascsubtag=4fa774ec666211e7886b0e774cf9d6690INT)

~~~
agumonkey
I often run from usb flash drives, never tried a recent micro sd; I wonder how
different they are in terms of everything.

------
SubiculumCode
I noticed and was irritated with the 4 Tb anomaly when I was setting up a
RAID6 trying to achieve 30 Tb of usable space after mirroring and a hot spare.

~~~
SubiculumCode
Enterprise class,btw

------
elorant
Judging by the price points it seems that they're not buying enterprise level
HDDs. There is no way you could buy an enterprise HDD today at $0.02 per GB. I
bought a 4TB Seagate Constellation a couple of months ago and the price was
more like $0.06 per GB.

~~~
breakingcups
They mention it in their article, as well as many of their other articles.
They use "consumer" drives.

For a more in-depth explanation, see
[https://www.backblaze.com/blog/enterprise-drive-
reliability/](https://www.backblaze.com/blog/enterprise-drive-reliability/)

~~~
brianwski
Copy and pasted from my response in a different comment:

> BackBlaze was typically only using consumer drives

Disclaimer: I work at Backblaze. If anybody is curious, in our blog posts we
list the exact serial numbers of all of the drives. See here for an example:
[https://www.backblaze.com/blog/wp-
content/uploads/2017/05/bl...](https://www.backblaze.com/blog/wp-
content/uploads/2017/05/blog-hd-stats-q1-2017-table.jpg)

We have been at this for a while, and the distinction between "consumer drive"
and "enterprise drive" is not what most people think it is. But in the end it
simply doesn't matter -> the differences between failure rates of individual
MODELS of drives in one line is larger than any difference in failure rates of
"consumer" vs "enterprise".

TL;DR - Backblaze uses both "consumer" and "enterprise" drives. But check the
individual drive serial number, that is more important than arbitrary
marketing groupings.

------
MR4D
When can we finally start measuring cost per TB ?

The need to constantly multiply numbers by 1,000 is rather annoying. Yes, we
can all do it, but still.

------
xyzxyz998
I want to hear from people who are:

1) Coders, not designers

2) Use machine only for production, nothing "fun"

Do you ever feel the need for going above 256 G?

I have 3 operating systems installed- Win10, Win7, Arch, About 20GB of media,
Office 2015, etc on my primary production machine but I'm still using less
than 120 GB. I'm asking because I'm wondering if any developer has a reason to
not go for sub-100$ SSD ~ 256 GB and instead opt for 4TB of HDD?

~~~
codemac
Only because consumer filesystems suck.

I'd love to have a completely versioned filesystem, at all times, regardless
of how big it got.

~~~
jl6
I am working on such a thing. Would you be interested in testing it?

~~~
codemac
I didn't see this before! Please feel free to ping it my way, and I'll let you
know what my experiences are.

------
galantbutterfly
The lowering costs of 8tb HDD's combined with it being Amazon Prime day, means
you can get one of these bad boys for only $150. I just ordered 2.

[https://www.amazon.com/Seagate-Expansion-Desktop-External-
ST...](https://www.amazon.com/Seagate-Expansion-Desktop-External-
STEB8000100/dp/B01HAPGEIE/ref=sr_1_3/130-8064997-0146330?s=electronics&ie=UTF8&qid=1499794547&sr=1-3&keywords=8tb+external+hard+drive)

