
Dropbox’s Exodus from the Amazon Cloud - moviuro
http://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire
======
jamwt
I want to point out one other thing about this project, if only to assuage my
guilt. And this community of hackers and entrepreneurs seems as good a place
as any to clear the air.

There is a necessary abridgement that happens in media like this, wherein a
few individuals in lead roles act as a vignette for the entire effort. In
particular, on the software team, James and I are highlighted here, and it
would be easy to assume we did this whole thing ourselves. Especially given
very generous phrasing at times like "Turner rebuilt Magic Pocket in an
entirely different programming language".

The full picture is James and I were very fortunate to work with a team of
more than a dozen amazing engineers on this project, that lead designs and
implementations of key parts of it, and that stayed just as late at the office
as we did during crunch time. In particular, the contributions of the SREs
didn't make it into the article. And managing more than half a million disks
without an incredible SRE team is basically impossible.

~~~
james_cowling
Couldn't agree with this more.

One of the things that's not immediately apparent in an initiative like this
is the huge amount of effort required to provision, deploy and manage a system
of this scale, across a whole bunch of different teams. We couldn't have done
any of this without a tremendous group of SREs with the wisdom to invest
heavily in automation, in addition to all the hard work.

We'll follow up in a week or two with a blog post highlighting some of the
work here. I think the framework for disk remediation (automatically detecting
failures, re-provisioning, etc) is particularly interesting.

~~~
tomatoman
Is it true a girl broke your arm while arm wrestling?

~~~
james_cowling
Haha, who am I to argue with a story like that?

------
riobard
> Measuring only one-and-half-feet by three-and-half-feet by six inches, each
> Diskotech box holds as much as a petabyte of data

This number is very interesting. Basically Diskotech stores 1PB in 18" × 6" ×
42" = 4,536 cubic inch volume, which is 10% bigger than standard 7U (17" ×
12.2" × 19.8" = 4,107 cubic inch).

124 days ago Dropbox Storage Engineer jamwt posted here
([https://news.ycombinator.com/item?id=10541052](https://news.ycombinator.com/item?id=10541052))
stating that Dropbox is "packing up to an order of magnitude more storage into
4U" compared to Backblaze Storage Pod 5.0, which is 180TB in 4U (assuming it's
the deeper 4U at 19" × 7" × 26.4" = 3,511 cubic inch). Many doubted what jamwt
claimed is physically possible, but doing the math reveals that Dropbox is
basically packing 793TB in 4U if we naively scale linearly (of coz it's not
that simple in practice). Not exactly an order of magnitude more but still.

Put it another way, Diskotech is about 30% bigger in volume than Storage Pod
5.0 but with 470% more storage capacity.

That was indeed some amazing engineering.

~~~
james_cowling
To be fair to Backblaze this level of storage density is really only possible
with recent advances in disk technology (higher densities, SMR storage, etc).

Also not everyone wants to be packing a petabyte into a box. At that level of
density you need to invest a lot of effort in replication strategies, tooling,
network fabric etc to handle failures with high levels of
availability/durability.

~~~
jamwt
Yes, a 1PB+ failure domain only made sense because Magic Pocket is very good
at automatic cluster management and repair.

~~~
samstave
Are you using spindle or ssd or flash? (admittedly I dont know if you consider
flash and ssd to be the same)

What is your price per GB raw?

~~~
jamwt
> Are you using spindle or ssd or flash? (admittedly I dont know if you
> consider flash and ssd to be the same)

We have flash caches (NVMe) in the machines, but the long-term storage devices
are spindle--PMR and SMR.

> What is your price per GB raw?

I cannot disclose, exactly, but it is well below any other public price I've
seen.

~~~
voltagex_
So NVMe is "ready for production"? Do you think you'd use it at home, or only
in the enterprise?

~~~
jamwt
I mean, if at home you need 400k IOPS, 1GB/s of writes, and 2.2GB/s of
reads... go for it! I sure don't, but more power to you. :-)

~~~
voltagex_
I'm tempted to get one and set it up as some kind of cache in my NAS. I'm
already in silly territory with it now, anyway. 1GB/s of writes sounds crazy
though - I haven't got anything that can write to it that fast!

~~~
stock_toaster
I have a PCIe NVMe card (with a Samsung 950 pro M.2 in it) in my home server,
which serves as an L2arc device for a ZFS pool. It is pretty nice. Runs a bit
warm though.

~~~
voltagex_
We are now well offtopic but I've seen some advice recently that advises using
an SLOG rather than L2ARC for a home NAS.

~~~
chronid
I think that a home nas would have a _lot_ of async writes and very few/none
sync writes. So a ZIL dedicated device (the SLOG) is not really
useful/helpful. I'm not sure even if you really need an L2ARC device, just
slap 8/16Gb of RAM on it and you will be happy...

Obviously if you are playing seriously with VMs and databases and whatever
else both of those (SLOG/L2ARC) may become important for you, I'm going for
the "i'm just using this to store my raw-format photos, backups for the taxes
and other big files" kind of usage here. :)

~~~
voltagex_
You don't have any contact details but I'd definitely like to talk to you more
about home NAS.

------
jamwt
Hi HN! A couple of us from the Magic Pocket software team are around to answer
questions if anyone has some.

~~~
abhv
Does magic pocket use any non-standard error-correction algorithms, or just
parity-style RAID5 or 6?

~~~
james_cowling
We use a variant on Reed-Solomon coding that's optimized for lower
reconstruction cost. This is similar to Local Reconstruction Codes but a
design/implementation of our own.

The data placement isn't RAID. We encode aggregated extents of data in
"volumes" that are placed on a random set of storage nodes (with sufficient
physical diversity and various other constraints). Each storage node might
hold a few thousand volumes, but the placement for each volume is independent
of the others on the disk. If one disk fails we can thus reconstruct those
volumes from hundreds of other disks simultaneously, unlike in RAID where
you'd be limited in IOPS and network bandwidth to a fixed set of disks in the
RAID array.

That probably sounded confusing but it's a good topic for a blog post once we
get around to it.

~~~
pnathan
That is _extremely_ cool. Please tell the author(s) of that system that a
stranger on the internet has appreciation for that feat!

~~~
willglynn
You might be interested in a paper Microsoft published a few years ago,
entitled "Erasure Coding in Windows Azure Storage", which describes some
similar concepts in greater detail:

[http://msr-waypoint.com/en-us/um/people/yekhanin/Papers/Usen...](http://msr-
waypoint.com/en-us/um/people/yekhanin/Papers/UsenixATC_2012.pdf)

------
barkingcat
The word "cloud" loses all meaning in this article.

"The irony is that in fleeing the cloud, Dropbox is showing why the cloud is
so powerful. It too is building infrastructure so that others don’t have to.
It too is, well, a cloud company."

Wait ... so using AWS is "cloud", having your own servers is "cloud" too.
Everything is cloudy!

~~~
jimbokun
Was discussing "in the cloud" with my wife yesterday, and I said "It just
means on some company's servers." Which suddenly de-mystified the whole
concept for her.

~~~
deanCommie
I was at PDC in 2008 when Microsoft announced "Windows Azure".

I've never been more confused in my life. "Wait, what does this version of
Windows do?"

Luckily they rebranded it as just "Microsoft Azure" which makes a shit ton
more sense.

------
asendra
So, just for fun. The video quotes "over 1PB" of storage per box.

I count 6 columns of 15 rows/drives. Might be 7 columns even, there's some
panels that aren't fully open on the video.

So, 90-105 drives. I'm guessing they are using 10TB drives, although maybe
they can get bigger unannounced drives? Roughly, the math seems to check out.

Quite impressive. Guess the Backblaze guys need a Storage Pod 6.0 soon :P (I
know, I know, different requirements/constraints)

~~~
atYevP
Yev from Backblaze here ->

Yea :D Though if we used 10TB drives we could get about 450TB in one pod, it
would just be hilariously expensive for us to do so. Granted that's not 1PB
per pod, but we'd imagine our costs are lower. But yes, different use-cases,
and we're working on 6.0 ;-)

------
echelon
Unrelated to the content of the article--I've never seen the oft talked about
"anti-adblocker interstitial" before. I was surprised to find that the website
blocks viewing of the article entirely because I'm running Adblock.

It's an interesting subject. I'll never click on or be persuaded by ads, so
they're not really gaining anything by showing me ads. I'm essentially
worthless traffic to them no matter how they cut it.

I do understand the problem they're trying to solve, and I would like to pay
for content that I find worthwhile. A convenient microtransaction protocol
where I _don't have to sign in or interact in any way_ would be nice. (I don't
want to waste time interacting with their bespoke account system / sign-on,
even if it is "frictionless".)

~~~
roel_v
"I'll never click on"

Doesn't matter, impressions are valuable to marketers as well.

"or be persuaded by ads"

Yes you will, you just think you don't/won't.

~~~
echelon
> Yes you will, you just think you don't/won't.

That's a really blanket statement. I'm rather minimalist, and I typically form
negative opinions of the things I see in ads.

~~~
roel_v
"and I typically form negative opinions of the things I see in ads."

Sure, until it's something that fits your world view, and then you don't.

Look, I'm usually the first to (excessively) qualify sweeping generalizations
I make with 'generally', 'statistically speaking', 'ceteris paribus' and other
weasel words. Yet in the context of pervasive marketing, I don't need to -
_everybody_ is swayed by _some_ form of advertising, whether you realize it or
want to admit it or not; 50 years of research and billions of dollars spend on
that research tell us so unequivocally. But if you don't, it's better you
start admitting it to yourself - if only because it gives you better insight
into your own internal psychological processes.

Then again who am I to give unsolicited paternalizing advice to strangers.

------
anp
steveklabnik also pointed out on the Rust subreddit that there's a Dropbox
blog post covering a few more technical items:

[https://blogs.dropbox.com/tech/2016/03/magic-pocket-
infrastr...](https://blogs.dropbox.com/tech/2016/03/magic-pocket-
infrastructure/)

------
myth_drannon
Now that Golang is making headways at Dropbox, I guess Python codebase will
diminish in its importance... I wonder how Guido Van Rossum feels about it.
First he was at Google but they created Go and he left to Dropbox which was a
large Python shop, now they are moving to Go too.

~~~
james_cowling
I probably wasn't sufficiently clear in my other message re Golang:

\- Python is our primary development language for most stuff at Dropbox.

\- Mobile development and UI code use whatever is appropriate for each
platform.

\- Go is the primary language for Infrastructure, meaning fairly deep-backend
stuff: databases, storage systems, message pipelines, search indices etc.

\- Rust is used on Magic Pocket (which is within Infrastructure).

\- We still have a bit of C++ code floating around but there's not much new
development in C++ within Infrastructure. C/C++ are still used for common
libraries for mobile development however.

------
hakcermani
Its an amazing remarkable feat indeed to move all those bits live to another
location ! But seems contrary to what Netflix did (move everything to AWS ! -
[http://www.eweek.com/cloud/netflix-now-running-its-movies-
ex...](http://www.eweek.com/cloud/netflix-now-running-its-movies-exclusively-
on-aws.html)) I guess they have their specific usage and reasons ?

~~~
jamwt
Both companies control the technology that most impacts their business.

Dropbox stores quite a bit more data than Netflix. Data storage is our
business. Ergo, we control storage hardware.

On the other hand, Netflix pushes quite a bit more bandwidth than almost
everyone, including us. Ergo, Netflix tightly manages their CDN. That's where
they really focus their technical work. Their storage footprint is smaller and
less important to optimize to the extreme.

------
turingbook
I hated the writing style of Wired guys. The blog from Dropbox is much more
clear: [https://blogs.dropbox.com/tech/2016/03/magic-pocket-
infrastr...](https://blogs.dropbox.com/tech/2016/03/magic-pocket-
infrastructure/)

------
matt_wulfeck
The open-source alternatives to S3 have a long way to go, especially since AWS
S3 is so simple and reliable. I ran Openstack swift in production once upon a
time and I remember the entire system coming down because the the rsyslog
hostname did not resolve. Ouch. I'll let somebody else work out those bugs.

------
steveklabnik
A short mention of their use of Rust appears!

------
cj0
Some requests for the DropBox Diskotech tech blog series: For which datacenter
cooling system are the Diskotech storage pods designed? Hot aisle cold aisle?
How evenly can the disks inside the Diskotech be cooled? With this question I
try to ask, how good is it in keeping all drives cool? For example for the
theoretical case where all disks in the Diskotech are dissipating identical
heat, how much variation is there in drive temperature at roughly 25ºC intake
temperature? How much power is consumed for cooling (despite that it will be
fractionally low compared to what the disks will consume)? Is there any drive
idle optimization done or is the expectance that not any disk will reach its
idle (non-rotating) state ever? Is there any optimization done inside a
Diskotech to reduce the (inrush) current surge when connecting the machine to
power?

------
Spooky23
I'm a recent Dropbox Pro subscriber -- I've been a user for many years.

Really glad to see what's been going on from behind the scenes, and I'm
looking forward to seeing what the future will bring to the front end of
Dropbox.

Thanks guys!

~~~
jamwt
Awesome! Thanks for being a customer. We'll continue to work hard to make
Dropbox better and better.

------
iskander
Q: Did the Dropbox devs working on the second version of Magic Pocket
encounter any language stability issues with Rust? Did updates to the core
language ever break existing code?

~~~
jamwt
We probably "pulled forward" every two months or so. Back in early 2015,
sometimes we'd have breakages that took a bit of work to remedy. But post 1.0,
the language has been very stable.

As usual, we actually have more challenges pulling the libraries forward than
the language.

------
bane
From a business sense, this is a great move. Amazon's cloud storage offerings
are pretty expensive, even with various deduping strategies in place, Dropbox
needed both store and move in and out massive amounts of traffic. If DB had
reached the point where they could staff the extra over head of doing it
themselves, and figure out how to spread the hardware cost out well, it will
give them more pricing flexibility in the near future.

------
virtuallynathan
It looks like the servers they make use of are purchasable, part of the Dell
DSS series - the DSS7000:
[http://downloads.dell.com/manuals/common/dss%207000_dss%2075...](http://downloads.dell.com/manuals/common/dss%207000_dss%207500_owner%27s%20manual_en.pdf)

90x 3.5in disks, 2x compute nodes with 2x E5-2600v3 CPUs.

~~~
jamwt
We actually use several different servers from different vendors, and some of
them are more customized than others.

~~~
virtuallynathan
Ah, yes I spot the 78 bay Quanta one:
[http://www.qct.io/Product/Storage/Storage-Server/4U/New-
Quan...](http://www.qct.io/Product/Storage/Storage-Server/4U/New-
QuantaPlex-T21P-4U-p291c77c71c150c222)

Can't identify the 3rd one yet...

EDIT: Found it... HPE Cloudline CL5200, 80 drive bays.
[http://www.nextplatform.com/wp-
content/uploads/2016/03/hpe-c...](http://www.nextplatform.com/wp-
content/uploads/2016/03/hpe-cl5200-server.jpg)

------
the_watcher
I worked at a company that moved off AWS and the changing a tire on a moving
car is exactly how the devops team described it.

------
JackPoach
I wonder how capital intensive the move is. Isn't Dropbox overall still
unprofitable as a company?

------
bluedino
>> Transferring four petabytes of data, it turned out, took about a day.

46GB/s, is my math right?

~~~
james_cowling
We were pushing over a terabit of data transfer at peak, so 4-5PB per day.

~~~
atYevP
Yev from Backblaze here -> we'd be curious to hear how! A lot of folks are
looking for quick migration assistants for our B2 service, that's pretty fast!

~~~
james_cowling
We have a big fleet of what we call "backfill nodes" which handle the transfer
but the most difficult component of the transfer is just the network capacity.
We worked pretty closely with Amazon here since we have a long-standing
relationship and a lot of data transfer between the two companies.

~~~
nilayp
Any plans on blogging about this and open sourcing your tools? From our
vantage point, this would be hugely popular.

~~~
jamwt
We might blog about it, these particular tools probably aren't very amenable
to open sourcing. They've heavily dependent on in-house Dropbox
infrastructure.

------
photonwins
I am curious to know, with so many disks densely packed in a 4U configuration,
I am guessing there is definitely increased heat generation and not to mention
vibration. How do you handle these? Also, does it have any effect on MTBF?

~~~
jamwt
Yeah, our hardware teams put a lot of qualification time into hardware
profiles before we buy in bulk. And part of what we test is power utilization,
heat, etc, under normal software loads vs extreme loads, to make sure we don't
exceed power budget, don't spin everything up at the same time, etc. This
process can take months.

A lot of the system design is about talking closely to vendors about MTBF for
specific loads, and we use a lot of metric data to characterize our loads
accurately in those calculations. Then, of course, we measure once we're in
the field to make sure we were on target, so the next round of estimations is
better.

~~~
scurvy
It is indeed scary to watch a storage node pull 1.5-2 kW at power-on. I think
my eyes popped out the first time I watched the PDU load dance around like
that.

------
dmitrifedorov
I'm done with Wired: cancelled its subscription a couple months ago (after
many years), now they demand I login to read their web site because I use ad
blockers. Done!

------
jessegreathouse
Is there a link without the anti-adblock crap?

~~~
bartvk
You can also save it with Pocket
[http://www.getpocket.com/](http://www.getpocket.com/)

------
ldom66
Unrelated to the article but these portraits are really beautiful! Props to
the photograph and studio.

------
ccannon
I applaud your efforts at answering all these questions and providing very
detailed responses.

------
wchrisn
Can you share your development practices in rust like TDD, Automated Builds
etc

------
max_
whats the "brand new programming language" the writer meant ?

~~~
yberreby
It is Rust[1]:

> So, in the middle of this two-and-half-year project, they switched to Rust
> on the Diskotech machines. And that’s what Dropbox is now pushing into its
> data centers.

See
[https://www.reddit.com/r/rust/comments/4adabk/the_epic_story...](https://www.reddit.com/r/rust/comments/4adabk/the_epic_story_of_dropboxs_exodus_from_the_amazon/)
for more information.

[1]: [https://www.rust-lang.org/](https://www.rust-lang.org/)

------
monomaniar
Awosome

------
ignoramous
Key points:

1\. Dropbox moved from AWS to its own datacenters after 8 months of rigourous
testing. They didn't exactly build a S3 clone, but something tailored to their
needs, they named it Magic Pocket.

2\. Dropbox still uses AWS for its European customers.

3\. Dropbox hired a bunch of engineers from Facebook to build its own hardware
heavily customised for data-storage and IOPS (naturally) viz. Diskotech. Some
8 Diskotech servers can store everything that humanity has ever written down.

4\. Dropbox rewrote Magic Pocket in Golang, and then rewrote it again in Rust,
to fit on their custom built machines.

5\. No word on perf improvements, cost savings, stability, total number of
servers, amount of data stored, or how the data was moved. (Edit: Dropbox has
a blog post up: [https://blogs.dropbox.com/tech/2016/03/magic-pocket-
infrastr...](https://blogs.dropbox.com/tech/2016/03/magic-pocket-
infrastructure) )

6\. Reminds people of Zynga... They did the same, and when the business
plummeted, they went back to AWS.

7\. Not a political move (in response to AWS' WorkDocs or CloudDrive), but
purely an engineering one: Google and then Facebook succeeded by building
their own data centers.

~~~
jamwt
> Dropbox rewrote Magic Pocket in Golang, and then rewrote it again in Rust,
> to fit on their custom built machines.

Actually, full disclosure, we really just rewrote a couple of components in
Rust. Most of Magic Pocket (the distributed storage system) is still written
in golang.

> No word on perf improvements, cost savings, stability, total number of
> servers, amount of data stored, or how the data was moved.

Performance is 3-5x better at tail latencies. Cost savings is.. dramatic. I
can't be more specific there. Stability? S3 is very reliable, Magic Pocket is
very reliable. I don't know if we can claim to have exceeded anything there
yet, just because the project is so young, and S3s track record is long. But
so far so good. Size? Exabytes of raw storage. Migration? Moving the data
online was very tricky! Maybe we'll write a tech blog post at some point in
the future about the migration.

~~~
james_cowling
Yup, Dropbox Infra is mostly a Go shop. It's our primary development language
and we don't plan to switch off any time soon.

We're always about the right tool for the job tho, and there are definitely
use cases where Rust makes a lot of sense. We've been really happy with it so
far.

~~~
SeanDav
Without getting into any sort of religious war, what are typical use cases
that make Rust a better choice?

~~~
james_cowling
I think @jamwt did a pretty good job of explaining this in
[https://news.ycombinator.com/item?id=11283688](https://news.ycombinator.com/item?id=11283688).

At a high level it's really a memory thing. There are a lot of great things
about Rust but we're also mostly happy with Go's performance. Rust gave us
some really big reductions in memory consumption (-> cheaper hardware) and was
a good fit for our storage nodes where we're right down in the stack talking
to the hardware.

Most of the storage system is written in Go and the only two components
currently implemented in Rust are the code that runs on the storage boxes (we
call this the OSD - Object Storage Device) and the "volume manager" processes
which are the daemons that handle erasure coding for us and bulk data
transfers. These are big components tho.

------
planetjones
Oh dear I didn't realise Dropbox had invested all of that time and money
moving into their own data centre. From my perspective the future of Dropbox
looks bleak. Mass storage with Amazon is much cheaper [edit: from a consumer
perspective]. I know Dropbox has superior software that works (as opposed to
the poor apps by Amazon and Google) but I imagine a lot of people are like me
i.e. Store most of the stuff at the cheapest location and use Dropbox just for
docs that you want to sync on multiple devices. Total income from consumers
like me equals 0.

Dropbox also had that email app didn't they that they recently announced was
closing down - mailbox if I recall correctly.

Can anyone convince me that this move by Dropbox isn't going to end very badly
for them?

EDIT: downvoters - is HN unable to have a debate about whether this was a
smart move or not.

~~~
gshx
Not at all, cost/TB of S3 is much higher compared to what you can build
provided you know how to build it yourself.

~~~
LeifCarrotson
And provided your management costs are lower than Amazon's profit on S3.

There's an infection point somewhere at which owning your own hardware and
paying someone to acquire, build, manage, and maintain it becomes cheaper than
renting from someone who does the same thing at scale and makes a profit off
of it.

Dropbox is probably there, but many small web apps are not.

~~~
iofj
Keep in mind that that doesn't just result in more costs : it results in
limits on scaling.

Suppose you are on S3 and you can serve a customer x Mb/$. Now you're limited
to products that you can charge more for. Suppose you get data served to
customers on dedicated at x/10 Mb/$ (easily doable, and compared to S3 I bet
x/50 would be doable).

Isn't this how products like facebook video, vimeo (and thousands of porn
sites) survive ? If you calculate how much it would cost to serve 20 Mb out
from EC2, you would never ever be able to pay that using advertising revenue,
even if you got as much as TV gets.

So that reduction in price that comes from not using the cloud doesn't just go
into your pocket : it enables new business models that just aren't accessible
to you otherwise.

Also can we stop pretending that the alternative to the cloud is buying
servers from HP and colocate them ? That's simply not true. There's any number
of projects that would enable you to slowly scale up that aren't EC2. It's
more work, certainly. But it's worth it at quite small scales.

And in some ways (e.g. geographic reach) clouds simply don't match existing
dedicated offerings. Not now, and at a 20% yearly price reduction it'll take
them decades to match dedicated server rental.

Obviously I'm not claiming clouds don't have advantages. But you're paying
quite a high price for them atm.

~~~
true_religion
I'd agree with that. Let me provide some numbers.

Currently, S3 will provide bandwidth out for 300TB/month at ~$21,000.

At our web host the same bandwidth would cost $1800.

At our CDN, the same bandwidth would cost $3585 to $4791 depending on the
number of points of presence you'd require (6 vs 17).

S3 and its extreme availability profile is nice, but its not cost effective
when the difference between pricing vs the lowest-priced option is almost
$20,000 a month.

To me, the main alure of S3 was never amazon's architecture or HA, but that
you no longer had to manage your storage servers and add capacity in a
stepwise process. 7 years ago, to do this you were stuck with GlusterFS or
even MongoFS.

Today we have OpenStack, Ceph, RiakCS and others providing battle-tested open-
source solutions that anyone can run so there's much less of a reason to go
S3.

I like to compare Amazon to Akamai with the statement of "yes, they're the
best or close to it; but do we _need_ the best?"

~~~
ec109685
If you are using a CDN, then the outgoing bandwidth isn't as large a concern
for some work loads.

It is sad that bandwidth charges out are so expensive because it does make one
question their services.

------
razster
My reasons for leaving DB was due to their director of board choice. They were
great but since then I have found equal if not better services.

------
known
TL;DR Amazon Cloud is expensive;

------
tschellenbach
Why would you reinvent a piece of technology offered by various cloud
providers? It doesn't make any sense, what a waste of engineering resources
(fun exercise though). They should have been able to reach some middle ground
with AWS about pricing.

I really doubt their in-house storage backend will be cheaper compared to the
lowest rate they can get from one of the cloud providers.

~~~
johnloeber
Because having your entire business model depend on the pricing whims of
another business (which in the grand scheme of things is a competitor, btw) is
unsustainable. Getting out of AWS is exactly the right move for Dropbox's long
term health.

~~~
tschellenbach
There are many cloud hosting providers and they are competing intensely.

AWS marginal cost increase for supporting Dropbox is far below the cost of
Dropbox building and maintaining their own solution.

Given these facts they should be able to negotiate a far better price/cost
than their current solution of building it in-house.

There is nothing unique about their requirements which justify this move. It's
a really weird decision. Probably we're missing something here. This must be
part of some larger strategy they are executing.

~~~
toyg
As jamwt mentioned above, storage is simply the most critical element of their
business, so it makes sense to control it as tightly as possible , similar to
what Netflix does with their CDN.

The cloud is for stuff you don't care too much about.

