

How Did Dropbox Scale To 175M Users? A Former Engineer Details The Early Days - harryzhang
http://techcrunch.com/2013/07/11/how-did-dropbox-scale-to-175m-users-a-former-engineer-details-the-early-days/

======
notJim
This article is horribly written, seemingly by someone with no actual
programming experience or knowledge, or even a basic command of English
grammar.

I mean, this passage reads like timecube (up until the very end where it
becomes somewhat more cogent):

    
    
      Out of this came some learning such as, in terms of app
      specific metrics such as ease and ‘fooplroofness’. Plus, 
      it emerged that “most graphs are useless”. Instead they 
      built dashboards to analyze the performance; they always
      put lefts on values (like failed log-ins etc); and they 
      kept some slack such as: extra queries were memcached, and 
      delayed optimization of SQL queries.

~~~
wikiburner
Either that or TechCrunch is closing in on the Turing Test.

------
bdon
From the comments, a link to Rajiv's technical writeup:

[http://eranki.tumblr.com/post/27076431887/scaling-lessons-
le...](http://eranki.tumblr.com/post/27076431887/scaling-lessons-learned-at-
dropbox-part-1)

which covers some of the points in the TC article.

~~~
eksith

      My only suggestion for choosing technology would be to pick 
      lightweight things that are known to work and see a lot of 
      use outside your company, or else be prepared to become the 
      “primary contributor" to the project.
    

This should be put on a header for anyone considering the sexy new thing to
come out, be it Node.js, MongoDB or the bazillion JS frameworks etc... Not
saying you should avoid these altogether; just be prepared to iron your own
wrinkles if you ever intend to become popular.

------
cldr
I think "replace the hard drive" is a terrible way to say it, since a hard
drive (or SSD or SD or whatever) is exactly what you need to be able to use
Dropbox.

~~~
pedalpete
In the long-term, I think you may look back at the 'replace the hard drive'
statement as on par with Bill Gates 'a computer in every home'.

If all of your apps save directly to Dropbox your 'hard-drive' and thumb-drive
are no longer important, and in that way they are 'replacing' the hard-drive,
even if not in technical terms, the sentiment is correct.

~~~
wslh
_In the long-term, I think you may look back at the 'replace the hard drive'
statement as on par with Bill Gates 'a computer in every home'._

That's not possible, since your local storage will always be faster than a
networked one.

~~~
Dylan16807
Always? Hard drives don't transfer at multi-gigabit speeds. Networks do.

~~~
cityguy
Err, no. The SATA 3.0 spec is 6.0 Gb/s. Most NIC's might run 1Gb/s locally,
but once you hit the WAN you're probably talking about 10 or 100Mbit.

Network storage has always been slower than local, and until we get 10Gb to
the home, will continue to be so.

~~~
Dylan16807
Hard drives can't saturate SATA. And many millions of people already have
gigabit service. That's roughly as fast as most hard drives.

------
general_failure
I don't know what the big deal is scaling up. I have worked in a couple of
"big scale" apps and it's really simple principles. At the end of the day,
it's all about caching, powerful machines, caching, normal coding skillz,
caching. And oh did I mention caching?

I honestly don't think it's that big a deal to 'scale'. Programs like early
wolf3d impress me more, much much more. And of course, things like the recent
quine relay leave me demoralized.

~~~
amix
Dropbox does about 1 billion file updates pr. day across millions and millions
of active users, petabytes of data and millions of devices. I think it's very
ignorant to think that their problem is just solved by using powerful machines
and caching...

~~~
general_failure
Ok. So can you give me a few pointers on topics to read up on how they might
scale up? Everything I have read in this field is all about - more machines,
split data into several machines aka sharing for nice map reduce queries etc.

Genuine question not snarky. I have even read yahoo's fast website tricks -
it's all really hacks and tricks, nothing mind blowing.

What I mean is given some extra time most people can figure things out by some
googling and trial and error. This is unlike wolf 3d. Unless you spend
considerable time understaning math and graphics you cannot do it.

~~~
smartwater
DropBox can't just easily Google an answer like you would with a Wordpress
blog. Their bottleneck wouldn't be web based, so how they load their
javascript or if they use image spriting is basically irrelevant.

Scaling a website with exponential growth is very difficult. Fix one problem,
two more pop up. It never ends. Oops just spent 50k on SSD's to find out it
was a load balancer issue.

~~~
general_failure
Please Bbe specific. I have never seen any real life load balancing code. We
just used elb, it was mostly configuration. Drop box uses was AFAIK.

------
betageek
From [https://www.dropbox.com/help/14/en](https://www.dropbox.com/help/14/en)
"Dropbox's performance may start to decline when you store above 300,000
files."

From personal experience replace "start to decline" with "not work at all." I
don't have much faith in Dropbox's growth if they can't fix this.

~~~
jamesaguilar
I think you are mistaken. Dropbox's growth will not be materially affected by
an edge case like this. I doubt even one in a hundred people would want to
store hundreds of thousands of files in their Dropbox.

~~~
wikiburner
If their long-term goal is to become the filesystem for all your devices, then
I think you're mistaken. However, I'm sure Dropbox will be able to resolve
this issue before it becomes more commonplace.

------
finkin1
Here's a link to the actual talk:
[http://www.ustream.tv/recorded/35654239](http://www.ustream.tv/recorded/35654239)

------
brown9-2
Can anyone explain what this means?

 _they always put lefts on values (like failed log-ins etc)_

~~~
oh_sigh
left = y axis

