
Scaling Instragram - tferris
http://speakerdeck.com/u/mikeyk/p/scaling-instagram
======
jvehent
I wish each and every single one of the open source developer who, every day,
build amazing tools to make things like instagram possible would receive a
tiny portion of that 1 billion dollars.

But that's not how the world works....

The real revolution of this century is not how many sheeps I can throw per
day, but how huge a success the open source movement is, and what incredible
companies we can build with it.

~~~
famousactress
That'd actually be a really cool pledge to make as a company. Sorta like the
1% for the planet initiative, but something similar to pledge a slice of
profits or revenue, or exit-event-cash to the OSS projects you depended on
most.

~~~
d503
A few companies _do_ actually, including DuckDuckGo and Tarsnap.

<http://fosstithe.org/>

[http://www.gabrielweinberg.com/blog/2010/11/help-me-
start-a-...](http://www.gabrielweinberg.com/blog/2010/11/help-me-start-a-foss-
tithing-movement.html)

~~~
pasbesoin
I recall the story of Yahoo giving Larry Wall enough pre-IPO stock that they
effectively paid his daughter's way through college.

Umm...

<http://www.salon.com/1998/10/13/feature_269/>

------
marcusf
Slide 54 is insanely important. At my current place we've invested tons in
instrumentation and metrics. Every IO operation is guarded with a semaphore
and access is monitored and exposed. Every cache in the system logs everything
from hit and miss rate to volume etc. Every async operation logs operation
trails. We also ship a ton of tools for everything from bread and butter stuff
like stack dumping, pinpointing long running threads etc. to more
sophisticated stuff like being able to break in to a page rendering and
profiling various element hierarchies, either over time or following specific
users.

As I said, we've invested so much in getting the entire stack monitored very
granularly, but every time our customers run in to issues it pays dividends.

For reference, we run on the JVM which is very pleasant. A ton of tools
interop with it and basic things like MBeans being integrated in to JVM makes
monitoring a lot easier.

~~~
SanderMak
How much overhead does this setup incur?

~~~
marcusf
Run time or development time? Runtime is mostly negligible except for places
with heavy congestion (caches + rendering) where you can turn the heaviest
operations on and off as needed -- in our case the call to
`System.getCurrentTimeMillis()` is usually too expensive to do unless we
really need it.

In terms of development, it's not a lot of work continuously, but it took some
while to write up, test and benchmark standard libraries for eg monitorable
semaphores. In general though, what it does is raise the entry threshold for
new developers a bit as you need to have a feel for when something requires
monitoring and when it doesn't.

------
jaredsohn
This talk was given at AirBnB in SF. Talks there are fairly new and typically
occur every other Wednesday. There is a talk tonight by Yehuda Katz and
upcoming talks are posted here: <http://www.airbnb.com/tech_talks>

------
rabidsnail
Audio: <http://soundcloud.com/bob-poekert/tech-talk-mike-krieger-of>

------
jaredsohn
Submission for a TechCrunch writeup of the talk, which includes some quotes
and also embeds the same slides (via scribd):

<http://news.ycombinator.com/item?id=3831865>

------
apu
On slides 61-69, it seems to suggest that they were storing their photos
directly in the database. Is this correct? It seems rather inefficient to do
this, rather than storing the images in the filesystem.

~~~
seanmccann
No, photos are stored on S3. Only photo meta data is stored in the db.

------
zdgman
Wasn't this same article up on HN the same day they got acquired? Lot's of
interesting stuff in here in general, especially when they get to talking
about when their app actually uploads a photo.

------
tferris
Instragram is the modern fairy tale showing that any "2 product guys" "with no
real backend experience" are able to rise from rags to riches.

~~~
DavidAbrams
And also that the bubble is back, big time.

~~~
ma2rten
I am starting to get a little bit tired of people screaming "bubble" whenever
someone mentions Intragram. Could you make a good case for it?

If one particular person (Zuckerberg) values one particular company higher
then most other people, I don't see that as a sign for a bubble.

~~~
Produce
It's a sign of... wait, strike that. It's a huge red flag the size of the moon
when a company with no revenue
([http://www.npr.org/2012/04/10/150372288/instagram-sells-
for-...](http://www.npr.org/2012/04/10/150372288/instagram-sells-
for-1-billion-despite-no-revenue)) is sold for so much money.

I would argue that Facebook and Google are bubbles too since they do not
provide value. Advertising is a joke of an industry - it produces nothing and
is essentially the business of mass psychological manipulation. For proof that
such bubbles burst, look towards to entertainment industry which also provides
nothing of value yet charges through the nose for it.

~~~
mseebach
> I would argue that Facebook and Google are bubbles too since they do not
> provide value

But you'd be wrong. Facebook and Google create enormous amounts of value.
Enough to support their high valuations? Only history will show.

> Advertising is a joke of an industry - it produces nothing

Please don't confuse your personal opinion of the advertising industry with
100+ years of empirical evidence.

> look towards to entertainment industry which also provides nothing of value
> yet charges through the nose for it

"Entertainment" is doing fine. The current industry establishment is not. Not
the same thing.

~~~
Produce
>Please don't confuse your personal opinion of the advertising industry with
100+ years of empirical evidence.

It really depends on what your definitions are. My personal take on it is that
advertising is duct tape for a broken economic model. If we have an entire
economic system where one of it's pillars is something so asinine
(psychological manipulation with no opt-out for certain flavors like
billboards), perhaps we should rethink the model.

So, from the perspective of a business which is trying to find
clients/customers, sure, there's a tonne of value. From the perspective of
trying to move humanity to bigger and better things, it has a huge negative
value, again, IMO.

>"Entertainment" is doing fine. The current industry establishment is not. Not
the same thing.

I actually wrote "entertainment industry" in my post :)

~~~
mseebach
> So, from the perspective of a business which is trying to find
> clients/customers, sure, there's a tonne of value.

Excellent. And I'm sure you'll agree that's something of a leap from "it
produces nothing"? Also, there's a corollary to this: This enables competition
by lowering barriers to entry, which massively benefits consumers.

~~~
Produce
>And I'm sure you'll agree that's something of a leap from "it produces
nothing"?

A diamond is worth a great deal in a lot of peoples' eyes yet it produces
nothing. Interestingly, the goal of advertising and marketing is to increase
the value of actual things by tricking people into believing that a) there's
not enough supply - buy it now! and b) that demand is too high - buy it now!
What exactly has been produced here? Demand? That's not a thing, it's an idea.
Intellectual "property" is an absurd term, therefore, nothing has been
produced.

On the subject of why I think that it has no value - well, most businesses see
no value in me scratching my hand because it has an itch. From my perspective,
there is a great deal of value in it. Similarly, I see no value in what some
businesses do, since, from my perspective, they are harming the people around
them. There is no absolute value in anything. Therefore, the value of
something is determined by the beholder, which in my case is me.

>This enables competition by lowering barriers to entry, which massively
benefits consumers.

Assuming that a competitive economy is a good thing, ofcourse.

~~~
mseebach
> Assuming that a competitive economy is a good thing, ofcourse.

Yes, indeed assuming that. Good day.

~~~
Produce
<http://www.charleswarner.us/articles/competit.htm>

[https://docs.google.com/viewer?a=v&q=cache:yWqCGDpfGLwJ:...](https://docs.google.com/viewer?a=v&q=cache:yWqCGDpfGLwJ:polis.unipmn.it/pubbl/RePEc/uca/ucapdv/ortona7.pdf+&hl=en&gl=uk&pid=bl&srcid=ADGEEShinQYNrk5z1nLumzZ1dwyPeYRnFG-
odfG5U-4MEWN6KRwjVWRvB6cADKfRwXZ6vg4E5Wy0wPi5Ib-
GmthlzoaNw_dDztTwpqM_iknBjmdGinSFnUrBHL0KV_boaCmPRjrrkmHO&sig=AHIEtbSqpxJauCTyGNr7tSKPlqHhsXCF9w&pli=1)

Google for more if you care to question your assumptions.

~~~
mseebach
The first is a paper by a psychologist and motivational speaker. Is that
really what you want to lead with?

The second paper I can't really figure out, but it's dealing with individuals
in a closed environment - not with matching supply to demand in an open
economy.

Anyway, I'm plenty open to questioning my assumptions. That doesn't mean I'm
willing to invest in lengthy discussions with random strangers in off-topic
threads.

------
rplnt
Is there a video/audio for this talk?

------
unwind
Won't somebody please think of the nit-pickers, and fix the typo in the title?
At least it's only in the submission to HN.

