
Distributed Computing at Airbnb - AirbnbNerds
http://nerds.airbnb.com/distributed-computing-at-airbnb
======
uptown
I'm always impressed by the technical write-ups that come from Airbnb. I'm
also always astounded by the complexity of what's happening on the back-end
for what appears, on-the-front-end, to be a relatively straight-forward CRUD-
type system.

~~~
old-gregg
It is a relatively straight-forward CRUD-type system, it's the amounts of data
it deals with coupled with (I am speculating) high volume of traffic what
makes it tricky to continue to look like a simple CRUD app from the outside.

The complexity could have been reduced dramatically if they were hosting on a
modern dedicated hardware configured for their workloads. Successfully
reaching this kind of scale on AWS (or any other massive cluster of exhausted
and unreliable virtual generics with limited I/O) requires this kind of
software wizardry on the back-end.

It would have been _even more_ complicated if all of AirBnB was running on a
cluster of randomly dying first generation iPhones.

Want to avoid layers and layers of software complexity and paying for brainy
guys to run it? Pay for a nicer hardware then. Sometimes it's cheaper (and
easier), sometimes it's not.

~~~
xal
I wish I could upvote this twice. Easily one of the most insightful comments
I've seen on HN recently.

------
lightyoruichi
You need to move away from Posterous. It's closing down in less than 15 days.

~~~
AirbnbNerds
We've got something in the works.

------
jaredwilliams
I'm curious what other components you use around these services. ZooKeeper for
properties? What do your Storm spouts feed from (Kestrel, Kafka, ...), and
what do your bolts write to? Are you using something for distributed locking?

The article is a good start, but leaves a lot to the imagination!

------
Diamons
Storm Framework links to Hadoop, just an FYI.

~~~
AirbnbNerds
Thanks, fixed.

