

Designing a Scalable Twitter - twampss
http://natishalom.typepad.com/nati_shaloms_blog/2009/04/writing-your-own-scalable-twitter.html

======
zepolen
I don't understand why he puts numbers on the scaling requirements. Either it
scales or it doesn't.

Otherwise a good read.

~~~
FooBarWidget
It either scales or it doesn't? You really underestimate the amount of effort
needed to scale even the simplest website to millions of users. Even if your
web app is written in OMG-uberfast x86 assembly, I can guarantee you that
you'll have trouble scaling to Google-size traffic.

There are different levels of scaling. The more users you have, the harder it
becomes. But it's generally not worth scaling beyond what you actually need
because if you don't need it, then you've wasted tons and tons of man power
that could have been invested in other areas.

~~~
zepolen
Scalable, by definition, means more nodes = more capacity.

Speed has nothing to do with it.

A simple stateless website that requires no shared data like from a db, is
scalable, since you can add more web servers to fulfill more requests.

Using memcached, however, isn't scaling, because while memcached itself does
scale, if the backend it reads from doesn't, the whole system doesn't.

~~~
FooBarWidget
Even so, there's a limit to how many nodes you can add given an architecture.
Scale from 1 concurrent user to 1000? No problem, just add a server. Scale to
100000? Add many servers, and load balancers, maybe master-slave replication
for database. Scale to 10 million? Putting more boxes in your data center help
you anymore because your data center is full, you don't have enough bandwidth
and even the master-slave database setup you had overloaded. You need to setup
multiple geographically distinct data centers in the world and shard your
database. What if you have 100 billion users? Adding more data centers on the
same planet won't help anymore; I don't even know whether anybody is
technically capable of serving 100 billion concurrent users.

Sure, the latter architecture can serve 1 concurrent user, but it's also
100000 times more expensive. The answer is not as simple as "it scales or it
does not".

~~~
zepolen
Fair enough, I just wish articles would stop using the word 'scalable' to
describe everything.

------
ajkirwin
I think Twitter is a bad usage case, because it's not something that NEEDS to
be scalable.

