
Twitter trouble - handling 11,000 requests per second w/ Rails! - joshwa
http://www.loudthinking.com/arc/000608.html
======
felipe
I totally agree with the concept that speed does not really matter because
hardware is cheaper than programmers. And I never understood the criticism
towards Ruby being slow... If it improves my productivity, who cares?

However, the REAL problem I see here is that apparently the very features that
make Rails productive must be thrown away for the application to scale. If
that's the case, then that's a _huge_ issue and even a show-stopper if you ask
me.

I think the problem lies on how clustering is usually implemented in Rails.
I'm not an expert, but my understanding is that Mongrel distributes the load
across multiple web servers, right? If that's the case, then the bottleneck
becomes the database, and that's the key issue IMHO (and the hardest one to
solve).

In Java (more specifically JBoss), clustering is implemented at the
application logic level: The app server manages the state of the application
and then it caches and replicates the state whenever necessary.

I really don't see how Rails will solve this problem unless with some kind of
application-level communication across the servers... And that's a tough one.
Suffice to say that the Java community took like 10 years to come up with
something effective!

Note: Please don't take me wrong: I'm not trying to start a Rails vs. Java war
here! I'm just sharing my experience on how Java solved this issue...

~~~
bootload
_'... I really don't see how Rails will solve this problem unless with some
kind of application-level communication across the servers ...'_

This is something that appears to popping up again and again in Web2 [0]. Is
Twitter utilising 'memcached' & 'MogileFS' [1] in their optimisations trying
to reduce the load on MySql?

Reference

[0] Myspace thread ~ <http://news.ycombinator.com/comments?id=1015>

[1] Distributed file storage: MogileFS ~
<http://news.ycombinator.com/edit?id=12345>

------
brett
To me this reads as relatively diplomatic for DHH. The interview in question
is pretty critical and while DHH seems to suggest that scaling like this is
going to suck rails or otherwise he does not address all the criticisms head
on and his tone is slightly conciliatory. I guess there's not a lot he can
say; arguing performance in the abstract is one thing, but if twitter is
saying ruby and rails are slow(er) there's something to that.

It still seems like a good trade off if rails makes you more productive and
you've got to get to twitter's size to feel it.

~~~
aston
There's only so much to that productivity argument. I suspect the time it
takes to make big changes to the Rails stack for performance sake far exceeds
the relatively small savings in initial development time. According to
Compete, Twitter's not _that_ big [0].

[0]<http://snapshot.compete.com/twitter.com+techcrunch.com?metric=uv>

~~~
brett
Yeah. It's a give and take to be sure. I'd take Twitter's current position any
day. But you're right they're still in a position to stay ahead of competition
and net productivity of development and scaling has to be considered.

Worth noting Twitter's audience is using it a lot differently than
Techcrunch's is, so relative audience size is not totally meaningful.

