

Twitter Said To Be Abandoning Ruby on Rails - jsjenkins168
http://www.techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby-on-rails/

======
jsjenkins168
If true, this is a pretty big blow to the viability of RoR as a large-scale
platform, at least as it is right now.

I've always thought RoR's tight coupling with traditional SQL would be a
serious scaling issue. RDBMSs in the traditional sense just dont scale easily.
Google figured this out a while ago (hence BigTable). And RoR reads and writes
soo much that you are left with a burden on the DB that might have been better
mitigated by doing more of the processing client side and persisting with a
simplified cloud-based service rather than a single, central RDBMS.

Not to stir up a hornets nest, but I really think this is a limitation of RoR
that should be addressed in the future. Abstract out the SQL so other methods
of persistence are possible.

 _EDIT: I am not trolling RoR. I am simply proposing that it might be better
to be able to use it with something other than traditional SQL for persistence
of data, so that traditional RDBMS can be replaced with a simplified cloud-
based service such as SimpleDB.

I speculate this might be possible now using Ruby -> JRuby -> JPA -> SimpleJPA
(or any other MySQL/SimpleDB wrapper) -> SimpleDB. I dont want to even get
into how much work that would be though. It needs to be easier I think.

Startup idea maybe?_

~~~
menloparkbum
Rails, and the alternatives mentioned don't really make sense for a messaging
application. You wouldn't write the back end for Gchat or AIM in Rails (or
PHP, or Django), either. Rails would be fine as the web front end that talked
to another messaging system. I've interviewed with the twitter guys and been
to their talks and they definitely are NOT doing the right thing. However,
"the right thing" is not easy and there are few people around and available
who can do it.

edit: their best bet is to try to find someone who is burned out from working
on wall street applications.

~~~
aneesh
Just curious, what would you write the backend for AIM in? I don't have any
experience developing messaging-type apps.

~~~
yan
Erlang is a fine option. So is C.

~~~
mosburger
Dang - I was going to say Erlang. This is an ideal application for Erlang...
scalability and uptime. There's a reason it's heritage is in the teleco
industry.

~~~
davidw
I went and had a look at the slides for their 'scaling twitter' presentation,
and it looks like they had a look at Erlang and reacted with some silly "ooh,
it looks ugly" sort of comment. Seems pretty silly to reject something out of
hand that is very much along the lines of what you need.

------
frankus
Twitter always struck me as a little lower on the complexity scale and a
little higher on the throughput scale than where RoR's sweet spot lies.

Since their requirements seem like they're pretty static now, I would think
writing a backend in C wouldn't be out of the question.

~~~
icey
The question is, do they have the skillset in-house for that sort of
undertaking?

~~~
systems
skillset is a really good word

i am really surprised how facebook, so quikly found the skillset to build a
10,000 server site and how come, twitter didnt! just why ...

its the skillset, but again why was facebook successful in aquiring the
skillset when twitter wasnt! ... why is twitter considering a rewrite and
facebook didnt ... they just grew !!

~~~
axod
Having 10,000 servers doesn't say 'skillful' to me.

~~~
bluelu
Facebook's system is also 1000x times more complicated than twitter's simple
message passing system!

------
bjclark
Yeah for people having never heard of Erlang.

When I wonder why Facebook doesn't have problem scaling their chat? Oh, erlang
+ flat files.

I can't wait for Twitter to move to Java or PHP so people finally STFU about
Rails.

Twitter moving away from rails doens't make rails look bad, it makes everyone
at Twitter look like the biggest jackasses in history.

"We can't get this wrench to hit this nail hard enough, so lets buy another
brand of wrench instead of a hammer and see if that works."

~~~
aschobel
Do you have any more info on Facebook's chat architecture or were you just
speculating?

A quick Google of "erlang facebook chat flat file" returns your initial post,
would love to read more about that if have anything.

~~~
bjclark
My knowledge of Facebook's chat architecture comes from drunken conversations
with Facebook engineers at bars. So no, I don't _think_ I know any more than
that. _hiccup_

------
andr
ev: FWIW: Twitter currently has no plans to abandon RoR. Lots of our code is
not in RoR, already, though. Maybe that's why people are confused.

<http://twitter.com/ev/statuses/801530348>

(Ironically, Twitter was down when I first tried to get the URL for this.)

~~~
handelaar
Currently: 500 Internal Server Error.

(With an HTTP Status header being dumped to the screen instead of, you know,
sent as a header.)

~~~
Hexstream
Do you obsessively check the headers for proper status codes whenever you land
on a page? I'm always amazed how people find out header problems like this
just by browsing.

~~~
eggnet
He probably has firebug.

------
henning
Michael Arrington loves taking things out of context.

I hate it when non-programmers write about programming! It's a giant festival
of stupid.

~~~
edw519
"I hate it when non-programmers write about programming!"

It doesn't matter what he says. All that matters is that you notice.

After 30 minutes, you are the #1 response to the #2 thread.

So people click through.

Cha-ching.

~~~
ashu
True journalism, indeed.

~~~
alaskamiller
Michael never claimed to be a journalist. You're likely mixing up the
professions because you think TechCrunch has gotten so big that it has and
cares about its reputation. That wouldn't be the case, he just writes whatever
he can to continue to generate pageviews.

------
RecycleBin
I'm embarrassed by a lot of comments here. I thought people on this forum
would know better. It's like I'm watching a car, with smoke and flames pouring
out from under the hood, pull over to the side of the road and people running
up and checking if maybe it needs new tires.

You could probably rewrite the front end in applescript and you'd see little
difference, and if you did just throw more servers at it. The problem is
probably that theres a DB at the heart of all the message crazyness, you can't
do that and expect to scale.

~~~
Jaggu
I am Ruby learner. The way ruby works with DB is problem for novice. If
programmer don't do careful coding for caching then it is bound to cause
problem. Ruby hype is all around auto code generation and simple
coding/scripting. But when you are writing enterprise level apps then there is
more to consider.

------
Xichekolas
Totally amused he threw Zed Shaw in there. From what I remember, Zed trashed
several members of the rails core team, not the framework itself or even the
entire community.

Surely they are just rewriting the slower core areas of the API in something
faster/more distributed. There is no reason they'd have to rewrite the entire
thing from scratch.

~~~
cstejerean
Zed certainly did trash the community. Take a look at

"This is exactly what makes Rails a ghetto. A bunch of half-trained former PHP
morons who never bother to sit down and really learn the computer science they
were too good to study in college. [...]

After this, I started thinking, since it was the first realization that I
picked the absolute last wrong community to make my living in. They were all
pricks, morons, assholes, and arrogant fucks who didn’t care about the art or
the craft.

They were all a bunch of little ghetto fabulous princesses trying to make it
in this tiny little Rails world and not enough brains between them all to make
it happen."

and about the framework

"Now, DHH tells me that he’s got 400 restarts a mother fucking day. That’s 1
restart about ever 4 minutes bitches. These restarts went away after I exposed
bugs in the GC and Threads which Mentalguy fixed with fastthread (like a
Ninja, Mentalguy is awesome).

If anyone had known Rails was that unstable they would have laughed in his
face. Think about it further, this means that the creator of Rails in his
flagship products could not keep them running for longer than 4 minutes on
average."

Although he does add that

" [...] Rails is a pretty nice idea, and it demolished the Java world I hated
so much."

------
subwindow
It shouldn't be an all-or-nothing situation. There are obviously parts of
Twitter that would work best as non-Rails. Ruby is too slow for certain
things, and so you determine which of those certain things are in your
application and you build them in C or Java.

For Twitter, I think some parts of the API need to be in C. But the whole
service? I doubt it.

------
anotherjesse
Twitter has components in several languages (even erlang for xmpp)

They say most of their traffic is API requests, and I would expect that they
would want to optimize that porition of their system. Rails is composed of
many components, many of which you don't need for a REST service that talks
json/xml. Blaine and other twitters have talked about their experimentments:
other app servers (Thin), queueing systems (before building starling), ...

------
swombat
I seriously doubt they are actually going to rewrite Twitter in another
language. That's a pretty dumb statement and prediction, and Michael Arrington
just lost another notch of respect from me (he lost the previous one for his
vicious attacks on that Blaine dude who archtected Twitter).

Sounds to me like Mr Arrington has an axe to grind and is putting that ahead
of objective journalism.

Daniel

~~~
axod
Why not? It'd probably take a week or so to rewrite in a different language,
similar to Reddit when they rewrote in python :/

Has ror become a cult or something? It's just a language, and if it doesn't
fit well with the application, best to move on and try something else.

~~~
jrockway
Rails is probably not their problem. The way twitter works is the problem. (In
other words, they need a better backend, not a better frontend.)

~~~
axod
I was under the impression rails was the backend and was tightly coupled to a
db. Is that not the case? :/ By 'better backend', what do you mean?

~~~
jrockway
I assume the difficulty is storing the data and then retrieving the right data
when generating a page. Yeah, you can use a SQL database, but that doesn't
seem to be working very well. So they probably need to write some custom
storage backend that is actually designed for the load that is put on it.
Relational databases are general solutions. General solutions are good for
general problems, but not necessarily specific problems.

Anyway, if the problem happens to be that the Rails frontend is too slow,
speeding that up isn't going to buy them anything long-term. Eventually the
database is going to bog them down even if the web GUI is hand-optimized
assembly.

JMHO, obviously I have no idea what the twitter codebase looks like or what
specific problems they are having. I also don't use Ruby or Rails, but I doubt
that's their problem.

------
wave
Regardless of the reason, if something doesn’t work for you, you need to
identify the root cause of the problem and don’t be too arrogant to move
forward even if it questions your core beliefs

------
peterb
A bike shed (any colour will do) on greener grass...

------
manvsmachine
not true: <http://twitter.com/ev/statuses/801530348>

------
andr
+1 for PHP + memcache + Thrift.

------
simianstyle
Arrington is a bag of douche.

------
jrockway
People still consider PHP and Java for rewrites? Usually that's what you
rewrite _out of_ , not in to.

~~~
rob
Stop. There's nothing wrong with PHP, and it's proven to scale.

~~~
jrockway
Well, yeah. What would prevent a _language_ from scaling?

The reason you'd ditch Java or PHP is because they are lacking a lot of
higher-order constructs. (PHP epsecially. Statically-scoped variables? First
class functions? An object system that isn't a hilarious joke? Unicode
support? PHP doesn't have 'em.)

