
Once Again, Twitter Drops Ruby for Java - alexwilliams
http://www.readwriteweb.com/cloud/2011/04/twitter-drops-ruby-for-java.php?utm_source=ReadWriteCloud&utm_medium=rwchomepage&utm_campaign=ReadWriteCloud_posts&utm_content=Once%20Again,%20Twitter%20Drops%20Ruby%20for%20Java
======
fleitz
"That along with the news that Twitter has hired 25 more employees kinda tells
that Java's code base is practically more maintainable than equivalent Ruby
code"

Uhh if you need 25 more people that tells me that your code is less
maintainable and you're writing much more of it. No one has ever debated that
in raw execution speed Java is faster, the problem is that most startups have
plenty of CPU and few devs.

Twitter really did things ideally, start on a rapid dev platform and then port
to a faster platform after acquiring the userbase and capital infrastructure
that makes such a decision make sense. I'm not sure that if I was working on
the JVM that Java would be my language choice, I'd probably use
Mirah/Clojure/Scala.

The extra 25 devs are probably to ensure T2EE (Twitter 2 Enterprise Edition)
compliance and refactor their POROs (Plain old ruby objects) to work with
their ETB middleware. /snark

~~~
bballant
I also thought that was a weird comment. But In my experience a big team
benefits from a compiled, statically typed language like Java (better yet,
Scala). Perhaps that's what the author meant.

~~~
rbanffy
> But In my experience a big team benefits from a compiled, statically typed
> language like Java

My experiences with Java have never been very impressive. Why do you think a
bigger team benefits from static typing?

~~~
bballant
I've worked in ruby, python, java, and most recently, scala -- on both small
and large projects. Tangled hairballs and hidden runtime exceptions can happen
anywhere. I've just seen less of it with Java. I think this is partly because
the compiler catches things that one would need to write tests to catch in
ruby. For me the compiler is another layer of defense keeping bad code out of
production.

Another advantage of statically typed OOP code for big teams is it allows a
verbose yet formal way to define interfaces as a team, and then break up the
work into smaller chunks. The formality can certainly slow an individual
programmer down, but in a big team I've found it makes breaking up the work
easier.

I wouldn't argue against the notion that ruby's expressiveness and use of
functional paradigms might make up for it's lack of static typing, and ruby in
the hands of a great programmer is pure pleasure, but most teams don't have
just great programmers.

Finally, I absolutely love Scala and I think it's worth mentioning in any
discussion of ruby and java. It, sorta, bridges the gap between the two
languages and I'd recommend it as a good choice for a team of any size.

~~~
justwrote
I also love Scala, but how do you do your web frontend? Using Lift? Play
Framework? I'm currently developing a side project with Lift and lets say I
like the Scala-part of Lift.... The web part doesn't feel nice (also I don't
like the last decisions made by the Lift Team). Currently I'm looking into RoR
and I like what I see.

So, what I would love to see is a Web Framework similar to RoR written in
Scala ...

~~~
bballant
I've heard good things about Play, but we've chosen to go with Spring MVC
because we didn't need the full stack -- we have a lot of existing Java ORM-
ish code and a bunch of existing JSP functionality which we wanted to re-use.
JSP is actually our biggest pain point at the moment (aside from generally
suckiness, it doesn't work with scala collections), so we're looking into
using SSP via Scalate, but we've yet to make that move.

Aside from JSP, we're really happy w/ Spring MVC. We've managed to avoid the
crazy amounts of XML that Spring is known for in favor of its more modern
annotation-based way of doing things.

------
theoj
Once you increase the scale sufficiently, it's more cost effective to hire a
team that makes the code run faster rather than add large expensive servers.
This is what's happening now at Twitter.

It's not that Java is now absolutely 100% better than Ruby on Rails, it's just
that Java is better for a site of that scale.

~~~
acangiano
Yep. Ruby on Rails: suitable for apps with less than 1 billion queries per
day.

~~~
theoj
It's more than just about queries. Ruby on Rails is an absolute pig in terms
of memory used. That's something you find out pretty quickly if you build your
own VPS setup.

~~~
aphexairlines
Going to the JVM because of memory usage issues is like going to KFC to lose
weight.

~~~
bballant
On my team such remarks cost a dollar. The JVM generally does very well in
terms of memory use compared to many languages. It's easy to test: write an a
small program in both languages that creates boatloads of objects and then
watch the memory.

The JVM will use up all the memory it can (the amt is configurable with some
runtime flags) before the GC kicks in and frees up space. It uses a couple of
buckets for different types of objects, and can easily clean small short-lived
objects with a bit of a CPU hit, but without affecting the performance of the
application. The upshot is you'll see Java's mem use slowly creep up, you'll
see a small spike in CPU, then the mem use will drop. If you don't see this,
it means the code is poorly written and its keeping around object references.

Ruby's GC is much less efficient. It has no concept of memory buckets like the
JVM and will effectively stop execution and traverse every object in the heap
twice, marking and then freeing up space.

Early versions of Java (pre 5) had some substantial problems with memory
management, but it's been very performant for quite some time and easily out-
paces ruby.

~~~
aphexairlines
There are no benchmarks showing JVM-based programs using less memory than most
any other roughly equivalent programs on different runtimes. The JVM's
reputation as a pig also comes from people's personal experiences -- even a
simple email proxy (davmail) is easily at 100mb resident, over 1gb virtual.

~~~
bballant
My main point is that the JVM manages memory better than ruby. Its dead easy
to benchmark it yourself by writing a pair of tiny programs. As I said, the
JVM will use up all the memory it can. You can probably tweak davmail's
runtime flags to use less. There is typically a CPU tradeoff if you cut back
on the memory a program can use, and usually a happy middle-ground.

In my personal experience, I've re-written a few ruby utils in Java for an
order of magnitude speed improvement. I've seen a ruby site crater under load
because of memory issues that its java replacement easily manages (which is
sorta what the OP is about). The JVM is especially well suited for handling
"stateless" http requests because it's ability to quickly create and destroy
small objects.

------
dasil003
I shudder to think of CTOs basing high-level engineering decisions based on
such content-less fluff.

~~~
lallysingh
Indeed. I'ts already turning into a "my language is more scalable than yours"
contest, here on _HN_. If it was normal posturing and trash-talk, that'd be
fine. But _engineers_ are going to _suffer_ for this --- their well-thought-
out arguments for one platform over the other are gonna wash to _big_ _site_
chose A over B. We will use A.

------
laujen
Twitter is beyond the scale of most businesses. This -- whether Scala or Java
or PhP or RonR or Python -- is the best choice for our code base generating
millions of hits per day is a problem many of us would love to have!

~~~
SpiralLab
Agreed. Build for the problems you face now - not the problems you may
_theoretically_ have in the future.

If a business grows that rapidly - it is most likely that the original systems
will have evolved dramatically from day one anyway - in ways you wouldn't have
imagined.

------
guelo
I have no insight into Twitter's decision making, but the reason I recommend
Java over the other popular languages comes down to readability and
maintainability. In my experience the dynamic-ness of the language begins to
work against maintainability and readability after a while, especially after
the original designers hand off the project to other programmers. Most
language communities have built over time best-practice guidelines for
readability and avoiding common bugs. But I think Java's age and enterprisy
culture have allowed it to evolve stronger guidelines (see Joshua Bloch's
Effective Java), and it's limited flexibility make it easier to maintain.

If you are in a highly competitive startup environment where break-neck
execution for the next 6 months to a year is a matter of survival then Java
might not make sense, but that doesn't describe most projects.

~~~
Sapient
I don't believe Ruby and Rails are prescribed for _most_ projects. Just the
ones where they makes sense.

That said, I have taken over development of a few legacy rails applications
and never had a problem adapting to the idiosyncrasies of the old developer or
team.

~~~
eropple
I have no idea why haploid's comment went dead - he raises a very good point.
RoR, for better or worse _, has become the trendy tool to use for web apps.
Java, once upon a time, did occupy that niche, too.

_ \- worse, IMO

~~~
Sapient
Java programmers commenting on this story seem to be claiming that Ruby is a
messy language and hard to grow a team around, while Java somehow solves all
these problems, yet I see only anecdotal evidence and opinion to back this
claim up.

One thing I cant remember ever having read is a horror story about people
taking over a legacy Rails app, while those stories seem to be fairly common
when talking about Java apps.

Seems to me that, for the most part, people who know Ruby and Rails have
taught themselves, while sub-par and mediocre Java programmers are produced in
their hundreds, fresh from their first programming experience in college.

Seems logical to me then that its going to be far easier to get experienced
Ruby programmers than to get experienced Java programmers.

~~~
eropple
Horror stories about legacy Rails apps? I have a couple, though not "taking
over" them but rather fleeing from them. (My employer is in the process of
replacing a home-built Rails-based code review tool with Review Board, and
I've looked at that tool's code - it couldn't be rehabilitated if you tried.
To be fair, it was a weekend project that spiraled out of control...but that
in itself is saying something.)

Bad code can be written in any language or on any framework. I would suggest,
however, that when at larger scales, there are significant benefits to a more
restrictive language and tool set. I write Java at work, and while there are
cases where it's not as expressive as I'd like (s'why I write my own code in
C#, which _is_ as expressive as I'd like), reducing the ability of developers
to Do Stupid Things is, to me, a very large plus.

The claim that people who know Ruby and Rails have taught themselves--well, no
shit, right? I taught myself Java, too. And C#. And PHP, Python, Ruby, and
half a dozen other languages. Of _course_ sub-par and mediocre Java developers
are churned out by college. So what? Hire ones who aren't. And if you give it
ten years (assuming the Rails fad has legs--I'm skeptical, but just for
argument's sake let's assume it's not) colleges will be churning out mediocre
Ruby hacks just as well.

Your comment reads as fanboyism and not much more.

------
grandalf
This isn't a knock against Ruby. Java and Ruby are designed to solve different
sorts of problems (though there is a lot of overlap). Twitter is not in the
overlapping region.

~~~
daemin
Additionally it makes sense to me to try building something as experimental as
twitter was back a few years ago in a language and framework that allows quick
iterating and rapid prototyping, such as Ruby on Rails. Now that the design
has largely been solved (shall we say), the application can be migrated to a
different architecture and programming environment. One that is more suited to
the large scale, low latency requirements, and minimal functional changes that
twitter finds themselves in now.

It's also funny to think but Java used to be the slow language on the block
for building websites and services in. Then a lot of people ended up using it,
and a lot of effort was put into the JVM, especially the hotspot compiler, the
garbage collector, etc, and now it's one of the best technologies out there
for doing large scale websites with.

I think if more people use Ruby and Rails, that we'll get more investment in
the platform and it will end up being significantly faster in the future.

------
noelwelsh
Twitter is well known for using Scala. Although the article says Java, I'd
like to know if this is the case (i.e. no Scala) or a simplification.

I've done more Java programming than a man ought, and now a fair bit of Scala.
The JVM is fast -- far faster than the Ruby/Python/PHP VMs. Harnessing that
power has been an issue, as Java is so inexpressive. I find with Scala I can
write code as quickly as in any other language I've used. It makes a great
combination -- code quickly and get quick code -- and as a result I don't see
any use for RoR or other interpreted frameworks in my coding.

------
Groxx
3x faster for the pain of Java? I'll stick with Rails until I have enough
money to pay people to migrate for me.

edit:

> _That along with the news that Twitter has hired 25 more employees kinda
> tells that Java's code base is practically more maintainable than equivalent
> Ruby code - at least when the code base is huge and the team size is large._

What? How does hiring more employees imply something is _easier_?

------
plinkplonk
Interesting why they chose Java instead of Scala for this move. The last time
they moved off Ruby they moved to Scala, not Java. Anyone at Twitter want to
comment?

~~~
cgopalan
From the original engineering article on the twitter site, it seems they
switched from a database backend (MySQL) to using Lucene and also heavily
customized it. Maybe it was just a lesser hassle to interact with Lucene using
Java. Would love to hear reasons directly from them.

------
ChrisArchitect
gah, this again eh. Friday's news. This is a pretty unique/upper tier end of
the spectrum problem. They needed to be able to cope with the ridiculous
amount of realtime indexing and searching going on around the SEARCH stack, so
ok, Netty etc. Not really anything about abandoning Rails etc.

------
sgt
When they say they are changing to Java, does this mean Scala, or do they
actually mean the Java programming language running on the Java platform?

~~~
bxr
Between their development blog and github account, they appear to use both
significantly. I'd like to know more about the proportions, but haven't seen
any numbers about it online.

------
gulbrandr
Please do not post links with utm junk in the URL.

