

Twitter, Ruby on Rails, Scala and people who don’t RTFA - mileszs
http://ikaisays.com/2009/04/02/twitter-ruby-on-rails-scala-and-people-who-dont-rtfa/

======
wheels
I think one thing especially for people coming to web development from other
angles in programming is getting used to the fact that you have to know so
many technologies / languages to get the job done.

In the early days of web apps, you could mostly get by with Perl, SQL, HTML
and a little bit of Javascript for mouseovers and whatnot. Framework? CGI.pm,
baby.

Now you generally need HTML, CSS, Javascript, a Javascript framework, a
backend language of choice, a framework of choice in that language, a high-
performance backend language of choice for the critical bits, and SQL.

I did web apps from 1997 to 2001 and then picked back up in early 2008. It was
pretty dizzying at first just knowing where to start. In the systems /
multimedia world where I lived in the 7 years between it was C or C++ and a
framework.

So what's the connection to the story? I think it's in the web world, perhaps
as a result of its relative infancy, we apparently have to get used to the
fact that the right solution to a problem is often using mixed-mode
programming where individual tools are selected for individual tasks -- that
development abilities in the web world is as much a question of breadth as
depth. We shouldn't be so disenfranchised that our tools aren't general
purpose.

~~~
jballanc
I'd take this one step further to point out that web programming is a
different beast from desktop programing in that you are attempting to tie a
fault-tolerant, low-bandwidth channel (the internet) to a strict, high-
bandwidth channel (the CPU/RAM). Add to that impedance mismatches from
latency, and I think anyone who expects to have a full stack written using
only one language is kidding themselves!

------
pc
_This power does not come without a price. Ruby is slow. Ruby is generally
recognized as being twice as slow as Python, and at least an order of
magnitude slower than C++ or Java._

The last two statements don't mean the first is true. Ruby's power is _not_
the reason it's slow. Smalltalk VMs are proof by existence that the Ruby model
can be fast. (See: Self, Strongtalk, Gemstone.) MagLev showed that Ruby is
basically just an alternative Smalltalk syntax.

 _Charles Nutter, one of the creators of the JRuby VM for the Ruby language,
recently posted an article about optimizations that could be made to the VM,
and he improves Ruby performance by as much as twenty times by removing much
of Ruby’s power and breaking compatibility._

"Remove much of Ruby's power"? Around 0.2% of Charles's speed-up came from
removing the ability to override the primitive math operations. Other than
that, his speedups removed _none_ of Ruby's power.

(And nit-picking: _Ruby is probably the most powerful programming language on
the planet for creating DSLs, or domain specific languages. It can do this
because any class or object can be extended, any method overridden, any
constant undefined and redefined, and so on, and so forth._

Not true:

    
    
      >> class StrangeArray < Array; def [](x); x; end; end
      nil
      a = StrangeArray.new(10)
      => [nil, nil, nil, nil, nil, nil, nil, nil, nil, nil]
      >> a.map{|x| x}
      => [nil, nil, nil, nil, nil, nil, nil, nil, nil, nil] # wrong
      >> (0..(a.size - 1)).map{|x| a[x]}
      => [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] # right

I.e., the built-in implementation of Array#map statically refers to the built-
in implementation of Array#[].)

(Edited to correct example, as pointed out by hjdivad)

~~~
stcredzero

        This power does not come without a price. Ruby is slow.
    

ARRRRGH! Why do the expectations of guys from the 80s and early 90s still rule
us? There are Lisp and Smalltalk VMs out there that are something like 70X as
fast as Ruby 1.8! If you add Traits (library for Mixins) to Smalltalk, it is
_equivalent_ to Ruby in power. (1)

The one of the biggest misconceptions we have to face in the Computer
programming and related professions, is that programmers are on the forefront
of technology. Actually, _culture lags technology_. And in that, I
specifically include the culture of computer programmers.

The foundations of OO languages like Ruby and Python were set forth in the 70s
and 80s. WYSYWIG word processing, ethernet LANs, desktop publishing existed
then in pretty much their current forms at Xerox PARC, only slower. The
concepts embodied in Dot-NET and Microsoft's Singularity were bandied about in
the 60s, 70s, and 80s.

The sad thing about our profession is, is that to learn what's coming next,
and what's possible you _still_ have to do some digging into _history_. Things
have improved, but because programming is also a cultural artifact, it lags
technology like any other cultural artifact.

I keep posting this. I used to wonder why it seemed like tilting at windmills.
After writing this post, I now realize why.

Alan Kay: "The Computer Revolution Hasn't Happened Yet."

<http://video.google.com/videoplay?docid=-2950949730059754521>

(1) - In fact, it is _homomorphically equivalent_. This is precisely why
something like Maglev can work. Another interesting thing -- many Lisps are
also homomorphically equivalent to at least 90% of the languages out there.
And one could probably take just about any Lisp and give it the power of the
other 10% (like Erlang) with moderate effort.

~~~
sho
There are NOT SmallTalk libraries out there that are 70x faster than Ruby -
that would be faster than raw C (Ruby is about 30x slower). Are you trying to
say that SmallTalk is faster on average than raw C? So why isn't _everything_
written in SmallTalk?

And I'll believe Maglev's outlandish performance claims if and when I ever see
them realised. If they are anything like those claimed at RailsConf 2008 (50x
or thereabouts), I'll eat my hat and any other hats anyone cares to send me.

I say this as someone who would probably seriously consider murder if it would
make my Rails apps run 10x faster. I _want_ it to be true. I just don't think
it is, unfortunately.

 _And one could probably take just about any Lisp and give it the power of the
other 10% (like Erlang) with moderate effort._

I keep hearing things like this, but no-one ever does it.

~~~
stcredzero
That's strange, I never said anything about the _speed_ of Smalltalk
libraries.

And actually, I know that some Smalltalk routines have run faster than the
same routine implemented in C. By 3% actually. This was over 5 years ago, and
it was a _block cipher_ \-- yes, low-level bit slinging. The DLL was the RSA
reference implementation. (1)

Also, you should be careful about reasoning about benchmarks as if they
applied the same across the board. Anyone who knows a little about them knows
that they are very contextual.

BTW, in the Smalltalk community, spelling the language name with a capital T
is considered a mark of the PHB.

But if you want a Smalltalk library that's much faster than Ruby 1.8, there
are a variety of benchmarks around. Just take some of that code, and declare
it to be a library. QED.

(1) - This is the magic of generational GC. if you know what you're doing,
this is like getting a Buffer Cache you didn't even have to implement. Also,
the implementor got the help of the VM engineer, who provided primitives for
handling 32 and 64 bit arrays.

~~~
sho
Oops, sorry for the faux pas - I thought that it had a capital T - and no idea
why I wrote libraries either. I meant VM, or even better, implementation.
Sorry, I was tired, and now I've wasted your time defending against something
I didn't even mean to say. I was also trying to agree with you about the
unreliability of specific benchmarks.

I'll try to restate. You said this:

 _"There are Lisp and Smalltalk VMs out there that are something like 70X as
fast as Ruby 1.8!"_

That is what I was trying to argue with. I have never seen any proof, beyond a
few cherry-picked examples of Ruby at its worst and Smalltalk at its
(presumable) best, where anything like a 70x speed difference is observable.
On average, ST (is that ok?) seems to be a mere 5x as fast (or so), coming in
at somewhere between 5 and 10 times slower than C.

Ruby 1.8 is in the average case around 30x slower than C. So if Smalltalk can
be 70x faster than _that_ , then it is generally twice as fast as C, on
average, in everyday use. This is not the case, as far as I am aware. I see
this "Smalltalk is SO fast" meme again and again but it just doesn't seem to
be grounded in reality.

Sorry for my idiotic error earlier, and for sounding like a PHB ..

~~~
stcredzero
True, 70X as fast as in benchmarks doesn't mean that much in the real world,
because in real world apps, you are bound by memory use, network, DB backends,
disk access, etc.

The real gain for Ruby, once there are reliable VMs that have good JIT
capabilities is the following:

A dynamic web page can then be compiled into a JIT-ed method, with static HTML
as data in arrays interspersed function/procedure calls. At this point, you
can increase the JIT memory cache so that it can contain your entire web site,
and at that point your web pages are _machine language routines_!

VisualWorks has a framework that does this. It's called Smalltalk Server
Pages.

ST is kosher.

~~~
sho
Look, no. I am not talking about "real world" bottlenecks like disks,
databases, etc. I am talking about ideal mainstream applications written in
community-accepted idiomatic code, no external bottlenecks. Smalltalk will
still not be 70x faster, no way, no how.

If it was, it would be faster - _much_ faster - than C, and if that was the
case, games, graphics drivers, hell, operating systems - would be using
Smalltalk. They are not. QED.

Smalltalk is great and its VMs are highly advanced. I accept that. I still use
Ruby and eat the performance cost because I prefer its syntax and "world". But
man, let's keep the performance comparisons grounded in reality. 70x faster is
ludicrous.

~~~
stcredzero
So long we we're saying, "Look, no I am not talking about..."

My point is that VMs that have been around for _decades_ capable of supporting
languages of equivalent power, and that these are some silly multiplier faster
than Ruby 1.8. This is going to be something like halfway to two orders of
magnitude for some silly benchmarks, and much less in real life. Really,
there's not much of a problem in practice, especially with the embarrassing
wealth we have lying around in terms of processing power.

The real problem -- the thing I am making a point about -- is people going
around saying that, "Oh well, you have to give something up to get the
power..."

No. You. Do. Not!

Ruby is not _that_ advanced. Yes, I would say that it's a lovely language. It
has everything I want, but it's not doing anything that hasn't been done
before. It's unique in that it combines lots of elements in a way that's
really dandy, but there's nothing magic about it. There's certainly nothing so
esoteric, that it couldn't be made to run fast. (There are a few things that
were just bad design decisions, like making every Object behave like a little
hashtable, which could've been left out and made it much easier to run faster
without losing power and expressiveness.)

Oh, and people did implement OSes in Smalltalk and Lisp way back when. Also,
ever hear about something called Singularity? That's a _recent_ OS built on
top of a VM. In fact, I think that more people should be doing just that --
building OSes that are "turtles all the way down" and built to not only
support a particular VM, but to have every single atom of it available to
tweaking by a high-level language. (Lua would be great for this.) But this is
a digression.

Again, going back to my original point -- the real ludicrous thing is that the
_programmer community as a whole_ is still in the thrall of expectations about
speed vs. programming language power that _date from the 70s_. And anyone who
thinks you have to give up _as much_ speed as Ruby has to get that power is
very much in the thrall of such an idea.

------
Hexstream
Ok, that was painful to read by moments.

"Ruby is probably the most powerful programming language on the planet for
creating DSLs, or domain specific languages."

If Ruby has anything over Lisp in the DSL domain (besides popularity), I'd
love to know.

"A web development platform as powerful and syntactically concise as Rails
could only have been done in Ruby."

Idem.

"This power does not come without a price. Ruby is slow. Ruby is generally
recognized as being twice as slow as Python, and at least an order of
magnitude slower than C++ or Java. Charles Nutter, one of the creators of the
JRuby VM for the Ruby language, recently posted an article about optimizations
that could be made to the VM, and he improves Ruby performance by as much as
twenty times by removing much of Ruby’s power and breaking compatibility. If
anything, it shows us skeptics that there is no such thing as a free lunch."

Lisp has all (?) the crazy features you love so much about Ruby and more, yet
it's very fast (try SBCL, it compiles to native x86).

~~~
beza1e1
Ruby has an advantage over Lisp: syntax

Many programmers like syntax, since coding AST-style can be tedious sometimes.
Sure, you can create your own reader macro with Lisp, but that is harder than
overriding some methods in Ruby. Rails is an example that opinionated software
has an edge sometimes and Ruby compared to Lisp is opinionated in terms of
syntax.

~~~
Hexstream
I don't see how the fact many programmers like syntax makes it better.

I also don't see how "coding AST-style" can be tedious, especially since it's
so much easier to automate the process with macros. Granted, you'll want an
editor with support for s-expressions.

~~~
rimantas
The same way why ease of use of something makes that something better (or more
pleasant to work with).

------
jimbokun
This linked article:

[http://blog.headius.com/2009/04/how-jruby-makes-ruby-
fast.ht...](http://blog.headius.com/2009/04/how-jruby-makes-ruby-fast.html)

is really outstanding. It shows you exactly why a language like Ruby is
inherently slower than a language like Java. Mainly, because Ruby does a lot
more work to give you the cool, dynamic capabilities that make people like
Ruby in the first place. Of course, much of the cool work on speeding up
dynamic languages like Javascript and Ruby is all about analyzing code to find
places where you can safely make assumptions about the code (+ is not
overridden here, so I can use the built in integer addition operator, etc.).

(I probably oversimplified that, but I hope it captures the general idea.)

The "Duby" language at the end makes me think of Common Lisp and how you can
optionally declare types the compiler can use to compile faster code. Would it
be worthwhile to add optional typing to Ruby?

------
mrduncan
_Ruby is generally recognized as being twice as slow as Python_

Am I the only one who has never heard of this? I'd definitely be curious to
see some relevant benchmarks if anyone has any.

~~~
wheels
It varies naturally per application. When I was doing a little prototyping
before deciding to write the core of our ranking algorithm in C++ I took the
tightest inner loop and compared C++, Java, Python, Ruby and Perl. For
floating point numerics and lots of list traversal the results were:

    
    
      - C++    | baseline
      - Java   | 2.5x baseline
      - Perl   | 4.5x baseline
      - Python | 6x baseline
      - Ruby   | 14x baseline
    

As a result, we're using C++ where it really matters, Java where it kind of
matters and Ruby where it doesn't. This was more than a year ago though, so
any performance gains since then naturally aren't included.

------
jpedrosa
To take the article in the right light, consider that in Ruby-land, syntax is
important, it matters. For example, optional parentheses. Spaces are mostly
disregarded unless they aren't in a few cases. Switches that support almost
any type, even though when folks play with entire Hashes as keys things can
fail to work even more during the 1.8 to 1.9 transition...

Multiple ways to create literal strings so you can save escaping things inside
of them or save some other action.

Closures with one block shortcut per method so you can save a little more
characters there.

These things encourage the developer to revisit their implementations and cut
the redundancies much more.

The p method coupled with the inspect feature that gives you the description
of a certain object so you can debug it or at least see its content very
easily.

No use comparing Ruby to LISP and Smalltalk in some "power subjectivity"...
Ruby is hackish and used by "hackers" alike. It's not perfect of course...

And I liked this article for it put into better words what I wrote in some
comments on the original article.

------
Tichy
Yes, Ruby is OK - buuut: I would like a language that is good for everything.
I don't really understand the argument that "Ruby was only replaced for stuff
it wasn't appropriate for". Some languages are good for everything, so why
make the effort to learn languages that can not do everything? Yes, there is
the DSL stuff, but atm I am not sure if I really care so much for it.

For example, yesterday I caused an infinite loop in my Rails controller
because I forgot the "@" in front of a variable that had the same name as the
action - so Ruby interpreted it as a method call, not a variable reference.
Being able to call methods without "()" is part of the Ruby-DSL stuff, but I
wonder if typing "()" would not actually be the lesser pain in the end. I
probably lose much more time to hunting such bugs than I lose time typing ().

~~~
jjames
The "()" thing is something that personally clicks with me in python. I
absolutely love methodname as reference always, methodname() as function call
always. I'm fetishistic about predictability though.

To your main point, all general purpose languages are good enough at
everything. That's how they get to be called "general purpose". But, I would
argue that there are 0 languages that are great for everything. There is no
language that can be shown to have 0 tradoffs when compared to all other
languages.

The thing that often gets over-looked with Ruby is the trade off that many
other languages make of developer time and enthusiasm (happiness, comfort,
engagement, morale, etc). If you don't believe those factors exist or that
they have little value then the trade off the language makes to afford those
features is lost in translation.

To make a crude analogy: if I were hiring to get a company off the ground I
would love to find a single person to do all of the product concepts, design,
implementation, sales, accounting, legal work and support. The problem is that
the jack of all trades is master of none and you're likely to get some
creative concepting all over your accounting. Languages specialize in
thousands of ways over the growth of the language (and hopefully consciously)
- much like people do with education and career development. Even at times
under the guise of general purposeness.

This is all to say that unfortunately your wanting a language that is great
for everything is impossible. But, you might find a language that is good
enough at all of the trade-offs you care about without ever trading off all
the other things you care about and for all projects for all time.

~~~
Tichy
It may be true that there is no language to rule them all, or it might not be
true. Just because people typically are not good designers, good coders and
good admins in one person does not prove that programming languages can't be
good at everything (they are all Turing complete, after all).

Of course there are always tradeoffs, but some tradeoffs might not matter
(that is, there might be no situations in which the tradeoffs would matter.)

I am not yet convinced that Ruby is so much more pleasant than other
languages. I don't know Scala yet, but what if Scala is actually pleasant to
use? Then Scala might be better than Ruby in every respect. (Not saying it is,
I don't know Scala).

~~~
jjames
Good point. There could be a language that made 0 trade-offs "that matter" vs
all other languages. The problem remains that there will likely still exist at
least one person for whom some trade-off of little or no substance is
important (even essential).

When people talk about power, elegance and abstraction of a language (not
compiler, VM, libraries, etc) they are talking about the language platformed
on the human brain (something unique to the individual). Ruby targets a
certain brain platform while C targets a different one and Scala yet another.
Twitter engineers sound like they have brains appropriate for both Ruby and
Scala. Scala makes less technical trade-offs toward their end and so there
arises a clear winner (for them).

The day we're all using the same brain platform then there could be argued to
exist a best language.

------
grandalf
uh I was using Starlng in production and switched to Kestrel expecting a
massive speedup.

It actually slowed down the queue by about a factor of 10.

Maybe the jvm offers GC benefits on massive hardware, but I have not been
pleased at all so far.

~~~
fhars
Just to make sure you are measuring the right thing: You made your
measurements _after_ passing a few hundred thousand messages through the
system to give the JIT in the JVM enough information to act upon? And using a
server VM? With which GC?

Benchmarks on the JVM are notoriously tricky...

~~~
grandalf
I put tens of thousands of messages through it before noticing it was slow.
And yes, using a server vm (sun's latest)...

I haven't given up on it... just posted the above comment b/c I had assumed
that Kestrel would just be way faster b/c it's java, which turned out to be
quite an erroneous assumption.

------
longint
I really liked this article, and remember reading the original article and
thinking that it does a good job of explaining an engineering approach that
invariably needs the ability to skillfully use many technologies.

------
MaysonL
Languages are neither slow nor fast, only implementations are.

