

Threads (in Ruby): Enough Already - wycats
http://yehudakatz.com/2010/08/14/threads-in-ruby-enough-already/

======
tptacek
This post is a useful rundown on where threads stand in Ruby now, but it hangs
on a bit of a straw man argument. The argument for evented concurrency assumes
threads _work_. Async design isn't a reaction to MRI's crappy green threads.

It's also not a very compelling argument for threading to say "web programs
don't share a lot of state, so you don't have to worry about synchronization".
If all you do are CRUD apps, you can indeed punt scaling to the database. That
doesn't mean threads are more effective than events; it means you made
concurrency and synchronization someone else's problem. There's nothing wrong
with that, but it's not a convincing demonstration of threading.

~~~
Nwallins
> _The argument for evented concurrency assumes threads work. Async design
> isn't a reaction to MRI's crappy green threads._

Truly? To my understanding, event loops require inversion of control (and
likely callbacks and broken exception handling). This is a large cost that
requires a benefit to be worth it. I understand that benefit to be: you don't
have to deal with threads (or bad implementations of such).

This blog post comes to mind:
[http://www.unlimitednovelty.com/2010/08/multithreaded-
rails-...](http://www.unlimitednovelty.com/2010/08/multithreaded-rails-is-
generally-better.html)

~~~
tptacek
Not being a patterns guy or a Fowlerite, I can't tell you whether async
"requires inversion of control". Apropos nothing, I'm also unlikely to
recognize "dependency injection" when I see it, as my first language was C,
not Java.

Async code isn't "likely" to require callbacks; it will almost certainly
involve callbacks, those being a central design feature of async code. You
should play with the idea for a bit before forming an opinion about it.

I don't know what you mean by "broken exception handling". This may be a
Rails-ism I'm unfamiliar with. I'm very familiar with Rails (we ship a fairly
large product built on it), but I've never tried to shoehorn async I/O into
it; like Twitter and presumably every other large site, we do Rails on the
front-end/UI and fast things on the backend, in our case with EventMachine.

Like I said: I'm _not arguing_ that Rails threading is bad, or even that Rails
should have better async support. If I cared that much about the performance
of my front end, I probably wouldn't be serving requests directly off Rails.
Rails developers may very well be better off with threads. But that fact has
little to do with the merits of threading and async _as concepts_.

~~~
jerf
'I don't know what you mean by "broken exception handling".'

Take the following Python-esque (but not Python) psuedo-code in a threaded
language:

    
    
        try:
            header = read(socket, header_size)
            if has_short_flag(header):
                 return read_until_closed(socket)
            else:
                 handle_message(header, socket)
        except SocketClosedException:
            log("lost socket while processing headers in packet")
    

The evented equivalent of this code will have to be chopped into pieces at
each of the read calls, and it is impossible to wrap the second read calls in
the same exception handler like this. You can manually route exceptions around
if you're careful, but that is definitely a pain in the ass and is sometimes
very hard to test (hard to test exception handling for an exception you can't
really fake for some reason). (Of course you can't always test it in threaded
code either, but it's radically simpler there and therefore less likely to
break, you don't have to test the plumbing.)

This is one of the reasons I've spent the last few years _fleeing_ event-based
code towards things like Erlang, rather than running towards it; I've been
doing event-based code for non-trivial work and things well beyond "demos" and
the plumbing just explodes in your face if you want to build actually-robust
software where simply crashing in the middle of a request isn't acceptable.
Despite my best application of good coding practices and refactoring you still
can't get close to the simplicity of something like Erlang code.

(By the way, if you are stuck in evented land, one of the things I have
learned the hard way is that anywhere your evented API has an error callback,
you _absolutely must_ provide one that does something sensible. I now always
wrap such APIs in another layer of API that does nothing but crash as soon as
possible if no error callback is provided. If you can't figure out what your
error callback for a given such call should be, that's your design trying to
tell you something's wrong.)

~~~
tptacek
If you're an Erlang advocate, I'm not going to convince you of the utility of
eventing in any other language. I respect your decision but am uninterested in
choosing a programming language based on a concurrency model.

That said, your (admittedly hypothetical) example is hideously broken. In
EventMachine, and written properly:

    
    
      def receive_data(buf)
        @buf << buf
        while have_complete_request?
          begin
            send_data(handle_next_request)
          rescue => e
            log(e)
          end
        end
      end
    

Notice the _single exception handler_ bracketing the entire request. (This
code is wordier than I'd like because I hoisted the exception handler out of
handle_next_request to illustrate it).

The mistake you made (and it's common to a lot of evented code) is in driving
the entire system off raw events. Don't do that. Buffer, to create natural
functional decomposition.

Evented code is rarely _as simple_ as synchronous code, but there's no reason
it has to be needlessly choppy.

That said, I think this design is overvalued. Yes, it's true, you (probably)
can't always wrap an entire request's processing in a single exception handler
in any evented Ruby library I know about. But I wouldn't wrap an entire
request handler in a single exception handler in any case! If I was preparing
to deal with a database exception, I'd wrap the code making the database call
in an handler for the database exception. If I was preparing for a filesystem
exception, &c &c &c.

Incidentally, I've been doing very, very, _very_ large scale evented systems
for going on about 10 years now (large scale: every connection traversing tier
1 ISP backbones), and, sorry, this stuff has never blown up on me. I may have
been shielded from exception drama by working in C/C++, where exceptions are
not the norm. I was a thread guy before that. Threads definitely did blow up
on me, a lot.

~~~
jerf
I don't think you can do that with Javascript or Node.js. When you have a
callback, you lose all stack above it.

See, I wouldn't even call that "evented code" in the way that people are using
the term, regardless of what it says on the tin, precisely because you aren't
losing the stack frame here and can still catch exceptions and such. Evented
to my mind is something like Node.js is when you have to chop up your code
manually. At least, I've never seen anybody demo Node.js code that isn't
chopped up manually and I am at a loss as to what features of Javascript would
let you translate that Ruby snippet directly without losing something
fundamental about the stack.

Under the hood, _everything's_ event-based (with optional preemptive
multitasking), there's just varying levels of compiler optimization that
affects how much you have to do manually and how much you have to worry about
it. The inner event loop of Erlang and the inner event loop of Node.js and in
fact the inner event loop of just about anything nowadays looks pretty much
the same.

That's not the way in which I say evented code blows up. If you can write like
that, it doesn't blow up, because you don't have to sit there and basically
manually implement your own stack handling if you want anything like that sort
sane exception handling, it all just works.

Since this is a terminology issue there is, as always, grey areas, but since I
mostly use the term evented in the context of the Node.js hype I tend to use
it that way. I've been doing stuff like your snippet for a while too and it
hasn't blown up on me either, which is why I'm so down on the style of coding
Node.js entails, which does.

The point of my snippet is not that that is a brilliant choice intrinsically,
the point is that you don't have the choice and end up implementing anything
like that manually.

~~~
tptacek
You've lost me. Nothing in this code depends on having a single consistent
stack from main() to handle_next_request().

~~~
jerf
You need a "consistent" stack in the whole of everything executing under
"handle_request" if you want that "one single exception handler bracketing the
entire request". You don't get that in Javascript; you need a language with
more power. There are several viable options: Threading, continuations, some
hacked up thing like Python's yield (which is neat, but limited). But
something.

Javascript's a very nice, dynamic language in most ways but its functions are
very, very weak. They'll get better in the next version of ECMAScript, though.
(Node.js will probably benefit from it.)

~~~
tptacek
I'm pretty sure you're wrong about this. The exact same idiom I used to buffer
up requests and feed them whole to a single function that could catch all
possible exceptions works just fine in Javascript. Is this an issue with JS
exceptions that I'm unaware of? I may just be talking past you.

~~~
jerf
You're talking about the function of what I posted, I'm talking about the
form.

    
    
        try:
            something = some_io_that_needs_an_event_callback()
            if do_something_with(something):
                more_io_with_callback()
                some_other_callback_thingy()
            else:
                yet_other_io()
                some_other_io()
        except AnyException:
            do_something_sane()
    

is not a structure available to you in Node.js, again, pending somebody
proving otherwise in Javascript (though I've made this complaint in other
places where somebody really should have shown me the code). That expands to
tons of code in Javascript, code which is very resistant to abstraction.

Your code block says it is available to you in Ruby. There's no reason why it
shouldn't be; it's not a hard compiler transform to manage that. Taking code
that looks like a "thread" but compiling it into events and using something-
like-continuations to keep the stack together is on the order of a homework
problem. Based on what I know about Ruby, the responsibility of those pieces
is split up differently but all the pieces must be there. But it's not
something Javascript can do (quite yet, and based on what I've seen in Python
the 'yield' approach has its own issues).

------
ericb
Ok, dumb question--I'm a bit naive on process forking. On Linux with Ruby, if
I fork a process, must the new process stay on the same core? Forked processes
begin by sharing memory initially in Ruby, correct? Can anyone enlighten me on
the mechanics of forking and if there are scenarios where it helps with
utilizing multiple cores (maybe relatively independent workers that just read
shared settings?).

Forking seems...a little weird to me coming from my previous Windows
background.

~~~
wycats
Forking creates a new process, initially "sharing" memory via copy-on-write
semantics (and therefore being eligible to run on another core). However, C
Ruby's default mark-sweep garbage collector touches pretty much all memory
immediately, eliminating the typical memory savings of a forking model.

As I commented earlier, Ruby Enterprise Edition (made by the same guys as
Passenger) ships an alternate GC that doesn't actually write to the memory
space of the original objects in order to mark them.

As a result, some memory (but not necessarily as much as you'd expect) can be
shared between processes forked from the same parent and running on multiple
cores. Sort of a middle ground between totally separate processes and shared
memory via non-GIL'ed threads.

~~~
ericb
So you won't get much by way of memory savings, but you might expand the
volume of work each batch of threads (now on separate processors) could
complete if you forked cores - 1 times?

~~~
ww520
There are different kinds of memory in a process, e.g. code segment, read-only
data segment, and read-write data segment. The code and read-only memory can
be shared between the parent and child processes. The read-write memory pages
are copied-on-write when they are modified. There can be fair amount of
sharing.

------
samstokes
This article is a good wake-up call, but even if Rails itself is now (since
2.2) thread-safe, I need to be fairly sure that _all_ my libraries are before
I can enable threaded request dispatch in my Rails app. That's a lot of code
inspection / thorough regression testing / inevitable bugs.

The Brightbox team tried to tackle the related problem for moving to Ruby 1.9
with a crowdsourced library review site: <http://isitruby19.com/> Does anyone
know of a similar site where people can report whether libraries are thread
safe? (The code for Is It Ruby 1.9 is available on github -
<http://github.com/brightbox/isitruby19> \- so it would be easy enough to set
up a clone, but I can't commit to maintaining such a thing.)

~~~
wycats
The Rails Plugins site (<http://railsplugins.org>) that we built at Engine
Yard has 406 registered plugins, and a criteria for thread safety
(<http://railsplugins.org/plugins?criteria[]=2>).

The way it works is that the author can specify whether or not he thinks it's
threadsafe (yes/no/maybe), which is then verified by users who can specify
whether they agree. If a user marks a plugin as not-threadsafe (or not-Rails
3, or not-JRuby, or not-Ruby 1.9), the author has 7 days to help the user come
around before it sticks.

So far, there are 60 plugins marked as threadsafe (which means that either the
author said "yes" and nobody disagreed, or the author said "maybe" and all the
votes so far say yes).

~~~
samstokes
That's awesome, and exactly what I was looking for. Thanks!

I found that the postgres gem, the most immediately important for me, isn't
listed. Do I need to be the gem author/maintainer to register a gem, or can I
just add it?

~~~
mml
Wycats noted in the comments of the article that the pg driver is indeed safe
for use in rails threaded mode.

As a psql fan, it makes me wonder what the rails world would be like had dhh
had picked psql from the start.

------
Tichy
Somehow the takeaway from all these posts seems to be to stick to JRuby. Am I
reading that right?

~~~
wycats
If you use C Ruby, you'll need one Ruby process per core, managed by something
like Passenger (mod_rails) or Unicorn.

If you use JRuby, you'll need one Ruby process per machine (for N cores),
managed by the JVM.

For boxes with a lot of cores, JRuby's larger memory footprint is overtaken by
the ability to share that memory across a number of cores.

This story is also somewhat complicated by Ruby Enterprise Edition, which adds
copy-on-write semantics to Ruby's GC, and is built by the same guys as
Passenger, making it possible to share SOME memory between processes.

With all that said, we're really talking about marginal amounts of RAM. The
real takeaway is that if you're running 6 processes per core (very common),
you're doing something very wrong.

FYI: At some point in the future (1.2?), Rubinius will also be able to run a
single Ruby process per machine.

~~~
evanphx
Rubinius wise, I'm actively working on making in concurrent. Yes, a future
release will incorporate that work, which allows one Rubinius process to fully
use as many cores as a machine has.

~~~
extension
Concurrent GC too?

~~~
evanphx
Concurrent GC is not currently planned, but it should be noted that our
generational GC does wonders to reduce GC pause time anyway.

Additionally, even Hotspot typically defaults to their stop-the-world GC. This
is because a concurrent GC typically spreads part of the GC time around, ie,
performance is slower to reduced GC pause time. But even a concurrent GC
typically has to stop all threads at some point to get everything consistent.

