
Rails threading nightmare - wheels
http://bibwild.wordpress.com/2009/02/11/rails-threading-nightmare/
======
patio11
Rails typically scales via adding processes rather than threads, and you can
do concurrency the same way. Rather than _ever_ doing anything blocking in the
HTTP request/response protocol, just offload your tasks to worker processes
and give your clients the page layout with Javascript ready to poll for the
interesting stuff.

This works, is fairly simple to implement, and kicks concurrency to database
programmers, who actually understand it.

The only reason I hesitate to give this advice is that substantially all of my
downtime can be traced to worker processes either failing silently or going
wild and munching my swap until the server died. Productizing the use of them
is a little tricky.

Concrete recommendation: I like DelayedJob, Thomas likes the Redis queue.

------
tptacek
Do not --- DO NOT --- thread long-running Rails applications. By doing so, you
expose yourself to every possible runtime bug, every possible C-code extension
bug, and every possible concurrency bug in all of the above _plus_ the whole
Rails stack.

We had exactly the problem this post describes with Playbook, even without
using threads. We suffered random MRI crashes for weeks and weeks. In
particular, the SWIG bindings for Subversion (two great tastes that make me
want to throw up in my mouth) would randomly corrupt memory, but only when
hosted in a process servicing Rails requests.

We solve this problem --- remarkably easily --- by creating a shim class in
front of our SVN code, hosting the "real" SVN code in a sidecar process, and
remoting the requests. In no other mainstream language is it as easy to remote
requests; you can literally just Marshal.save({method => args}) and chuck that
over a socket. It's even easy to pass exceptions back and forth.

If you have a task-based architecture, there's already a Rails best practice
solution: use DJ or Resque to factor tasks out into worker classes that run
off job queues. Both DJ and Resque are extremely easy to set up. There's no
reason to overthink this.

------
pilif
I might be old-fashioned, but I would not start any background processing in
the context of the process that handles the request. Depending on the web
server handling the request, your process (threads are subprocesses) can be
killed at any time when it's not serving a request.

That could either kill your threads or prevent the webserver from correctly
shutting down the process or handling another request on the same process.

I would hand the background jobs off to some daemon process or write
instructions describing the job into some queue and then have a cron job
process them.

I would never expect to be able to keep anything running after request
processing has finished.

------
wheels
I posted this when I hit similar problems and ended up largely isolating them
to ActiveRecord. I have a class which I use for doing background processing in
Ruby:

    
    
      class Future
        def initialize(postprocessor = nil, &finalize)
          @postprocessor = postprocessor
          @future = Thread.new do
            begin
              finalize.call
            rescue => ex
              warn "Exception in background thread: #{ex}"
            end
          end
        end
    
        def method_missing(method, *args, &block)
          data.send(method, *args, &block)
        end
    
        def to_s
          data.to_s
        end
    
        private
    
        def data
          @data ||= @postprocessor ? @postprocessor.call(@future.value) : @future.value
        end
      end
    

However this was causing random 5 second delays in my Rails app -- even if I
never called "data" (and hence never joined). I added the postprocessor so
that I could pass in a lambda that does all of the ActiveRecord stuff back in
the calling thread, which alleviated the problem.

But in contrast to what bensummers said -- this is actually a Rails problem,
not a Ruby problem. Ruby's threads suck, but it seems to be ActiveRecord's
locking that's at the root of the problem.

------
andrewcooke
These are green threads, according to the article. That means that they must
work cooperatively (they're not system threads). That means that if any of the
threads blocks - say on reading a secondary web service - then all threads
will block. That's because the thread switching is managed by the VM, not the
kernel, so it has no control over kernel level blocking calls.

I know _nothing_ about Ruby, so perhaps I am misunderstanding, but this would
explain what he is seeing and is consistent with what he describes.

------
ericb
Seems like a flaky architecture that depends too much on the internals of the
web server.

I would launch a stand-alone ruby process for the communication and connect to
it via Drb to get results. Note that this will not work if you have greater
than about 40-100 things trying to connect to the object, but it is simple and
will work pretty well at small scales-it is less hairy than threads.

Alternately, you could put rabbitmq or a messaging queue in the middle, and
look at EventMachine for the communicator process. That will scale nicely.

------
steveklabnik
> February 11, 2009

Not that threading is perfect. But really, if you're doing anything intensive,
you should be switching to background job + polling, anyway. You don't want to
starve your mongrels/unicorn workers!

This isn't to say that your background job wouldn't have concurrency... but it
doesn't need the whole Rails environment.

Ruby threading has changed a lot since 2009. Still not perfect, but Rubinius
is supposed to be removing the GIL...

------
bensummers
This article seems to be more about the limitations of the standard Ruby
interpreter, than about Rails.

Switching to JRuby would be an easy solution to these woes. Most gems are
compatible these days, and there's even experimental support for C extensions
as a temporary solution during migration.

