
"Ruby developers need to stop using EventMachine. It's the wrong direction." - caseorganic
http://www.slideshare.net/KyleDrake/hybrid-concurrency-patterns
======
programminggeek
If you really like evented programming, use node. Everything there is evented.
I tried EventMachine and it just felt worse than node, documentation is worse,
libraries are worse, it just wasn't a whole lot of fun.

I haven't tried celluloid yet, but I've learned one lesson building things
with PHP, Ruby, Python, Node, Java, Scala, etc. That lesson is to use tools
with a community of people around it that are using those tools to solve the
same problems as you. The more community is around a project the more likely
that someone else has stumbled upon whatever bug or problem you have hit
already and found a solution if it exists. Also, documentation tends to be
better and easier to find on more popular languages and frameworks.

So, if you like busting out CRUD web apps, you probably should look at PHP and
Zend/Symfony/etc. framework or Ruby on Rails or Python Django or Java Play. If
you like doing single page JS apps you should look at Knockout, Backbone, and
Ember. If you want to do more evented/parallel networked apps you should look
at node, scala, clojure, go, and erlang because those communities care a lot
about threading, evented, actor pattern type programming.

Evented/multicore/multithreaded programming is just not something that say
PHP, Ruby, and Python have embraced as much as a community because it's not
for the most part a problem that the average PHP, Ruby, and Python dev is
trying to solve most of the time.

~~~
tptacek
Python programmers have been doing evented code for almost a decade using
Twisted. EventMachine happened later, but then, so did the mainstreaming of
Ruby. You realize, don't you, that Tcl has all three of Javascript, Ruby, and
Python beat when it comes to evented I/O?

Node does _not_ own evented I/O.

~~~
programminggeek
I agree that Node doe not own evented I/O, my point was more that if you care
about concurrency, parallelism, evented io, you should work in communities
that care about those things and build tools around them. Ruby as a community
cares more about Rails than evented I/O. Node is purely evented I/O, so that's
basically ALL they care about. I guess Python is probably more in between.

I didn't mean to say node owns evented I/O, just that their whole community
embraces it.

------
ericb
I've looked at Celluloid. It offers a much nicer paradigm than eventmachine.
It also provides a nice way to distribute actors over multiple machines
(DCell). I strongly agree with this slidedeck. Callback muck is maddening.

~~~
danielszm
I wonder why nobody ever talks about drb, a distributed system that ships with
ruby. It's active in Japan, and there has been an effort to make it better
known in the rest of the world with a book dedicated to the topic in english
(published by Pragmatic Programmers). Still, it hasn't stirred much interest.
Even if it's less advanced than DCell, it's still a recommended way to start
learning about distributed computing.

~~~
tptacek
It's brittle, opaque, and relies on Marshal, an interchange standard nothing
else uses. It's so easy to build alternatives to Drb that use better
interchanges that nobody uses Drb. And once you do that, you find it's easy to
back your system with Redis or a database, which you do because it makes
sense, and then you start getting pulled towards message architectures ---
which are themselves _usually_ superior to direct invocation of remote
methods.

~~~
ericb
In my previous load testing of Drb, at 86 connections (remote objets) with
even moderate use, things consistently went bad. This was independent of
number of machines.

------
newobj
Not to judge this presentation up or down; I'm not a Rubyist... but a
reflection for myself - you know you're getting old when you see ideas that
have come and gone and come and gone come again.

~~~
astrodust
Like rings on a tree, you can judge how old you are by how many times a
particular idea has resurfaced after some time in obscurity.

------
tptacek
We write a _lot_ of EventMachine code here; I have some questions. He writes:

EventMachine is • A frankenstein guts the ruby internals • Not in active
development • Makes non-blocking IO block • Requires special code from Ruby
libraries • Hard to use in an OOP way • Is really difﬁcult to work with •
Poorly documented

I'm not sure I understand how EventMachine "guts the Ruby internals" (I didn't
watch the talk). It's true that EventMachine's internals are C++, not Ruby;
there was originally a reason (again, I think it had to do with green threads)
that it was designed this way. I'm not sure I can think of the Ruby
functionality that EventMachine changes, or the manner in which EventMachine
mucks with the interpreter or its runtime. I'm obviously ready to be
corrected, but I'm missing how this impacts me as a programmer. Maybe he's
talking about exception handling?

I also wasn't aware EventMachine "wasn't under active development". Because
it's just an IO loop. Is libevent in active development? Do I need to be aware
of that to use it? The underlying OS capabilities EventMachine maps haven't
changed in over a decade. I think I'm actually _happy_ they aren't constantly
changing it.

I also don't understand how EventMachine "makes non-blocking block". All
EventMachine I/O is nonblocking; it's essentially a select loop.

I also don't understand what special code EventMachine demands from libraries.
Maybe he means database libraries? That is, maybe he's referring to the fact
that you can't use standard Ruby database libraries that rely on blocking I/O
inside an EventMachine loop? I'm wondering, then, what he expected. We wrote a
little library (a small part of it is on Github) to do evented Mysql, but we
stopped doing that when we realized that Redis evented naturally, and we just
hook Mysql up through Redis.

"Hard to use in OOP way" just seems wrong, given that the ~30 evented programs
I can find in my codebase directory all seem to be pretty object-oriented. So,
that's not so much a question on my part.

Really difficult to work with? I've taught 7 different people EventMachine, in
a few hours each. EventMachine is _easier_ than Ruby's native sockets
interface, in several specific ways.

I think maybe the issue here isn't so much EventMachine, but the idea of using
EventMachine as a substrate for frameworks like Sinatra and Rails. _That_ idea
is whack, I agree. Trying to retrofit a full-featured web framework onto an
event loop seems like an exercise in futility.

But on the other hand, I've been writing Golang code for the past 2 months,
and Golang is _militantly_ anti-event; it doesn't even offer a select
primitive! Just alternating read/write on two different sockets seems to
demand threads! And what I find is, my programs tend to decompose into handler
functions naturally anyways. I try to force myself to write socket code like I
did when I was 13, reading a line, parsing it, and writing its response, but
that code is brittle and harder to follow than a sane set of handler
functions.

So, long story short: I'm not arguing that evented code is the best answer to
every problem, or that web frameworks should all be evented, or that actor
frameworks aren't useful. It's probably true that a lot of people rushed to
event frameworks who shouldn't have done that. But there are problems ---
like, backend processing, or proxies, or routers and transformers, or feed
processors --- where event loops are the most natural way to express a
performant solution.

~~~
bascule
> I'm not sure I understand how EventMachine "guts the Ruby internals"

EventMachine does not use the Ruby IO primitives (e.g. TCPSocket, UDPSocket,
etc) as the basis of its IO abstraction, and instead has reimplemented its own
set of primitives for doing IO.

Because of this it can't take advantage of work being done in Ruby core to
advance Ruby's socket layer. For this reason IPv6 support langered, among
other problems.

This also severely complicates making multiple implementations of the
EventMachine API, such as its JRuby backend (which maps onto Java NIO)

I think the real problem is EventMachine's original goal was to be a cross-
language I/O backend similar to libevent or libev, but since it wasn't a
particularly good one, the only language that wound up using it was Ruby.
Compare to Twisted, which is built on libevent, or to Node, which is built on
libev/libuv

~~~
tptacek
I like libevent and all, but I don't think Twisted's use of libevent is a
particularly big win for Twisted.

Again, this is apocryphal, but I remember someone trying to wrap Ruby around
libevent and failing; I remember there being a reason this had to be done
bespoke. And having fallen into this particular NIH trap many times before:
there's just not a whole lot to a simple socket I/O loop.

What's been your experience with JRuby/EventMachine?

~~~
gnufied
Zed shaw tried to make libevent work with Ruby, but this was when we had Ruby
1.8.5 and there were problems with Ruby interpreter that prevented this
(around threading IIRC).

------
chrislloyd
After using EventMachine in heavy production at Minefold for the last year and
a half we're finally ditching it for Go. Our experience has been similar to
the OP's. However we had random exception gobbling, internal threading issues
and increasingly worse performance + code maintainability.

We've been happily using Go for a few internal systems are are slowly
galvanising most of our backend with it too. Life is happier.

------
dacort
I just replaced an EventMachine-based project with Celluloid::IO - it was a
pretty simple replacement, although I had some issues with concurrent
connections (<https://github.com/celluloid/reel/issues/11>).

I have yet to test it in production, but it looks pretty promising.

I had a lot of problems with EM blocking networks connections if an event loop
was tying up CPU, but I suppose that's to be expected.

------
PaulHoule
Well, when I hear about Actors I reach for my gun.

I knew a guy who chose Scala for a project so he could use Actors for
concurrency.

The system never gave the same answers twice and wouldn't peg all the cores on
a 4-way machine.

I spent two days trying to fix it, then I got wise and switched back to Java
and got it working in 20 minutes with ExecutorService with (i) no race
conditions, and (ii) nearly perfect scaling up to eight cores.

------
numbsafari
1999 called and it wants its DCOM back.

------
johnkchow
There's a fork out there called EventMachine-LE, which is currently in active
development for latest features (i.e. IPv6).
<https://github.com/ibc/EventMachine-LE>

Anyway, just wanted to throw it out there that I've deployed several
production apps (gaming/messaging servers) using EventMachine, and combined
with em-synchrony, I'm pretty comfortable with its performance and
limitations. There seems to be good community support (thanks to Ilya
Grigorik's articles and em-synchrony) with plenty of examples/documentations.

Just wondering, are there any people out there with Celluloid app experience
that's currently in production? Like any other geek, I love programming "the
right way" but I know nothing about Celluloid and it's real-world
benefits/drawbacks (performance, API, code maintainability, code
documentation, community support, blog articles, etc).

~~~
cmer
Sidekiq is probably the best example of Celluloid in action.

~~~
danielszm
Correct, and for those interested in approaching Celluloid, here is post that
demonstrates a concurrent "Hello, world" program:
<http://danielsz.posterous.com/167870244>

