
Coroutine Event Loops in Javascript - kilovoltaire
http://syzygy.st/javascript-coroutines/
======
jlongster
I can't believe he doesn't mention task.js: <http://www.taskjs.org/>

There's tons more potential that this guy describes, too. If you combine this
with promises, which is what task.js does, you can immediately start using it
in all of your node/client-side code that uses promises to turn your callbacks
into `var x = yield getAsyncThing();`.

~~~
kilovoltaire
Good point, that is certainly relevant!

I tried to keep this writeup focused on comparing callback functions to
coroutine event loops, especially regarding the differences in how they deal
with state.

But I'll add a reference to task.js at the end, once I've tried it out.

------
a1k0n
The fact that you must call send() on a coroutine to start it is very
confusing -- what distinguishes function from a coroutine? The presence of
'yield' inside a function body magically means calling that function doesn't
actually return the return value of the function, but a new coroutine which
hasn't executed any of its body? This is even more confusing than the
function-as-an-object syntax, where at least the new keyword distinguishes the
behavior.

I am very much in favor of coroutines in javascript, but I'm kind of surprised
by this choice of implementation.

~~~
pfraze
I'm not sure what's with OP, but according to [1] (and the example on
tasks.js) a generator-producing function should have an asterisk before the
name.

    
    
      function *myCoroutine() { ... }
    

1\. <http://www.2ality.com/2012/07/esnext-classes.html>

~~~
kilovoltaire
Cool, I didn't know about that syntax!

Unfortunately it isn't yet supported in Firefox, but I'm glad something like
that is in the works.

~~~
pfraze
Hope I didn't offend. I really enjoyed your post; the coroutine syntax seems
like an extraordinarily intuitive approach to state machines. Game-changing,
maybe. Look forward to trying it.

------
majke
I'm glad javascript people are excited about generators, but I must warn -
generators are not a replacement for real coroutines.

Most importantly - generators don't compose.

(I'm talking about python-style generators, the type of generators javascript
adapted.)

To give you an idea of what I mean:

    
    
        second_sound = coroutine(function() {
            yield;
            console.log('Tick!');
            yield;
            console.log('Tock!');
        });
    
        hour_sound = coroutine(function() {
            yield second_sound();
            console.log('Ding dong!')
        });
    

It's pretty obvious what programmer wants to achieve. But with generators you
can't call "yield" from deeper-level function - generators are one-level-deep.
The only way to compose is to 'yield' a new (deeper) generator. At this point,
the caller of hour_sound() needs to understand what to do if the callee
returns a generator, instead of 'null' or a valid value. And it needs to run
this "deep" generator before doing 'send' on the hour_sound again.

This is just the beginning of the mess. Errors get very hard to track as
tracebacks get completely unreadable. It's not easy to decide what to do when
one of the nested generators throws an exception.

Python frameworks went through this long time ago, see here (look especially
for the "yield" keyword): <http://www.tornadoweb.org/documentation/gen.html>

Here's a bit of the underlying logic of this "simple" "coroutine-programming-
syle" layer of tornadoweb:
[http://www.tornadoweb.org/documentation/_modules/tornado/gen...](http://www.tornadoweb.org/documentation/_modules/tornado/gen.html#engine)

And here's more!
[https://github.com/facebook/tornado/blob/master/tornado/gen....](https://github.com/facebook/tornado/blob/master/tornado/gen.py#L342-L396)

All this magic is heavily based on the way python exceptions are integrated
with generators, for example one can try to catch "StopIteration" to do a
cleanup within generator. I doubt this is possible in javascript as the
exceptions mechanism is much less mature.

Edit: Don't get me wrong - I love generators! But one-level-deep generators
are not coroutines with stack that can be "blocked" from any deeper function.

~~~
pdonis
Python 3.3 fixes this with the "yield from" keyword:

<http://docs.python.org/3.3/whatsnew/3.3.html#pep-380>

If Javascript did the same thing, you would write yield from second_sound();
in hour_sound, and then you could run hour_sound as a generator and it would
Just Work.

~~~
jlongster
ES6 does the same thing with the `yield*` syntax.

------
ambrop7
How useful is this mechanism in practice? It seems very limited to me, because
you can't just _call_ a coroutine from another coroutine, and expect it to do
its job asynchronously, without the caller having to manually deliver events
to it. How can the caller be expected to know what kind of events the
coroutine is expecting, and how to deliver them?

What would be useful if this would allow writing asynchronous code in a
synchronous style, so that you could do things like:

    
    
      function requestHandler (client, request) {
          data = sendToDatabase(request); // "blocking" operation
          client.sendReply(data);
      }
    
      startServer("0.0.0.0:1234", requestHandler);
      // now server is running and any request that comes in
      // will be handled by requestHandler, and MULTIPLE
      // requestHandler's can be running at the same time,
      // handling different clients.
    

Am I missing something, or can this be achieved by building on top of the
yield primitive?

~~~
thejsjunky
The value is the way you can control execution: pausing, resuming, reaching in
to alter state. That's useful even if everything is synchronous. Sometimes out
of desire or necessity you need to do something async, and then you will
always have to have some sort of typical callback/async style code...but what
you can do with this is control where that code lives and express it in
different ways.

In your example, have startServer start a server, then on each request create
a generator, keeping a list of the active ones.

Your request handler would typically be written like this:

    
    
      function requestHandler (client, request) {
          sendToDatabase(request, function(data){
              client.sendReply(data);
          });
      }
     

What this allows you to do is write something like this:

    
    
      function requestHandler (client, request) {
          data = yield sendToDatabase(request, client);
          client.sendReply(data);
      }
    

Where the callback has been moved to the sendToDataBase function. It would be
something like this:

    
    
      function sendToDatabase(request, client){
          myDBLibrary.query(request, function(data) {
              Server.clientHandlers[client].send(data);
          }
       }
    

There's a bunch of different ways you could do it, but that's the general
idea.

------
lazyjones
If you want these JavaScript 1.7 features on the server side, you can use
RingoJS (<http://ringojs.org/>), which has supported 1.7 since 2008 or so (and
now sports some 1.8 features) ... It's a much less known alternative for
Node.js, better in several ways IMHO (e.g. multithreading).

------
VMG
Does anyone know how V8 is coming along regarding support of JavaScript 1.7?

~~~
jlongster
I can't speak to the overall progress (they've implemented a bunch of stuff I
know there are big missing features). As for `yield`, I spoke with the
developers a few months ago and they said that it's one of the next big
features they'll work on. Don't expect it soon though, I bet it will take a
while.

Considering that ES6 has pretty much been approved, and is expected to be
finalized in about a year, we will definitely see this in most browsers in a
year or two. It's a little ways off, but it's definitely happening.

------
vjeux
Using generators, you can also convert any callback code into a serial one.

Here's a demo: <https://github.com/vjeux/AsyncAwait>

Compare

    
    
      function main() {
        multiply(1, 2, function (res) {
          multiply(res, 3, function (res) {
            multiply(res, 4, function (res) {
              console.log(res);
            })
          })
        })
      }
      main() // 24
    

to

    
    
      var main = async(function () {
        var [res] = yield await(multiply)(1, 2);
        [res] = yield await(multiply)(res, 3);
        [res] = yield await(multiply)(res, 4);
        console.log(res);
      });
      main() // 24

------
firefoxman1
Why didn't they simplify it (and make it more like current JS) by leaving out
the .send() method?

Wouldn't this make more sense?

    
    
       var tester = test;
       tester();

~~~
kilovoltaire
I assume you mean:

    
    
      var tester = test();
      tester();
    

And yeah I agree, straight functions seem way nicer, which is what the
coroutine() wrapper in the writeup ends up accomplishing.

Javascript 1.7 copied the generator/coroutine objects straight from Python,
and it's a weird fit...

One thing that you can't do with just a function is have the .close() method,
which may or may not be a fundamental issue.

~~~
firefoxman1
I mean you could refer to the function by its name, and then run it with ()
when ready.

~~~
kilovoltaire
Oh, so you really did mean `var tester = test;`

The reason that might be less good than the current approaches is that you may
want to spawn multiple 'instances' of the generator/coroutine. If I'm
understanding your proposal correctly, you'd only ever be able to run it once.

~~~
firefoxman1
Oh, that's a good point. Didn't consider that.

------
rachelbythebay
This worries me. Imagine some fake code like this:

    
    
        function f(x, y) {
          switch(x) {
            case 1: goto a;
            case 2: goto b;
            case 3: goto c;
          }
    
          a: do_foo(y); return;
          b: do_bar(y); return;
          c: do_other_stuff(y); return;
        }
    

Calls to it would look like f(1), f(2, "cat"), f(3, "dog"). It's horrible, but
at least you see the first arg changing to get some idea of what's going on.

This yield thing makes it possible to call the same function with the same
arguments and get completely different behavior depending on how far along
it's managed to get.

In that situation, f("cat") != f("cat") != f("cat"), and that just seems scary
to me.

(BTW, I am aware that JS doesn't do goto. But this basically adds it in,
kinda-sorta.)

Like so many things, I'm sure it can be used for great evil or great justice,
but I don't have a whole lot of faith in people to do the right thing with it.

~~~
ufo
Isn't this just the same situation we already have right now where a function
can have different results if one of the hidden variables it closed over has
mutated?

------
orph
I wrote this Erlang-in-Javascript API 5 years ago using coroutines:
<http://beatniksoftware.com/erjs/>. I'm not holding my breath for pervasive
`yield` support.

------
spenrose
If this sounds like the Next Great Thing, you really want to read the Beazley
piece he cites at the end. Beazley comes to the conclusion that you want to
use these techniques sparingly. That said, I found Beazley's tutorials well
worth working through carefully. He finishes by implementing an operating
system kernel in cooperatively-concurrent Python :-).

------
vladev
Coroutines (ability to send values to generators, actually) have been in
Python for quite a while, but I'm yet to see a decent use of them... When you
add exceptions in the mix things start to get downhill.

------
halayli
yield != coroutine.

if f1 -> f2 -> f3 -> yield, you only go back to f2. Coroutines requires
yielding all the way to f1.

------
handler
so comprehensive!

i wouldn't mind a tl;dr ;?j

~~~
kilovoltaire
perhaps the easiest tl;dr is to try: <http://syzygy.st/javascript-
coroutines/#demo>

but as for an explanation: it turns out coroutines let you "invert"
memory/variable state into control/flow state!

~~~
bonobo
Nice explanation, thank you. I was indeed wondering if having something like a
statefull procedure (if I can call it this way, this is a fairly new concept
to me) wouldn't make the code more complex; I didn't actually notice that the
state is already there, it was just moved to another place.

