
Python co-routines: follow-up to Python generators presentation [pdf] - irahul
http://www.dabeaz.com/coroutines/Coroutines.pdf
======
irahul
Python generator tricks is garnering attention
<http://news.ycombinator.com/item?id=3349429>

This is a follow-up presentation.

I would advice against using raw co-routines for co-operative multitasking
though, barring a few special cases; gevent/greenlet will be a better option.

------
cocorroto
In section 9.1 and 9.2 of Programming in Lua,
<http://www.lua.org/pil/9.2.html>, there are some explanations of coroutines,
and the example of consumers and producers. Also there is a comparison with
pipes, a key point is: "The cost of switching between processes is high. With
coroutines, the cost of switching between tasks is much smaller (roughly the
same cost of a function call), so the writer and the reader can go hand in
hand.

------
nigma
I'm observing Python language elements being rediscovered on HN recently.
These things have been in use for a while now and I'm wondering why they are
getting so much attention several years after they were first introduced.

Is this because you are new to Python (or programming in general) or because
these concepts and use cases were never explained properly, or you simply like
Python and up-vote related stories?

(I'm actually asking myself similar questions about PostgreSQL stories as
well.)

Anyway, I encourage you to take a look at
<http://docs.python.org/whatsnew/index.html> (and
<http://docs.python.org/dev/whatsnew/index.html>) from time to time.

~~~
jflatow
It's not specific to Python, it seems to happen in every topic. On a related
note, it would be nice to finally see that new posts of the same link would be
tied to the other instances (or rejected).

~~~
scott_s
The same URL can't be submitted twice. If you submit a dupe, it just gives a
point to the original. Note that for the generator tricks submission, the
submitter put a "?" after the URL to get it past the dupe-checker. Other
tricks people play are to link to different parts of the site (say, the page
containing a presentation instead of the presentation itself).

------
atdt
I find generators powerful and I use them all the time, but I haven't found a
use for coroutines in my code yet. Maybe I don't grok them fully.

~~~
j2labs
Coroutines + a scheduler become an awesome replacement for threads. They're
lighter weight and don't thrash the GIL the same way Python threads do.

Greenlet is an optimized implementation of coroutines for Python. Gevent or
Eventlet combine greenlet + a scheduler and even monkeypatch your blocking
calls into nonblocking ones.

I use eventlet for my load testing tool, Micro Army, to create nonblocking SSH
pools. I can SSH to 2 boxes and run config scripts in parallel in 58 seconds,
but I can SSH to 100 boxes and run the same tasks in 106 seconds.

The code couldn't be simpler too. Here's the proof.

SSH details for a single host:
[https://github.com/j2labs/microarmy/blob/master/microarmy/fi...](https://github.com/j2labs/microarmy/blob/master/microarmy/firepower.py#L149)

SSH to every host in a list:
[https://github.com/j2labs/microarmy/blob/master/microarmy/fi...](https://github.com/j2labs/microarmy/blob/master/microarmy/firepower.py#L188)

Notice that the code is basically just spawning a coroutine and iterating
across a list of coroutines. Done. No callback spaghetti, yet all of this is
async and nonblocking.

~~~
pault
Very interesting. Does Greenlet use threads behind the scenes? Do you run into
the same problems with deadlocks, debugging, etc?

~~~
anonymoushn
It's single-threaded. I haven't looked under the hood, but I would expect it
to work by copying stacks around. In my experience it's been very easy to use.

------
anonymoushn
I'm surprised how much he accomplishes without the ability to yield from a
nested call.

