

Concurrence: a Python framework to create massively concurrent network applications - anuraggoel
http://opensource.hyves.org/concurrence/

======
heyadayo
The concept is a neat addition toa growing list of python non-traditional
concurrency options: Stackless, py.greenlet/eventlet, kamaelia, fibra, and
twisted's defer.inlineCallbacks offer similar featuers.

Of that list, Concurrence is most like fibra and kamaelia, which also use
python 2.5+ generators to achieve microthread concurrency.

------
jcl
Correct me if I'm wrong, but it seems that this solution, like others, is
still constrained by Python's Global Interpreter Lock, ensuring that only one
thread per process can be executing Python code at any time.

If you want true concurrency of Python code (like if you wanted to run it on
multiple cores at once), you'd have to use something like pyprocessing, which
spawns off new processes instead of threads.

~~~
RoboTeddy
I think "concurrence" here refers to serving many requests concurrently, as
opposed to executing concurrently on multiple cores. But yeah, I think you're
right, it's not concurrent in the strict sense.

If you wanted, you could probably run as many separate python processes as you
have cores. I doubt it would help that much, since the main bottleneck in this
sort of application is usually the I/O involved in handling lots of small
packets.

When Facebook made memcached multi-threaded (it's also based on libevent),
things improved, but if I recall correctly, not that dramatically. They're
probably getting more benefit out of it now that they've hacked the kernel to
improve the packet/second bottleneck issues, but for most people that's
probably not worth it.

------
durana
"It takes a Lightweight-tasks-with-message-passing approach to concurrency."

As someone who really likes Erlang's approach to concurrency, I think it is
very cool to see it starting to become available in more and more languages.

~~~
jerf
Unless I'm very much mistaken, this does not implement "no shared state
between threads/processes/whatever", which I consider a bare-minimum
requirement to actually have Erlang.

You can try to discipline yourself to use such a philosophy anyhow, but Python
is going to be one of the worst places to try that. It is mutable to its core,
which is one of the reasons it is cool, but doesn't lend itself to this
approach very well.

~~~
durana
Yeah, what I really meant by my comment was it is cool to see this style of
programming (lightweight procs & messaging) becoming easier to do in more
mainstream languages. While implementation details differ greatly, I think it
still gives a lot of credit to the guys who developed Erlang.

~~~
ralph
Are you aware that using channels for communication and synchronisation has
been prevalent in languages other than Erlang? Occam was one, and Bell Labs,
creators of C, Unix, etc., used it in Alef and Limbo amongst others. They all
hark back to C.A.R. (Tony) Hoare's CSP. Here's a good overview.
<http://swtch.com/~rsc/thread/>

