
Offset: Goroutines ported to Python - scardine
https://docs.google.com/file/d/0B2gNkR7hgkIRVFgwSWR4Y2JaZHc/edit?hl=en&forcehl=1
======
comex
Mildly related - it's been my opinion that Go's M:N threading system should be
considered a workaround for a missing OS feature: the ability to manage and
schedule native threads in userspace, without incurring a context switch.
There are many reasons why goroutines would be better implemented in
cooperation with the kernel - better debugger support, thread local storage,
probably less scheduler overhead - but an important one is that it would make
them easier to port to other systems than the Go runtime, such as this one.
This is not just because they wouldn't have to reimplement the scheduler
(which could be addressed with a regular library), but because they wouldn't
have to contort every interface that touches syscalls into taking the userland
scheduler into account. No need to put syscalls in their own threads or pass
the context object around or anything, the kernel would just return to another
goroutine while working on a syscall, just like any other thread.

I can dream.

~~~
MetaCosm
Not really a work-around, even with great native thread support and tooling as
you dream of (and don't get me wrong, I am all for it), ultra-light
"goroutines" or "erlang processes" fit a different niche, you are never going
to spin up 3+ million OS threads, you absolutely can (and do) in the ultra-
light process space.

~~~
comex
Having 3 million threads doesn't change the usefulness of being able to call
into native syscalls and native libraries that use TLS, locks, etc. It does
change the usefulness of native debugger support, since existing debuggers
would have a rough time with 3 million threads, but it's not like goroutines
currently have any solution to that - Go just uses a GDB script to implement
goroutine commands similar to the thread ones. So I think it should absolutely
be possible to spin up 3 million OS threads. The kernel needs to change to
trust memory mapped into userland to keep track of threads rather than
tracking all the data structures itself, and the toolchain needs to support
split stacks. This would require a lot of change, but the result would be a
lot cleaner than having two strongly overlapping concepts of threads.

------
oddurmagg
Stackless Python support Go style concurrency with tasklets( goroutines in go)
and channels for scheduling and data passing between them.

Pypy now has stackless support, which makes it the obvious candidate for doing
CSP style concurrency with python.

It will not however, scale that concurrency to execute in parallel in multiple
cores, since it is still bound by the GIL.

If you want to stay with CPython, you could look at Gevent.

~~~
benoitc
gevent is based on an eventloop to do the scheduling, it's quite different
than the scheduling done in offset and offset will soon support the
possibility to launch your goroutin on different OS processes for //. More
over gevent monkey-patch the standard library which make sometimes the
integration with your code difficult, offset however use its own syscall
module.

Also the implementation of channels or stackless in pypy is a little different
than the one you can find in offset (or go for that matter). offset channels
support buffering, and you can also select them. offset like the
implementation of stackless in pypy is using the continulets to do the job
when you are using in pypy and python-fibers on cpython which bind the
stacklet lib from pypy as well.

------
jzelinskie
Is this specifically for CPython? I thought Stackless Python already provided
something similar barring Go-specific things like channels.

~~~
benoitc
It is also working for Pypy. this is a different impementation from Python
Stackless and also give you the possibility to select operations just like in
Go.

------
emilyst
Anybody have this content in a form that doesn't require a logged in Google
account?

~~~
deckiedan
Select 'File->Download'

~~~
emilyst
You can't see any menus, any content, anything but a login page.

~~~
benoitc
odd when i'm not logged to speaker deck I'm able to see the slides and
download the PDF on the left :/

