
Multiprocessing.shared_memory – Shared memory for direct access across processes - pjmlp
https://docs.python.org/3.8/library/multiprocessing.shared_memory.html
======
sametmax
Great, the support is still basic, but it's a huge improvement over message
passing when you need speed. 2 months ago a student asked me exactly that
question: how to share data between processes, and when I told him that with
the stdlib he had to manipulate mmap by hand
([https://blog.schmichael.com/2011/05/15/sharing-python-
data-b...](https://blog.schmichael.com/2011/05/15/sharing-python-data-between-
processes-using-mmap/)), he was not happy.

Now that asyncio is production ready (we really have had a decent doc and
asyncio.run() for a short time), and this, the concurrency story in Python is
starting to look decent.

Since the 3.6, things are really going nicely.

Maybe we'll see an async django-like framework emerging from now on instead of
the myriad of async flask copycat and django-channels. This would open the
gate to live settings, manipulating tasks queues in URL enpoints, having
RPC/PUB-SUB baked in. After all, I don't need async to just have exactly the
same features as a WSGI framework.

~~~
j88439h84
I like [http://trio.rtfd.org](http://trio.rtfd.org) way more than asyncio.

~~~
sametmax
Me too, although the scheduling syntax is not to my taste. But trio is not in
the stdlib, and is incompatible with most of the current ecosystem. There are
talks to integrate it in the future, meanwhile there are compat layers, but
it's still in infancy.

For now on, I just use asyncio with a lib to add nurseries to it.

~~~
j88439h84
I'm curious what you dont like about the syntax, and which parts of the
ecosystem you're waiting on?

------
ldng
Now combine that with Apache Arrow and you can do some interesting things.

------
malux85
This is great! As someone who codes lots and lots of multiprocessing + queue
code I'd love better shared memory support

------
neurohacker
This rules, but until 3.8 is out I've been using
[http://semanchuk.com/philip/posix_ipc/](http://semanchuk.com/philip/posix_ipc/)
which unfortunately doesn't work on Windows.

------
jmakov
Doesn't Apache Arrow solve this problem partially already?

------
zozbot123
Both 3.7 and 3.8 have so many performance improvements. But still, what people
like most about Python is code readability. Performance isn't as important as
in the past, because of how hardware performance has scaled.

~~~
gpderetta
wait, hardware performance hasn't really progressed much in the last 10 year!

~~~
anonymou2
Not CPUwise. But they have increased the number of CPUs available and that's
where python has problems because of the GIL. Fortunately the multiprocessing
library seems like a good workaround to the GIL issue.

~~~
gpderetta
Agree, which means that python actually got worse!

