
Progress on the Gilectomy - etrevino
https://lwn.net/SubscriberLink/723514/4e850e6870991c76/
======
Animats
Multithreading in Python is all uphill. In Python, any thread can store into
anything at any time. By name, even, using "setattr". There's no thread-local
data. This implies excessive locking activity. That's why this guy is working
so hard to do things that ought to be simple.

A few restrictions on the language would make it easier, but when this is
suggested, Guido has a cow.

~~~
nine_k
Is there a link explaining the suggestions succinctly?

~~~
Animats
\- Multiple interpreters with mostly non-shared data in the same process.[1].

\- VecPy, for number-crunching.[2]

\- Newthreading, which I wrote in 2010.[3]

[1] [https://lwn.net/Articles/650489/](https://lwn.net/Articles/650489/) [2]
[https://www.andrew.cmu.edu/user/dfarrow/proposal.html](https://www.andrew.cmu.edu/user/dfarrow/proposal.html)
[3]
[http://www.animats.com/papers/languages/newthreadingintro.ht...](http://www.animats.com/papers/languages/newthreadingintro.html)

~~~
simcop2387
To be fair at least towards #1, that's a really difficult one to get right.
Perl has had that kind of threading for a long time and it's almost always
considered a bad decision to try to use them. I'm sure it could be done
better, but perl did it via a complete copy on write setup in order to create
the mostly non-shared data. But you end up with a lot of locking and other
contention for the shared data because there's no good way to expose the fact
that it's shared and changing it requires such restrictions. Instead the
method usually used is to take advantage of most OS's built in COW when
forking a new process and communicating via sockets, or some other method
instead that forces you to deal with the limitations (copying data, or if you
used shared memory you have to do explicit sharing and locking/serialization
yourself).

------
stickfigure
I don't really understand the problem with the GIL. Sure, people doing exotic
high performance stuff probably hate it, but Python probably isn't their go-to
language anyways.

For boring old web development and business processing, the GIL seems to be a
very efficient way of utilizing a CPU+caches without resorting to the horrible
event-driven programming model of Node. Want twice the throughput? Run twice
the processes. I kinda wish Java would bring back the option of green threads,
it was a perfect programming model for web development.

~~~
saghul
The problem is with CPU intensive tasks. Being as expressive as Python is, it
would be nice if CPU intensive algorithms could be written in it, without
blocking all other Python threads while the algorithm runs. For example.

~~~
crdoconnor
>it would be nice if CPU intensive algorithms could be written in it

It would be nicer if instead of trying to turn python into something it is not
(a language for writing CPU intensive tasks), it was made better at what it is
(glue code).

It would be especially nice if there were tighter, more seamless integration
between python and rust so that hot paths in python can easily be migrated to
rust.

~~~
sevensor
I'm not a Rust guy, so take this with a grain of salt, but I was under the
impression that Rust did a great job with C interoperability, which is all you
need for writing Python extensions. Some hasty googling brings up
[https://github.com/PyO3/setuptools-rust](https://github.com/PyO3/setuptools-
rust), which appears to be exactly the thing you're asking for. In other
words, it's not heavily advertised, but it appears to be entirely feasible to
build Python extensions in Rust, and not spectacularly complicated either.

~~~
steveklabnik
The language does do great at the core of it, but there's a lot of stuff that
makes it easier that's not the language's job.

compare [https://doc.rust-lang.org/1.2.0/book/rust-inside-other-
langu...](https://doc.rust-lang.org/1.2.0/book/rust-inside-other-
languages.html#ruby) to [http://usehelix.com/](http://usehelix.com/) for
example; the former is only using what the language knows, the latter is a
nice library built on top.

There's been some effort for nicer Python integration, but I haven't seen
anything super super slick yet.

------
krallja
Thanks for posting this summary. The video of the talk was posted in May, but
I never got around to watching it, because video always feels like it needs a
much larger block of uninterrupted time than reading.

------
akx
Previous thread (5 days ago):
[https://news.ycombinator.com/item?id=14432854](https://news.ycombinator.com/item?id=14432854)

------
absoluute
Getting rid of the GIL is easy - just move everything into the interpreter
instance struct and add an interpreter pointer as the first parameter in every
C API call except create. This should have been done in Python 3 since they
broke the API anyway. Now everyone will have to go through another painful
transition when they come to their senses. The current thread local storage
hacks and
PyThreadState/PyEval_SaveThread/PyEval_RestoreThread/PyThreadState_Swap are
ridiculous.

