

An Inside Look at the (Python) GIL Removal Patch of Lore - mace
http://dabeaz.blogspot.com/2011/08/inside-look-at-gil-removal-patch-of.html

======
justinsb
I think this patch could certainly do with revisiting:

1) reference counting used atomic operations on Windows but on Linux used a
full pthread mutex; if that was necessarily back when the patch was written
it's definitely not necessarily today.

2) Garbage collection could eliminate the reference counting problem entirely.

3) There's been a lot of work on immutable data structures that could address
the dictionary locking issue.

I do wonder if this is really a deeper issue - programming isn't nearly so
simple once you have multiple threads concurrently modifying your data
structures and you have to start locking etc. A lot of the appeal of Python is
that it's so simple, and I think threads and locks would greatly affect that.

~~~
lobster_johnson
Yeah, a mutex to protect refcounts? That's ridiculous. (If you want low-
overhead locks, David Bacon's thin locks
([http://www.research.ibm.com/people/d/dfb/thinlocks-
publicati...](http://www.research.ibm.com/people/d/dfb/thinlocks-
publications.html#Bacon98Thin)) is a good place to start.)

I also question the logical premise offered here that data structures must be
thread-safe. It's not something guaranteed by most languages (eg. Java), so
why should Python guarantee it?

~~~
koenigdavidmj
Every Python object has a backing dictionary, so if any object has to be
shared between threads in a threadsafe fashion, then dictionaries need to be
threadsafe.

~~~
raymondh
This is tricky because dictionaries need to call __hash__ which can run
arbitrary Python code, so it's non-trivial to make the dictionary threadsafe.

~~~
dfox
It's perfectly possible and even reasonable to call __hash__ before acquiring
lock on the dictionary (and cache keys' hashes for data in the dictionary).

On the other hand, when almost every object is backed by dictionary, the
locking has to be very fast, which seems to me like almost unsolvable problem.

------
rbanffy
I like the idea of exploring other models. Shared data and threads is not the
only approach to parallelism.

And, usually, when you are stuck solving a hard problem, it often pays to take
a step back and make sure we are failing to solve a problem you should solve.
The problem we want to solve is not how to get rid of the GIL, nor improve
Python performance with threads but how to use Python more effectively on
multi-core/thread architectures and gain performance from that.

This is not a problem only Python has. The machines I work on most of the time
(a Core i5 laptop and an Atom netbook) rarely experience loads larger than 2.
There are simply not enough threads to keep them busy.

That's not to say they never get slow - they do - but I'd like to emphasize
that the limiting factor here is that we are not extracting parallelism from
the software already written. We stand do gain a lot from that.

