GIL or no GIL, threads run concurrently in Python. The difference might be in performance:
- IO-bound tasks: GIL is released
- CPU-bound tasks on N-core on a single host. To exploit multiple CPUs:
a) no GIL (hypothetical): N times speed up (optimistically)
(it suggests weak data dependency i.e.,
multiprocessing can be used to the same effect)
b) option a) with multiple processes (shared memory or communication-based approach)
Code complexity is the same on average (except on Windows)
c) C extensions (existing or new): speed up 10*N or more on numerical code
Cython makes it easy to write new extensions.
Currently due to dynamic nature of Python, GIL or no GIL,
C extensions might be necessary to exploit hardware fully
(though C extensions might not be an option in some projects)
- scalable to multiple hosts tasks: different processes i.e., GIL is not a problem
Benefits of GIL:
- C extensions (and the interpreter) are simpler to write correctly.
Multithreaded programming is not trivial we need all the help we can get.
- no performance penalty for single-threaded code
- it encourages a synchronization through communication concurrent model (builtin in Go)
Disadvantages:
- some applications have no localized performance bottlenecks.
So writing a small C extension won't help to get possible
performance benefits of running on multicore in parallel.
If performance is critical; Python might not be the right tool in this case
- there are pathological cases when performance suffers greatly due to GIL
(though other approaches would have their own pathological cases)
- a (non-informed) perception that Python can't benefit from multicore
OK, sure, they run concurrently, but they can't be in the interpreter at the same time. The result is that using python threads to do python things, you won't utilise more than one core, most of the time.
There are a myriad of ways around it, but it is another thing you need to take into account when writing python programs or deciding to use python for a project.
So if someone writes code that makes heavy use of nonlibrary python code in multiple threads, you're automatically going to blame them as writing in the style of some other language? That's pretty No True Scotsman.
I'm not arguing for or against the GIL here but it's something to keep in mind. "Threads run concurrently" is misleading. And it's not a matter of being pythonic, the GIL isn't even a python feature.
