
Concurrency with Python: Threads and Locks - yingw787
https://bytes.yingw787.com/posts/2019/01/12/concurrency_with_python_threads_and_locks/
======
1996
Having multiple asyncio loops and multiple threads do not mix well.

Too bad, because you want to isolate things sometime.

~~~
sametmax
It does after 3.5.3 and as long as you pop your loop after the thread has
started.

~~~
1996
In theory, yes it works in 3.7+

In practice, no

Can I ask you one example of what you mean? I want 2 separate event loop for 2
separate threads

------
aaron_m04
The Counter example has a bug -- it doesn't join the threads before printing
Counter.count. this could partially explain the low answer.

~~~
yingw787
Hi aaron_m04, I have posted a correction to the post and given you credit.
Please let me know if I have made any more mistakes, I don't want to mislead
any more readers. Thanks very much for your time!

~~~
aaron_m04
Great. Thanks for giving me credit and for updating this. Good work, by the
way.

------
stendinator
Global lock much?

~~~
chrisseaton
You can use threads, locks, and concurrency usefully even if you have a GIL.

~~~
amelius
Sadly, that's not practically possible.

Yes, you can use multiple processes to mitigate some of the problems, but you
will create other problems (for example: you now have to serialize all the
objects through a message pipeline, which is expensive, or you have to use
shared memory, which also requires copy operations, unless you want to write
your entire code in a low-level kind of way but in that case, you might as
well use a different language like C). In an existing system, moving
everything to a different process can be a cumbersome experience.

What you really want is multicore support with a garbage collector that is
concurrent and a runtime that doesn't have a global lock. Sadly, few
environments support this. Erlang comes to mind, but also doesn't support
structural sharing. Ocaml is working on multicore support [1]. GoLang, Haskell
and Java seem your best choice for GC+multicore.

[1] [https://github.com/ocamllabs/ocaml-
multicore](https://github.com/ocamllabs/ocaml-multicore)

~~~
nicolaslem
> Sadly, that's not practically possible.

Why? With cpython there are plenty of cases where parallelism is possible with
threads because the GIL is released (for instance most IO operations and many
C based number crunching operations), making threads and locks useful.

~~~
heavenlyblue
So technically if you wrote most of your number-crunching code as a C module
for python, then you could use threads...

Can't you see a problem here?

~~~
objectified
Lots of problems are merely I/O related, which can be solved just fine with
Python threads. As for number crunching (let's assume that means CPU intensive
tasks), you can always still resort to multiprocessing (which can also be
combined with multithreading, I fail to see why such paradigms cannot be
combined).

~~~
heavenlyblue
Well, io-related problems can be solved even better by using gevent.

The io-related problems I had with paramiko, on the other hand - last time I
checked could neither be solved with threads or gevent.

So that’s a weak point to be made anyway.

