Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The usual answer is: in the case of blocking I/O, the thread running send/recv can block while other python code runs.

In practice, this doesn’t work particularly well, as you rarely have massively I/O bound things in Python.



Not sure what kind of applications you work on, but most web apps are IO bound (waiting on network calls).


I'm not the person you were originally responding to, but I know Python is popular in data science and such where there would be a lot more tied up in pure computation/number crunching.


I know that you already got an answer to this effect, but to provide some more information, most data science workflows in Python rely heavily on calls into numerical libraries (numpy, scipy, pandas, tensorflow, pytorch, matplotlib) that are Python wrappers over compiled binaries (mostly C and Fortran, a not-inconsiderable amount of handwritten Assembly), that have been constructed so that the wrapper safely yields the GIL before invoking the underlying binary. This is all the more important when considering libraries like tensorflow or pytorch that may involve complex long-running interaction with training resources across a network. Control is yielded to allow the interpreter to continue carrying out tasks like displaying the ongoing training progress, or loading training data.


To that point, most third-party libraries used also delegate the number crunching to C/C++ code, so most time is still spent outside the GIL.


I should have clarified: when I meant "doesn't work particularly well", I meant that its incredibly difficult to saturate say a 100 thread box. (Which with Rome isn't crazy!)

Only being able to run a single thread of "logic" basically makes that impossible, as you need to usually do some computation to figure out what bytes to send/receive, do something with the results, and so on.


The program doesn't have to be massively I/O bound for threads to be useful. Even with only a UI thread and a single background thread they're useful for animating a progress indicator in the UI thread while waiting for a slow network request to finish in the background thread, which works just fine in Python.


> you rarely have massively I/O bound things in Python.

Citation needed.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: