Completely insane that python decided to hijack "def" to be used to create an object that very much is not a function. At the least they could have gone was create a different keyword
It's a function, it's just been transformed into a function with a different return type (that of a generator or coroutine, you can see the type signatures here [1]). You could do something similar in vanilla Python rather than language-level sugar, eg using decorators.
I think the type signature and example makes the parent's point, frankly. It has three types, on of which is contravariant, two different usage of the yield keyword, and two exit points.
Now granted most uses of generators are simpler than that, but still that's a lot of special syntax. And if you are writing Python without type-checking, that's a lot of bookkeeping you'd have to do when reading code.
That type signature is gnarly, I agree. I've seen people get tripped up because they think the function is the generator, and are confused about what behavior to expect from concurrent readers of different instances of the generator (because they don't realize that they are different instances in the first place). So my position is more about how to work with Python effectively/understand it accurately than in defense of it's syntax.
What? There is no tragedy here. It sounds like it just put DX semantics ahead of…what? some sort of language purity? I can’t say that I’ve ever thought “gosh! that async function is…an async function, and that’s tripped me up!” before. Not even once.
The bigger problem with Python to begin with was that you have to use def for functions, meaning you can't inline them like with JS arrow funcs or C++ lambdas. And this isn't a language purity thing, it's just inconvenient.
Yes, I mean the async keyword fixes the hijacking although the hijacked version is still there for compatibility. So much for Python 3 breaking stuff for the purpose of fixing old cruft.
The tragedy is the whole Python 3 debacle, and if they were going to make a break like that, they should have introduced BEAM-like concurrency instead of async.
For reasons I don't understand, the Python community seems to have an intense phobia and hatred towards threads. I've always used threads instead of async though, and by writing in an Erlang-like style (message passing and no shared mutable data) you can keep things reliable. This has worked ok for me with 1000s of threads, though not millions or anything like that.
Python by now is mostly legacy cruft, it feels to me sometimes. Everything else of course has its own problems.
As someone who started learning to code with C++ followed by Python and then Rust, proper threading approaches weren't drilled into my head until I started working with Rust. I agree with you that threading with message passing is a safer approach. Threading does add a level of complexity that isn't always necessary.
AFAIK it's not outlined in a PEP, but my intuition is that Python was designed to be accessible and "simple". It wasn't built to be a systems programming language. So for all its faults with the GIL I don't think it should be a major point of contention. For any task it's always about choosing the right tool for the job.
There is considerable effort at the moment to remove the GIL in future versions and implement better threading support. At 3.11/3.12, it has come a long way in terms of performance.
The problem with threads in Python is they're 1:1 with OS threads in every major implementation. It's not like Golang where you have greenthreading, and you don't even get all the benefits of regular threads due to the GIL. It's been too long since I've used Erlang, but the way I remember it, the concurrency is almost like using RPCs. Which is great in some applications, but Python users want shared mutable data for simple things and will make microservices for larger systems.
For a simple language, Python has so many different ways to do concurrency, some of which are misleading to new users. Event loop ("async") would've been nice and simple from the start, but they were late to that party. Everything vaguely related to packaging is also confusing, from the __init__.py files to the PIP/Conda/whatever stuff. They had a whole breaking Py2->3 transition and still didn't fix that.
The GIL is an implementation artifact mostly due to the reference counting memory management. Besides phobia towards threads, the Python crowd also seems phobic towards tracing garbage collection. There is an ongoing project to make CPython run without the GIL yet still use reference counting. I wonder how many bugs this will surface or introduce.
Most concurrent Python code that I've written has not been cpu intensive, FWIW. Just an application talking to a bunch of i/o ports or sockets, but mostly idle. My CPU intensive stuff has tended to be single threaded so I can just launch instances in separate processes. YMMV.
Yeah the tooling situation is crazy enough that I don't begin to understand it.
Dunno if they meant this, but I feel that adding async to a language/ecosystem where it was earlier not used makes everything awkward for a long time. Especially if it's done liberally like python did, with both sync and async styles still supported.
It feels like the worst of both worlds at times. I think Rust suffers from this too, whereas Node.js has managed to make async the default mode of operation for enough things that it doesn't feel so bad. Especially with Typescript.
