It's a really exciting proposal. AFAIK, it'd make Python 3 the first mainstream imperative scripting language to have these async/await concepts built-in (there is a similar proposal for JavaScript/ES7 built on top of generators and promises[1]). While I've commented before about how asyncio is still a bit more nascent than I'd like, I'm really excited to see where the community takes it.
Mainstream is not a "moving goalpost fallacy". It was stated from the starting comment that he was talking about a "mainstream" language.
Of course the "mainstreamity" of languages is hard to determine, but Dart is so low in any kind of related metric that makes this a non-issue.
Let's put it this way: the Dart team announced a few months ago that they'd be concentrating on transpiling [1] and they won't put the VM officially in Chrome, and nobody gave much of a fuck judging from reactions in blogs, forums, HN etc.
If a mainstream language changed something, even something minor, there would have been blood (as it often happens).
An exciting thing that's happening here is that this adds another thing to Python 3 that is enticing to developers. There's been a lot of great improvements to the language in the 3.x line and the more things to lure people away from 2.x the better.
Counterpoint: I'm already uneasy about how complex the language has gotten. I like asyncio in principle, but here we see that addition spawning another addition.
Nah, asyncio (including both the library and these new language features) is a huge simplification of the ecosystem for people writing network code, while people not writing asynchronous code can pretty much ignore it if they want to.
Currently there’s a large split between folks programming against Twisted, Tornado, Eventlet/Gevent, rolling their own thing, or just writing blocking code and getting crap performance. (Not to mention Stackless, etc.)
These will all continue to work going forward, but they will now be interoperable and based on a common foundation which is solidly implemented and included in the standard library. Everyone wins.
It's interesting how over the past several decades we've learned a lot about which features should be part of a language's standard distribution and which can be farmed off to third-party libraries. String types, standard data structures, package managers, concurrency/async features, type systems, object models, date formats, command-line parsing, log libraries, parsers for the language itself, documentation generators - all of these can be implemented as add-on libraries and have been in at least one language, but we've found that this results in chaos in the language ecosystem. Similarly, protocol parsers, web frameworks, ORMs, command-line utilities (readline/curses, not argparsing), GUI frameworks - all of these have been wrapped into the stdlib of some library or other, but we've found that they aren't essential, and are often better off as a third-party package.
The one feature that IMHO should get baked into a language but nobody's done right yet is API versioning. Wish more people paid attention to the evolution of a program over time rather than a snapshot of where it is now.
Well, you're talking about things where time and experience has more or less found that having everything in a language working the same exact way makes for huge productivity gains and in most cases it's easy to escape hatch out if you really need special behavior, but most of the time you don't versus things where almost no one wants to use the same shit.
> ..., while people not writing asynchronous code can pretty much ignore it if they want to.
I don't think this is strictly true. If there is code which is available only in async form, then it's either unusable or awkward to use at best for someone using the non-async form. And vice-versa.
This is something I've found to be a problem in C#. Specifically, trying to use Microsoft async only API from sync code. And that async API really requires the whole stack to be async, so using the APIs from sync code requires hacks found at Stackoverflow.
It's a split in the code base, and extra work and complication for developers. I'm not saying that it's not worth it, or it shouldn't go in.
> If there is code which is available only in async form, then it's either unusable or awkward to use at best for someone using the non-async form. And vice-versa. [...] It's a split in the code base, and extra work and complication for developers.
This is already the case today. As I pointed out, there is currently a split into about 4–5 (at least) mutually incompatible versions of I/O and network protocol related code. The new features should cut that down to 2–3, which are easier to understand and reason about, to boot.
When I say people can ignore the changes, what I really mean is: to the extent they can ignore the complexities of syntax and semantics around asynchronous code today, they can continue to do so in the future.
Alternately, having language support and a solid built-in library might encourage some people to write their code using asynchronous rather than blocking I/O, without too terribly difficult a learning curve or too much mental overhead from the new abstractions.
What other option do they have to try to draw people over to Python3 that isn't a lot of work? I agree though. It's a kitchen sink rather than a smartly designed, concise language. I like to keep up on goings-on, but I work in a Python2 shop, I don't foresee myself or our business ever using Python3.
While I love Python2, my Python3 will be Go.
I like that Go is not a kitchen-sink language. I've also learned more from it, and I believe it adds more to my skillset and resume than Python3 ever will. That's my plan at least, to stop using Go just for fun, stop using Python for daily work, and use Go for fun and daily work.
That's when the time comes that Python2 is no longer relevant and doesn't have the library support it enjoys today. I expect 2.7 library support to last a very long time.
I believe 2.7 will be supported until 2020, and RedHat has indicated they might support it until 2027.
Having said that, porting your stuff to Python 3 isn't that bad. The automatic conversion tools get you 99.5% of the way, unless you're doing really crazy stuff.
I work in one of the largest Python shops in the United States. We don't have 100% test coverage (I don't know anyone who does). So there will be stuff breaking. With hundreds of thousands of lines of Python code, the testing/fixing burden would be enormous. It just doesn't make business sense to move.
This codebase will never be on Python3. And I think it would be a serious mistake even if it could be done, as the Python3 "migration" isn't done, until it's actually happened. Right now the numbers I've seen are that 10-20% of the Python userbase is on 3.x. The future for Python3 is sketchy at best, those support dates are nothing more than political propaganda. If they meant something, they wouldn't have already pushed it back to 2020. Even if it did matter, code doesn't just stop working when the ball drops in 2020.
So if I could snap my fingers and port us over to 3, and have no breakage (which would cost us millions if not sink the entire business)- I still wouldn't do it. The best strategy is keep on using Python2, which with ~80% of the userbase still at this point in 2015. Python 3.0 was released late 2008. It would be a mistake to switch even if it were risk-free.
We will be using Python2, and bringing Go in-house for new components where we would have used Python3 in an alternate universe where the migration wasn't botched by the CPython core dev team.
You know you can add a few "from __future__ import X" statements and start writing Python3 in your codebase today? That way you can at least take advantage of new language features.
Go mostly adds a lot of busywork for exchange of a speed bump (that you could get with another decent language with types, like D or Nim, if only they weren't unpopular).
Nothing about Go makes someone a better programmer the way learning C or Lisp or Haskell does.
This is true, but hopefully by the time Python library maintainers drop 2.7 support, the Go space will be more than ready.
Should be plenty of time for that to happen. There's probably a few decades of 2.7 library support, given the stats I've seen show 3.x at 10-17% of the userbase. Even if that doubles in 5 years (won't happen), it'll still be ~34% on Python3.
Which is so long from the 2008 Python3 launch date, and a small enough number migrating that we can effectively concede these are two separate communities going forward.
The majority popular Python libraries have gone the polyglot route of supporting both Python 2 and Python 3 from the same codebase, so its really not accurate to characterize Python folks as two separate communities.
Two problems with this. There is a very long tail of libraries on 2.x only that are not moving to 3. Second problem is that many 3.x only libraries are not widely used or tested and you'll find obvious errors that would not be there if they were widely used.
And as you said, some popular libs are 2/3 compatible.
So what should an end user do? Build on 3, which still has many things missing and less reliable 3.x libraries? Or stick with 2 which has everything the 3-only ecosystem has and much more.
3.x in reality makes no sense unless you have a political bone to pick. The big attraction was the unicode by default.
Python has supported unicode since 2.6. Python3 presents a new default way of handling it, nothing more. Py3 is pure technical churn at its worst. There's no innovation there. As a result the users outside of a vocal minority aren't coming.
When I started my current project two and half years ago I stuck with Python 2 because I was concerned about library support even though the major libraries I was using (Pyramid and SQLAlchemy) already supported Python 3.
While I think that was the right decision at the time, for me the balance shifted about 6 months. Needing access to the much improved multiprocessing library in Python 3 I ported my application to polyglot with the help of the python-future library. It really wasn't very difficult.
This was a modern codebase of about 20K LOC with reasonably high test coverage. For much larger or older applications then not porting is probably the right choice. But for new applications I think Python 3 is a fairly good bet now. Even if only 15-20% of the Python ecosystem have switched then that's still a significant enough userbase for most libraries to have had their Python 3 bugs shaken out.
For me specifically, StatsModels, sklearn, REPL plotting (more of a language feature) with seaborn/matplotlib, scipy, various NLP libraries, pandas for parsing CSVs and fast vector operations on table-like data, an easy to use requests library (compared to http://golang.org/pkg/net/http/) and 10^6 C libraries that either someone already made bindings for or that you can easily make bindings for with Swig/boost (not sure if it's as easy with Go).
Not to mention, all the "convenience" libs that are easy to implement but would take forever to make them all yourself. Examples like jaro distance, jaro-winkel distance, etc.
Unrelatedly, I wish Nim was more popular and I wish they concentrated on math more. Having a general purpose fast language that you can prototype in is amazing. Although you can't really blame such a small community for not focusing on a niche.
Julia might be great eventually, and it's really fun to write in and much easier to implement new things in compared to Cython, but not so great right now for my purposes. You end up importing the Julia library from Python and it takes a month to JIT compile some things. Also, string processing is not nearly as easy as it is in Python.
The problem for scientific users seems to be to achieve enough consensus about which language to use as an "obvious choice" that a bunch of other scientific users make packages.
Since we are talking about "fast" languages like Nim and Julia, I wonder why R and Python have done so well with the scientific audience but C++, Java, C# and the like never developed anything quite comparable. Or did they and I just didn't notice?
It would also make it the first programming language that starts with the letter P and ends with letter N to have those concepts.
Seriously, why does it matter? Is Python 3 really a "scripting language"? I don't think so. It's fairly close to, for example, C#, that has had these particular concepts for a while now. As have many other languages. Again, I don't think it matters that much who was "first".
I don't think it matters who's first either; I'm just excited that Python's one of the first languages to implement these concepts beyond C#. Only used the term "scripting language" as a way to say "lighter-weight."
What is the point of the community taking this anywhere when it is likely to be superseded by something else soon? It wasn't really that long ago that we got the original coroutines PEP and "yield from," now they are being obsoleted in favor of something that isn't clearly better.
The problem I have with async with is that you've already got issues with the syntax for "with" context managers and generators.
def my_generator():
with something() as bar:
for x in baz():
yield bar(x) # context manager will call bar.__exit__() before the second time through this loop.
It'd make a LOT more sense to just make "with" understand when it is in a continuation and only invoke __exit__() when the continuation is being destroyed.
Are you sure? I only see __exit__() being called after the loop:
Python 3.4.2 (default, Dec 23 2014, 17:01:26)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from contextlib import contextmanager
>>> @contextmanager
... def printme():
... print('enter')
... yield
... print('exit')
...
>>> def my_generator():
... with printme():
... for x in range(3):
... yield x
...
>>> for x in my_generator():
... print(x)
...
enter
0
1
2
exit
It seems to me like this is a feature that will only be used by 1% of Python programmers. I'm lacking a lot of context but personally I'd much prefer it if Guido spent his time fixing the Python 2/3 conundrum. And even though these proposed changes are Python 3 only, the fact that only so few people will use them doesn't do much in that regard.
I feel that concurrency is something used by far more than 1% of programmers. That it will be built-in the language itself, rather than through libraries, is pretty important in terms of delivering simpler code.
The Python 2/3 divide isn't that big of a deal anymore. This is not 2010. Almost all of the important libraries support Python 3; thinking especially to scientific computing. More and more distros are offering Python 3 by default. Big projects, like Red Hat's package manager, have switched to Python 3 (in Yum's case, by its replacement DNF).
It's only a matter of time before Python 2 isn't something that will be encountered in mainstream projects, only legacy systems. Which is nothing new (there's still lots of Fortran projects out there.) For them, the old version still exists. It will always exist. Should they want to make the jump to Python 3, the resources required are much less as compared to making the jump to another language.
Python 3 fixes fundamental flaws collected over the first twenty years of its inception. For those that cannot use a newer version of the language at this time, the old version is available.
The nice thing about Python generator functions was that they didn't need a lot of syntax - if a function contains the "yield" keyword, it's a generator, otherwise it's not.
Given how closely tied generators are with async/await, why do we need "async def"? Isn't the presence of "await" or "async for" or "async with" enough to mark a function as asynchronous?
A major motivation for the new syntax is to support refactoring (currently moving "yield" into a sub-function makes the calling function no longer a generator).
See the PEP's "rationale and goals"[1] and this article[2] summarising the PEP.
There's already a decorator @asyncio.coroutine which "async def" replaces. It's used for error checking, but also for some functions which are coroutines but don't use "yield" - they just return a Future
It's a really exciting proposal. AFAIK, it'd make Python 3 the first mainstream imperative scripting language to have these async/await concepts built-in (there is a similar proposal for JavaScript/ES7 built on top of generators and promises[1]). While I've commented before about how asyncio is still a bit more nascent than I'd like, I'm really excited to see where the community takes it.
[1]: https://github.com/lukehoban/ecmascript-asyncawait