I find a lot of 3.x new features to be improvements from 2.x but everyone still uses 2.x. I fear that we will be on Python 4.x by the time 3.x adoption starts to actually happen en masse.
From what me and my coworkers have discussed we are basically all waiting on Django to get Python 3.x adoption. I've tried submitting multiple patches for this and helping Django as much as possible but they don't seem to want outside help.
Come on Django, come on, a lot of us are waiting for you.
As I've said pretty consistently: getting Django onto Python 3 is not a technical code-porting issue. In fact, experimental Django-on-Python-3 branches exist and, so far as I'm aware, run pretty well.
The problem is and continues to be the social and community issue of gracefully deprecating our way out of Python 2.x, which we're basically doing at a rate of one Python version per Django release. All the patches in the world can't really make this process go faster :)
And this is understandable, but as long as the branches for 3.x Django are "experimental" or created by a third party I'm not willing to put them into a stable production environment. Maybe Django just needs to come up with a 3.x branch to develop as stable along side the 2.x branch till 2.x starts to die out?
The situation hasn't changed. If you want to work with Django on Python 3, go ahead; the code's right there for you. We need the help: it's going to take a lot of burn-in to make sure we get everything right.
If you're waiting for it to hit trunk, fair enough I guess (though it's a purely arbitrary decision on your part; the only difference between the branch and what'll hit trunk is the URL). That should happen shortly after 1.4 ships, so quite soon now.
Lot's of huge working code bases (in-house apps) are using Python 2. It works, it's been tested, it's in production daily. What's the compelling business reason to change? And 2 to 3 is a substantial change. If we're going to do this and pay the cost to do so, we may as well investigate other options (outside of Python) while we're at it.
I'm not sure there's a compelling reason to upgrade old, working code to Python 3. The cost of having Python2 and Python3 installed on the same machine is minuscule. The hardest part is making sure scripts use "#!/usr/bin/env python2" or "#!/usr/bin/env python3" appropriately. Even on Windows, the "Python launcher for Windows" [1] will launch the correct version of Python.
My rule of thumb is write new stuff in Python 3, keep old stuff on Python 2 unless I have to rewrite it for some reason anyway.
Upgrading just to upgrade seems like developer problem more than a technical problem.
Yes you should investigate other options outside of Python, on a regular basis I keep up to date on other frameworks/languages and on a regular basis I decide that Python is still the right choice for me. If you aren't doing this on a regular basis (at least once a year seriously considering where your framework/language of choice is on the market) then I think there is a serious problem. The CS world changes so fast that you always need to consider if what you are doing is simply outdated and you should pay the cost to stay current. However if that cost still isn't worth it and everything is working on your outdated, legacy hardware and software and you can still keep up with the market than you're good.
I simply like to keep myself up with the market/community, when I release new software I try and make sure I'm not releasing something on a platform that will be dead in a month.
then there's appengine, just now got up to 2.7. it will probably be another few years before it's on 3.x. doesn't fedora still ship with 2.4? ideally we would all like new toys, but reality sucks most of the time.
You're confusing Fedora and Red Hat. Fedora is generally very cutting edge and has been shipping with 2.7 for quite a while now, Red Hat is far more conservative and was, at least up until the latest release, shipping with 2.4
I am not really that excited about any of these features that's in 3.3 (or even in Python 3).
The biggest problem I have is that Python 2 works "good enough" and upgrading to Python 3 does not have many benefits, other than lots of work and slower performance. I guess a lot of people feel the same way, hence the slow or non existing adoption of Python 3. I guess Python 3 might end up as Perl 6 if this continues since we keep investing time on Python 2.x libraries and features.
What we need is a reason to upgrade and the killer feature of Python 3 could be performance. I would upgrade to Python 3 in a instance if it offered a lot better performance than Python 2.
How do we achieve better performance? Adopt PyPy as the reference implementation for Python 3 and all future references. Making PyPy a centerpiece of Python 3+ would probably mean it would mature faster.
PyPy is focused right now on Python 2.X, but I would rather see it focus on the future of Python and being a killer feature of Python 3+.
I find your claims are unsubstantiated and would require some evidence to be convincing.
For example, the performance of Python 3.0 was slow on I/O operations. The reason was because the I/O modules were written in pure-python in order to get the ball rolling. This wasn't a bug. However, the code was eventually ported to C in 3.1 and unsurprisingly the performance improved: http://www.gtsystem.eu/blog/2009/08/improved-performance-of-...
And that's an old post from 2009! The performance is much better today.
Features they're releasing in this new version include zero-copy in os.copyfile() which is totally sweet. No user-space buffers involved in transferring bytes from a file to a socket. This should speed up such operations well past what any version of python 2.x is capable of.
There isn't a feature in Python 3 that is not substantially better than 2.x -- they're fixing all the fiddly bits and producing a much better language. There is no feature they can create which will open the dam to a flood of porting projects. The reason is that the problem is a social one.
An honest-to-goodness effort to port the large libraries requires a plan to support the existing 2.x versions alongside the 3.x versions so that the people who are still running applications built on the old versions will still work. This requires extra effort that is difficult to acquire in open source projects let alone commercial ones.
Even if the Python 3 devs added a feature that materialized ponies out of thin air there would still be projects running the 2.x code.
> I find your claims are unsubstantiated and would require some evidence to be convincing.
It's been 3+ years since Python 3.0 has been released, and, while I agree that there's no "substantiated evidence" for its demise, some of us who witnessed the whole Zope2 to Zope3 "transition" know that in the end "evidence" doesn't count at all. What count are actual smart people, lots of them preferably, building cool and useful things with the new tool.
Yep. I won't port a single bit of my existing code until 100% of the libraries I depend on are ported first. As much as I like Python 3, it just doesn't make any sense for me to deal with it right now.
Part of what I was trying to get across is that there will never be a compelling reason to make the port. It will never "make sense" so long as what you have now continues to work.
If you're ever going to make the port, you have to decide to do it and make the effort. Nothing the Python 3 devs release will ever convince you to port otherwise.
Python 3 has been making performance improvements over 2.x. 3.2 included the new GIL. 3.3 includes more memory-efficient Unicode strings.
Pypy is focussed on supporting 3.x. There are two major initiatives going now, one of which is numpy support and the other is Python 3 support.
People aren't "investing time on Python 2.x libraries and features". Python 2 is going to just receive security and bug fixes and all new work is on Python 3. Major libraries are currently supporting Python 3, including almost all of the widely used ones (numpy, scipy, sqlalchemy, virtualenv, etc). The ones that aren't, are being ported (django, twisted).
Overall, the tone and information in your post is quite outdated. Your sentiments were shared by a lot of people circa Python 3.1, but now it is 2012 and the Python ecosystem has mostly got a handle on how to straddle two versions, after being help up for a bit by discovering that the initial "lib2to3" approach wasn't the best solution for many projects.
PyPy has folks actively working on getting Py3k support. Check out their status updates on py3k and consider donating to the PyPy project to make this happen.
At least Perl 6 is aiming for some amazing advances that will trade off quite well (if it is ever delivered) against breaking compatibility with Perl5.
Py3 broke compatibility (at least at the extension layer) for what exactly...now I can say print() instead of print ""? What a joke.
Should we develop on Python 3.x series or 2.x series?
Libraries aren't really ready for 3.x series and developing on 2.x series is not making any sense as if 2.x ultimately goes away then migration effort on that is not worth.
At the same time 3.x ecosystem is not ready either.
The general advice I give is: "Python 3 if you can, Python 2 if you must."
That is, if you can use Python 3, you should: it's the future of the language, and it feels it. I find Python 3 significantly more pleasant than Python 2; nearly all the bits that annoy me about Python are fixed in 3. Python 3 is cleaner, the standard library is de-crufted and easier to navigate, and (most important to me) handling unicode doesn't suck.
However, as others in this thread note, significant parts of the Python ecosystem haven't finished porting yet (Django, Twisted, yada yada yada). If you rely on one of those things, then stick with Python 2. If that code lives fo a long time you will have to face a porting effort later on, but that's improving somewhat as the community discovers new techniques and creates new tools.
Most significant libraries and tools work with 3.x by this point. SQLAlchemy, numpy, scipy, cherrypy, WebOb, pymongo (as pymongo3), virtualenv, pip, pygments, jinja2, tox, nose, ...
It's too bad that django, zope and werkzeug (flask) aren't there yet. I understand that a lot of people on this site use django and so these discussions are always dominated by people complaining that they can't use Python 3 because their dependencies (django) aren't available.
But the Python community is much bigger than django and today most of the rest of the ecosystem is ready and working fine. The parts that aren't, are getting there rapidly.
Note that he advises leaving print statement be; it's more familiar in 2.x land, editors often are confused with print function syntax highlighting, and it's easily handled by 2to3.
I don't think they do, myself. Adoption will and is happening gradually. More and more libs are being ported, and new Python coders often learn "Python 3 and how 2 differs" (rather than the other way around), e.g. thanks to Lutz' book. Eventually there just won't be any reasons left to use 2, and the usual reasons to use the latest version kick in - incremental improvements and continued maintenance.
The migration was always expected to be a long-term thing, and so far has been pretty pain-free imho. But then I haven't written any code that doesn't run in 3.x since 2010, finding that 3 increasingly meets my deps and is more enjoyable and productive to code in, and when I do need to support 2 that the common subset of 2.7 and 3 is large enough to do many things.
Unicode strings by default is a big one for me, but the little things sum up too: The removal of x*() functions in favor of making things that should return iterators just do so, removing old-style classes, various little consistency gains in standard object interfaces/protocols, lots of standard lib organization cleanups. I simply find things in Python 3 are the way I expect or want them to be more often than in 2, reducing the number of gotchas I need to keep track of. Heck, even print() is a solid improvement in my book (keyword args, hooray!) and I appreciate not needing the __future__ import for it.
The list gets longer of course if you fairly add all the things eventually backported to 2.7, e.g. set literals and comprehensions.
So, does 2.x work? Sure. Is 3 better? Yup. If I get to pick one, I pick 3.
Unfortunately, Python 3 is the new Perl 6. It offers reduced performance[1], changes some of the hardest-to-debug parts of the language (like Unicode), and capriciously introduces backwards incompatibilities. For no obvious gain!
As just one example, could Guido not have made the print statement a function through the simple mechanism of:
from __future__ import printfn
Then new code could use printfn and old code could use print, and all would be well. But instead, literally billions of dollars will be wasted changing every single print keyword statement to a print function invocation. Sure, you can use 2to3. But the tool is that worst of all worlds, a compiler that doesn't really work 100% and actually reduces the performance of your code in the compilation process.
For those who do not want to waste their time on this port, Python 3 is going to accelerate the move towards full stack JS just like Perl 6 accelerated the move towards Python/Ruby. This JS stack will be MongoDB + node + Backbone/coffeescript or some similar permutation.
After all, if you are going to literally have to recompile your language with 2to3 and deal with all the edge cases, why not go whole hog with something like emscripten[2] or a souped-up pyjamas[3] to convert your legacy Python 2 code to JS? Then people just need to start porting C/C++ numerical libraries over to V8 through something like v8-juice[4] and we will have all the missing pieces.
I'm pretty sure they'd have announced it if Python 3 ever got faster than Python 2. But this is not mentioned in the 3.1, 3.2, or 3.3 notes on performance:
That one is easy. The Python 3 conversion is a serious project for core devs at Django and Numpy. Check out their mailing lists about what this takes/took.
While the above procedure of using the 2to3 tool works
relatively for pure Python code, there is no automatic
conversion tool for extension code. NumPy is mostly
written in C and makes extensive use of the Python C-API.
So converting NumPy will require much more than running
the 2to3 tool.
... it will be a while before most Python software catches
up and is able to run on Python 3.0. For larger Python-
based software like Django, the transition is expected to
take at least a year or two (since it involves dropping
support for older Python releases and so must be done
gradually).
A "year or two" of the core Django team's time is easily several million bucks. Multiply by the Python codebases of every company on this list [1], and everyone on github, and every egg out there, and we are talking billions for the conversion.
From the full article, "Python 3.2 [beta] looks promising coming out just a little faster than Python 2.7.". This same article has PyPy coming in at 3 times as fast as Python 3.2 and Python 2.7 on the pystone benchmark, but if you have used PyPy it is not fast.
Looks like we need another benchmark. Does anyone have the Python 2 code from the Shootout? It seems to have been removed so that you can no longer compare Python 3 to Python 2 directly.
I am positive that Python 3 could be a lot faster than Python 2. If people are interested in what's possible, they should follow the corresponding mailing list (python-dev).
Jacob, there look like about 20 or more programmers on that page. At even $100k per programmer, isn't that about $2.4 million? And isn't the number much more than that given that you are among the most skilled engineers?
* We have about 30 people with the commit bit; at any given time I'd guess about a dozen are active.
* "Even 100k" implies that you think a six figure salary is reasonable for programmers. This is true in some rarefied echo chambers likethe Bay Area where programmer salaries are overinflated, but it's far from a universal The majority of our committers don't live near San Francisco. Actually, I think most live outside the US these days. I don't believe the median salary of a Django committers is 100k -- though for my fellow committers' sake I hope I'm wrong :)
* I only wish "being a core Django committer" was some magic "Get a Great Job Paying Six Figures Card". No good hiring manager assumes that "has a Django commit bit" translates to "among the most skilled engineers"; I only wish I could pull down "much more than" 100k based solely on my Django status. It's certainly a good time to be a Django dev, but while being contributor might help you get your foot in the door, it won't land you a salary increase over your peers.
* None of us get paid to work on Django anywhere close to full time. We get paid to do other things; we're lucky if we get a few hours a week to hack on Django as a side project. This again shows that our value in the salary marketplace isn't linked to our willingness to volunteer time to Django.
* Finally, this whole silly exercise assumes it would take "a year or two" of ALL of our time to get Django onto Python 3. That's laughable: it's mostly done already with perhaps a couple of man-months of activity. There's nowhere near 20 programmer-years worth of work; suggesting such is simply insane. Toss me that 100k you're bandying about so easily and I'll have it done by May :)
You seem to be making the assumption that many of those people are working on Django full time. It's pretty clear from their descriptions that most of them have other day jobs.
Also, in the original "year or two" mention, that's wall-clock-time, not applied-effort-time. While waiting for the world to adjust at a gentle pace, plenty of other productive work will be underway.
> I'm pretty sure they'd have announced it if Python 3 ever got faster than Python 2. But this is not mentioned in the 3.1, 3.2, or 3.3 notes on performance:
Do these documents really need to come out and spell it out for you? If you were to read what the document is telling you, you could probably figure it out.
> That one is easy.
Your original comment was about the print statement changes costing so much time and money, but now they are about the entirety of Django and NumPy. Yeah, it certainly costs money for a big project like Django to port (and people have completed it in their free-time), but print function/statement changes are trivial and do not cost on the order of billions of dollars.
From what me and my coworkers have discussed we are basically all waiting on Django to get Python 3.x adoption. I've tried submitting multiple patches for this and helping Django as much as possible but they don't seem to want outside help.
Come on Django, come on, a lot of us are waiting for you.