

Django's future, and Python 3   - jimmyjim
https://www.djangoproject.com/weblog/2012/mar/13/py3k/

======
randlet
Great news. The two biggest hurdles for Python3 adoption were/are Numpy/Scipy
and Django. Now that Numpy/Scipy both officially support Python 3.x[1] and
with Django coming soon, I expect a lot of people will make the transition to
Python3.x for new projects.

1\. [http://scipy.github.com/faq.html#do-numpy-and-scipy-
support-...](http://scipy.github.com/faq.html#do-numpy-and-scipy-support-
python-3-x)

~~~
togasystems
Do you know of any other large hurdles preventing the adoption of Python 3?

~~~
SiVal
It may sound strange, but we're the other way around. With Python 3 being the
only future of Python, the inability of something like Django to support
Python 3 is, for us, a hurdle to adopting Django, not Python 3.

~~~
peteretep
This has retarded any progress of mine toward learning Python. Every so often
the urge strikes me, and then I remember that I want to learn Python 3, and
then I remember that means no Django, and I tend to go and learn something
more fun and less practical...

~~~
lloeki
> learning Python

By any means, if you feel the urge, you really should help yourself and
scratch that itch.

I understand that for someone new to python (hence lacking the knowledge to
get what's different between 2 and 3) you might feel uneasy about it, but
really you should not.

Just learn py2.7 (+ django if that's you want) _now_ , and bulletproof your
code by using _u""_ , _b""_ , _.encode_ and _.decode_ at the edge when needed,
also _123L_ in your code, together with _from __future__ import division_ ,
_print_function_ , and whatnot. The remainder is mostly syntax (like _except_
) or namespace (like _http.server_ ) changes and _will_ get caught and thus
easy to fix when you try to run in py3 (contrary to if you don't use u""/b""
as values can "move" through code and blow up in a completely unrelated area).
I might be forgetting something, but that's the gist of it.

As for learning, really you will have _zero_ trouble acclimating yourself to
any breaking change py3 introduces. The trouble only lies in _existing_ ,
'legacy' code, not for the one you write from scratch straight in py2.7, and
especially if you aim for compatibility with the provided forward-
compatibility features of python.

~~~
maaku
Great post. A point I want to emphasize is that learning py2.7 _is_ learning
Python 3, if you are careful to pay attention to the recommended/pythonic way
of doing things.

For the most part Python 3 removes syntax and functionality that was long-
since depreciated or at least strongly advised against.

~~~
sirclueless
Perhaps, but the lack of bytestrings seems to be what bites everyone. It's the
only "deep" change and it affects anyone who wants to slice raw bytes (which
includes most web frameworks).

~~~
koenigdavidmj
Python 3 has bytestrings. They look like this:

    
    
      b'foo'
    

And you can slice them up as you wish.

------
pilif
I think it's crazy that they are only now talking about dropping support for
Python 2.4 which came out in 2004 and has been replaced by Python 2.5 in 2006.

So only now they are moving from depending on 8 years old technology to
depending on 6 years old technology.

The reason for this is probably the proliferation of really old versions in
the default installation of still supported OSes like RHEL. Worse: Due to the
fact that these distros largely depend on the Python installation they came
with, it's likely that admins are reluctant to touch anything python related
on the system.

Compare this to ruby/rails where the common practice is to treat system ruby
as a hands-off thing: Usually, the deployment payload just includes its own
ruby (using rvm or rbenv). This allows for much less conservative minimum
requirements.

Being forced to constantly target way outdated software during development is
really annoying and kills motivation as you are always reminded that if only
you could update, you would be spared re-implementing feature x or you could
use much more expressive code while implementing feature y.

And don't get me started on PH-4.x is good enough-P.

~~~
Arcanum-XIII
Python 2.4 seem to still be a version you can find on "recent" system, like
CentOS 5. So it made sense to be compatible with it, don't you think ? And RoR
was conservatively still working with ruby 1.8 too... I know not of any
equivalent of rbenv for python, but I've not looked after, so here we are.

~~~
tbatterii
I wonder why more people don't realize this. Fact is, there are those who have
no other option at their day jobs then to deploy to "ancient" systems for one
reason or another, developers often do not have a say in the matter. It's good
that these useful frameworks are willing to support the real world for as long
as they can.

------
taybenlor
Django team: "We're planning to shift to the 4 year old Python 3 over the next
2 years"

Rails team: "New Ruby alpha? Deprecate everything!"

~~~
spitfire
Now which would you bet your business on?

It'd be nice if things could move a little faster. But the alternative is
wild-wild-west.

~~~
taybenlor
In both worlds high-stakes businesses are going to stick to the "stable"
release while personal projects explore the new waters. Django's failure to
make the shift has resulted in all those personal projects not making the
shift.

Django, from my point of view, seems to be approaching this from the wrong
direction. They are waiting for the community to shift to Python 3, but at the
same time the community is waiting for Django to shift.

~~~
ubernostrum
Er, no, we're not "waiting for the community to shift".

When Django 1.0 was released, we supported minimum Python 2.3. Dropping 2.3
and then 2.4 were major logistical hurdles; they were incredibly widely
deployed, and were the stock Python for many platforms with long-term support.
So it was going to take a while, no matter what, for us to sensibly deprecate
those Python versions without just throwing a bunch of our users under a bus.
And, of course, deprecating everything older than 2.6 is a prerequisite for
starting to actually adopt 3.x.

Meanwhile, the first few releases in the Python 3 series weren't really meant
for widespread adoption, _ever_. They were there so people could start getting
a taste of Python 3 and working out a migration process while shaking out
remaining issues for Python itself.

We are now at a confluence of two very happy things:

1\. We've dropped our support for the older Python 2.x releases, without just
screwing over a bunch of people, and by year's end we'll be a minimum-2.6
project, and

2\. Python 3.2 (the current 3.x) is really quite a nice release, and Python
3.3 (later this year) looks like it'll really put the spit and polish on the
Python 3.x series, meaning it's a very attractive time to be targeting Python
3.

This is the first time we've realistically been in a position to actually make
the jump, and we're taking that opportunity.

~~~
taybenlor
Fair enough, the impression I got from the above link was that Django was
waiting for the community.

I'll hold back on commenting now because my insight into the Python community
is limited to what I hear from friends who develop on Python. Though something
I've noticed is that many of them are making the switch to Node.

~~~
tbatterii
there wasn't even an agreed upon spec for WSGI on python 3 until last year
IIRC. That needed to happen first.

~~~
maaku
And, to be fair, as soon as the py3k WSGI spec was agreed upon the Django core
developers also started implementing and pushing this transition strategy. It
was a big deal at DjangoCon '11.

~~~
tbatterii
I think every framework did. cherrypy landed there first.

------
rhizome31
Here is a Python3 packages listing on PyPI:
[http://pypi.python.org/pypi?:action=browse&c=533&sho...](http://pypi.python.org/pypi?:action=browse&c=533&show=all)
There's some interesting stuff for web developers in there: Pyramid, Bottle,
SQLAlchemy, pymongo3, Mako, Jinja2, nose, etc. It seems that the whole web
development stack is pretty much covered. Although it's not listed there,
Tornado should also run on Python3. I considered starting my new project with
Python3 but eventually sticked with Django because of the time-saving reusable
apps available, but it's interesting to note that building web apps with
Python3 should already be feasible.

------
mattdeboard
Maybe I spent way too long at PyCon but I'm getting very weary of hearing
about Python 3. Why is anyone surprised, or cheered, or dismayed, or excited,
that one of the flagship Python projects will eventually be moving to Python 3
(experimental!) compatibility?

This is a sincere question by the way, just tempered by post-PyCon weariness.

~~~
jacobian
If you think you're tired about hearing about Python 3 try sitting in my seat!
I got asked "when's Django moving to Python 3?" about thirty times a day at
PyCon, and I think the other core devs did, too. We realized that although we
had a plan in mind we hadn't communicated it well. Hence the blog post.

Sorry to bore you. Back to your regularly scheduled hacking.

~~~
sown
> I got asked "when's Django moving to Python 3?" about thirty times a day at
> PyCon, and I think the other core devs did, too.

Get custom T-shirts printed? ;)

------
tferris
Discussing Django/Python on HN isn't easy—slightest criticism leads to
immediate downvoting, outfaded comments and lowered karma. People either avoid
strong opinions or just stay away from Django/Python threads. So, it's very
hard to get an good understanding and unbiased opinions on various web
framework compared to Django.

------
mladenkovacevic
I'm sorry for the noob question, but how much additional
performance/functionality/flexibility could be gained by moving to Python 3?

Maybe my standards are too low since I am using Django as a vehicle to learn
programming and web-development as a hobby right now, but as far as I can tell
the Python/Django environment seems to work exceptionally well meeting, and
even exceeding, its intended objective (The web framework for perfectionists
with deadlines)

It seems a lot of people have this child-like concept of time where a few
months is a "long time" and upgrading to the latest and/or greatest thing is
so much of a priority that they are regarding it as a "fix". But as far as I
can tell nothing is broken by having Django run on 2.x. If a bicycle works
really, really well; is replacing the chain a top priority?

------
eta_carinae
> Python 3 is the future of the Python language

... and it always will be.

------
fauigerzigerk
I don't understand these guys (and girls?). The language sounds like Oracle or
SAP talking about migrating WalMart to a new inventory management system. If
"porting code is relatively straightforward" it would be a good idea to get
that port done and let people decide for themselves when to use the new
version.

I don't understand what makes phraseology like this useful to anyone but a
corporate lawyer: "the Django project is and will remain committed to
achieving Python 3 compatibility"

Yes, I know, I'm not supposed to talk like this about work being done by
volunteers free of charge. It's of course their right to do whatever they
like. My anger is more about the Python 3 migration going off the rails in a
way that poses an existential threat to my favorite programming language.
Django isn't helping at all to say the least.

~~~
jacobian
> If "porting code is relatively straightforward" it would be a good idea to
> get that port done and let people decide for themselves when to use the new
> version.

In fact, that's exactly what we're doing. Django's going to support Python 2
for quite some time yet; this move will simply let those who want to switch to
3 do so.

> I don't understand what makes phraseology like this useful to anyone but a
> corporate lawyer: "the Django project is and will remain committed to
> achieving Python 3 compatibility"

Because we weren't clear that Python 3 support was in the pipeline some folks
in our community started to wonder if we planned to move the Python 3 at all.
That's pretty much the point of the post: to clarify that we do indeed plan to
port to Python 3 (and when). How would you have worded it differently?

> Yes, I know, I'm not supposed to talk like this about work being done by
> volunteers free of charge.

That's not true at all: if you have substantive critiques to make, then you
should make them. We try incredibly hard to listen to our critics and take
away what we can. However, your criticism here is pretty thin and rather
personal -- seriously, accusing a bunch of volunteers of being like Oracle or
SAP? Low blow, man, low blow. It's hard for me to figure out exactly _why_ you
felt compelled to post this criticism, so it's hard to figure out what the
action you'd like us to take.

> My anger is more about the Python 3 migration going off the rails in a way
> that poses an existential threat to my favorite programming language. Django
> isn't helping at all to say the least.

OK, so here we go -- there _is_ something you'd like to see us doing
differently. But what?

~~~
fauigerzigerk
I thought it was rather obvious what I would like to see you do: Always
support the latest version of Python as soon as possible after it is released.
Not supporting Python 3 years after it was released isn't just a problem for
Django, it is a problem for Python. I'm assuming that Django's developers know
that as well and that they (you) care about Python, so that's why I don't
understand why this hasn't been fixed a long time ago.

I can totally understand that you don't like my Oracle/SAP comparison and I do
apologize for the tone, but there is actually some substance behind it. Oracle
and SAP provide core transactional capabilities to very large corporations.
These corporations make money based on their scale, not because they are
particularly nimble on the IT side.

They're not startups. They rarely do any real greenfield stuff. They don't
hire developers who want to be on the bleeding edge. Their IT culture is one
of mediocrity and that's why it doesn't matter how many generations/years the
software they use lags behind the latest platform version.

I would have thought and hoped that Django is not just for that type of
corporation.

But judging by the reactions here you seem to be in sync with the majority of
your users, so the problem isn't yours so much as it is mine. I should
probably not get agitated about it and just move on.

~~~
jacobian
I think the problem is that our definitions of "as soon as possible" diverge.
To wit:

* Database drivers (psycopg2, MySQLdb, etc.) weren't updated for Python 3 until... last year some time, I think? Django supporting Python 3 is somewhat academic if you can't use a database with it, don't you think?

* The WSGI spec wasn't updated for Python 3 until summer 2011 (July, IIRC). Again, Django supporting Python 3 is pretty academic if you can't run it under a web server, right?

* The previous suggested approach for dual Python 2 and 3 support (using 2to3) proved to be untenable for Django [1], and it wasn't until this last year (or so) that a better approach (single shared source) emerged.

So to my mind, "as soon as possible" was something like July 2011. We had
unmet dependencies before then; moving would have been... well, not
impossible, but incredibly painful and nearly useless.

I will, however, take full responsibility for not getting Python 3 support
shipped between then and how. Our current release workflow [2] got in the way,
and about 7-8 months because of it. That's my fault, and if you want to rake
me over the coals for those months go right ahead: I deserve it.

[1] Details omitted to be concise, though I'm happy to share them if anyone's
interested.

[2] Which we're changing, mostly because of this silly workflow-induced delay.

~~~
fauigerzigerk
I'm fully aware that the whole migration disaster isn't primarily Django's
fault. But Django is in a position to put pressure on others by simply
creating facts and that hasn't happened.

I'm sure you could have found a way to talk to web servers without waiting
years for the WSGI spec to be finalized. And if Django had supported Python 3
using py-postgresql, which was released in 2009 (I know Python 3 only), others
would have worked a little faster to make their database systems available on
Python 3 as well. Upgrading psycopg can't be such an incredibly difficult task
either.

The reality is, everyone has basically been waiting for everyone else to make
a move and it has resulted in deadlock.

~~~
jacobian
> But Django is in a position to put pressure on others by simply creating
> facts and that hasn't happened.

How exactly does one create a fact? That would be a neat trick.

> I'm sure you could have found a way to talk to web servers without waiting
> years for the WSGI spec to be finalized.

Indeed, we could have, and it would have benefited Django at the expense of
diversity in the Python web world. It probably would have helped our "market
share". But that's not how we roll. I'd rather work on something that the
whole community can benefit from, and that means standards and specifications.

> And if Django had supported Python 3 using py-postgresql, which was released
> in 2009 [...]

That's true - I'd forgotten about that library.

> [...] others would have worked a little faster to make their database
> systems available on Python 3 as well.

That... I'm not so sure about. Volunteer community, limited by free time, etc.
I'm fairly sure most people have worked on this stuff just about as fast as
they can.

You do realize, of course, that the criticism here applies to _you_ as well.
If you feel this strongly _you_ could have added a py-postgresql backend in
2009, _you_ could have come up with a way to run Django under Apache and
Python 3, etc. I'm happy to take a lot of the blame for Django's slow
movement, but you're also responsible for not channelling your frustration
into action.

> The reality is, everyone has basically been waiting for everyone else to
> make a move and it has resulted in deadlock.

I don't see a deadlock. I see steady movement -- slow, too slow, but movement
nonetheless -- towards Python 3. First Python 3.0 came out, and it was
horrifically slow, taking away any impetus to use it. So people fixed that.
Then, developers ported the low-level libraries (database drivers, network
protocol implementations, etc). Then we started moving to higher level
libraries and tooling (numpy, wsgi, ...). Now the highest-level libraries (web
frameworks, scipy, matplotlib, ...) are porting. Next, and finally, you'll get
to port your apps. It's a classic dependency upgrade chain. You can't upgrade
layer N until layer N-1 is done.

I share your frustration about the slowness, but that's frankly a
characteristic of the Python community. We're relatively conservative, tending
to prioritize stability and backwards compatibility over rapid movement.
Everyone pretty much expected this process to take a while, and turns out
they're right.

~~~
fauigerzigerk
_How exactly does one create a fact? That would be a neat trick._

As I said, by implementing something instead of waiting for the spec. I
totally disagree that it would have helped Django at the expense of everyone
else, unless you patented it. Any implementation, even lots of fragmented
ones, would have been better than years of stalemate. I guess we'll just have
to agree to disagree on that one.

 _You do realize, of course, that the criticism here applies to you as well_

Yes, absolutely, that's a very fair point. I just didn't expect things to move
_that_ slowly. With the benefit of hindsight I want to punch myself in the
face for not doing it myself.

