
Making Python 3 more attractive - explosion
http://lwn.net/SubscriberLink/640179/06c2868dffb2aec3/
======
mangecoeur
I think the obsession with the GIL is sort of missing the point - people on
python2 live with the GIL and don't really miss it, I don't think that's
really the killer feature to drive python3 adoption. Especially considering
its almost impossible to do without breaking something (most likely C
extensions) and when you see the wailing and gnashing of teeth that came from
python3 forcing people to fix their text encodings it doesn't look seem like
more breaking changes are going to drive python3 further. What's more, python
already has quite convenient multiprocessing, and python3 concurrent.futures
makes it even easier - frankly I think too many people complain about the GIL
without having tried multiprocessing (doesn't help that it seems every
tutorial starts with multithreading, then tells you it doesn't really work,
and only then tells you about multiprocessing). You only get to complain if
that doesn't work for you!

Personally I think the driver is going to turn out to be type annotations.
When you see the enthusiasm for adding type annotations to JS (typescript,
ES6, etc) its easy to see that translating to python. Static analyzers can be
a huge help (you can already get a taste of it with PyCharm) and I for one
would like to move away from "traceback driven development" where you just
have to keep re-running the code until all the preventable glitches are worked
out...

~~~
keyneus
Anecdotal data to agree with type annotations:

We have a fairly large Python codebase here. I've been wanting to move away
from Python for a whole host of reasons, a top one being that it's dynamically
typed. However, when mypy was released, that was enough of a reason to port
from Python2 to Python3. Now we have a mostly-annotated codebase, and I'm a
lot happier.

We'll probably move away from Python eventually (I still don't think it's an
especially good language, and type checking with it will never be as useful as
with a language where it's built in and _required_ ), but it's not as pressing
of a concern.

I think type checking turns Python into a bearable language.

~~~
ngrilly
To which languages could you possibly move, as a replacement for Python?

------
plesner
Making python 3 more attractive is not the solution, it's part of the problem.

I can't use just python 3 because python 2 is still widely used. I can write
code that works on both but now I'm using the worst of both worlds, and even
worse I now have to test on both. And that'll last until python 2 goes away
completely which, - when has a language ever gone away quickly?

These aren't fun problems. Improving python 3, making it more attractive,
that's fun. But that's problem 2. Making migration less painful for me should
be problem 1. Who's working actively on that?

What particularly grinds my gears is the apparent disregard for migration in
the design. Take the changes to the print statement. Often this is the only
thing that prevents my existing code from working in python 3. And it's in my
muscle memory so I _always_ get snagged when debugging on python 3. For what?
Take a read through the rationale:
[https://www.python.org/dev/peps/pep-3105/#rationale](https://www.python.org/dev/peps/pep-3105/#rationale).
Most of the benefits you could have had if you'd named the new print function
something else. I can understand that you regret adding it in the first place.
What's even worse though? Adding it and then removing it in an incompatible
way.

If you want to make python 3 more attractive maybe make migration easier
before going on to the fun improvements? There are low-hanging fruits for
migration too. How about adding back the print statement?

~~~
bufordsharkley
Removing the print statement is, to me, an extremely frustrating breaking
change to make.

If nothing else, I feel that backward compatibility should be given toward's a
language's canonical "hello world" example.

~~~
k_bx
Yeah, this was a very unwise thing to make, at least this could be postponed
for later.

~~~
Latty
What? And make another language breaking version down the line? Better to rip
off the plaster in one fell swoop.

~~~
k_bx
It's better to make multiple breakages one by one than not doing them at all.
On a plus side, maybe at some point you'd realize this print statement isn't
that important anyway to make thousands of edits all over the world.

------
sago
GIL seems to me to be only a niche problem in reality. But it is a bit of a
storm in a powerpoint (i.e. people see it on a list of features of python and
panic. Even though, as the article says, Javascript has much more constrained
single-threadedness).

I've been deploying python for almost 20 years, and I haven't had a single
performance issue that was caused by GIL and couldn't easily be worked around.
In my experience, building big multithreaded applications with shared memory
access isn't great design anyway. I prefer systems that share as little as
possible, and can therefore scale beyond a single machine when needed.

So I think the focus on the GIL is a false quest. It isn't a bad compromise to
a thorny implementation issue (it allows certain performance optimisations,
without forcing you to worry about re-entrancy and atomicity when writing
simple code). Removing the GIL will be a big thing for pundits, I think, but
won't make much of a difference to big python deployments. It certainly isn't
the killer app for Py3 adoption.

~~~
kaoD
Disclaimer: I used both Python and Node.js at job, and use Node.js regularly
for side projects.

I think the difference is that Node embraces the single-threadedness as part
of its architecture. I don't use Python that much now so I'm not sure if it's
changed, but I found it clumsier by default.

I know there's Twisted and other nice evented/multiprocess libraries that make
Python closer to Node, but it surprises me that a batteries-included language
with a GIL has to be, as you say, worked around. I don't work around single-
threadedness in Node, I work _towards_ it! And the platform encourages me to
do so.

EDIT: I'm not sure why you seem to have been downvoted (man, I hate HN
sometimes). Here's an upvote to counter it.

~~~
sago
GIL is a bogeyman, particularly from folks who've never deployed large python
codebases, so I was fully expecting downvotes. Unpopular opinions attract
them. Though it would be more gratifying to see downvotes along with people's
use cases for when GIL was a genuine issue for them.

I agree that Node has its single-threadedness baked in, and Python doesn't
clearly have one way to do it. Still, I think that multithreaded programming
isn't very scalable in the long-run. Learning Erlang and figuring out that
message passing buys you much more headroom was important to me. I like that
you can do simple things in Python simply, but it is important to have some
understanding of why different languages solve the problem in different ways,
and what approach is best for a problem domain. My sense is that Node's
default approach is actually very good for a large range of situations, a
larger range than multithreading, anyway. But that could be because my use-
cases are often either UI or CPU bound.

In your example, is your domain primarily internet servers (what I assume is
Node's sweet spot for commercial deployment).

------
rdtsc
> The Unicode support that comes with Python 3 is "kind of like eating your
> vegetables", he said. It is good for you, but it doesn't really excite
> developers

Saddly I agree with that. There needs to be either a big stick (Python 2 being
really bad, but it is actually pretty good) or a large carrot ("Oh look 3x
performance improvement!").

Something like a carrot was presented during Pycon and that was gradual types
(optional types). These reduce some cases covered by unit tests, make code
more readable for new developers, help with IDE support, and of course assist
with general static checkers. According to Guido 3.5 should start having a
partial support for it.

But in general I would have liked either one of these instead (some are
contradictory, arbitrary hard, or downright impossible):

* At least 2-3x performance improvement

* No GIL

* Merge greenlet library in the core (to make eventlet or gevent work)

* Some kind of an ahead of time compiler that bundles just the needed interpreter library parts into an executable

* Firefox and Chrome agree to add browser suport for it

* Mobile support (native Android support or Apple ditches Swift and uses Python instead).

~~~
bjwbell
Firefox added Python support in the past and there was no uptake. Brendan Eich
has a hnews comment on it.

I think 2x-3x perf improvement /is/ possible. I mean we have the example of
javascript that went from terrible perf to almost native parity. Of course
Mozilla & Google each dedicated an entire team to get there.

We already have the example of PyPy too which /today/ averages a >5x
([http://speed.pypy.org/](http://speed.pypy.org/)) speed up with CPython!

~~~
juliangregorian
Looks like this
[https://news.ycombinator.com/item?id=3096459](https://news.ycombinator.com/item?id=3096459)
is the Brendan Eich comment.

~~~
BrendanEich
And again, recently:

[https://news.ycombinator.com/item?id=9188011](https://news.ycombinator.com/item?id=9188011)

------
eeZi
Reasons I am excited about Python 3:

* "yield from"

* Unicode support (I'm German and the clear distinction between bytes and unicode really makes my life easier)

* function annotations (PyCharm interprets them and uses them for static type checking)

* cleaned up stdlib (not only names, but also features)

* asyncio

Library support is very good nowadays, pretty much all of the important
libraries are either ported to Python 3 or have an active fork. Even OpenStack
is working on Py3 support.

~~~
reidrac
So far I've focused on making new projects both Python 2 and 3 compatible, but
"new stuff" in Python 3 is not always available when:

\- Laptop running Ubuntu Trusty: Python 3.4

\- Production servers with Debian Wheezy: Python 3.2

\- Laptop running Fedora 20: 3.3.2

You don't usually install Python in Linux using upstream, you install the
version provided by your distribution and unfortunately Python 3 has
differences between these versions (eg, "yield from" was introduced in 3.3,
IIRC), whilst 2.7.x it's been without changes for a long time.

Even Apple includes 2.7 now (handy when I distribute a game made with Pyglet);
so I'm excited too, but Python 2 still makes my life easier.

~~~
eeZi
Debian Jessie and Fedora 21 both ship Python 3.4, so that problem is going
away.

In the meantime, it's always possible to install a self-contained Python 3.4
interpreter and a virtualenv. It's pretty straight-forward, actually. The only
thing you'll miss is distribution-provided binary packages (like lxml), but
pip/setuptools compiles them for you.

~~~
eeZi
Refer to this comment for an alternative way to install Python 3.4 without
compiling it yourself:
[https://news.ycombinator.com/item?id=9389742](https://news.ycombinator.com/item?id=9389742)

------
cookiecaper
I don't think any of the things proposed will really drive conversion from Py2
to Py3. I don't think developers need to be excited about it, it just needs to
be plausible. Python 3 has some significant momentum now (these talks seem
overly negative about it). Developers are waiting for the signal from the
enterprise Linux distributions that Py3 is "truly stable", i.e., they're
waiting for it to be made the default Python. Once this happens, any remaining
holdouts, which, again, are not that many of the actual libraries that people
regularly use (cf.
[https://python3wos.appspot.com](https://python3wos.appspot.com) ), will stop
their moaning and finally catch up.

I think if someone hasn't moved to Python 3 yet, no iterative change is really
going to get them to do it. It's OK if old software is resistant to breaking
changes; this is about building a good ecosystem for the software to come. If
it was about making things easy for people who've already written their
software, Python 3 would not have been released in the first place.

Honestly I don't really see why anyone cares about whether Mercurial is Py2 or
Py3, since it's not a library and isn't holding up new development. Mercurial
can use the Py2 interpreter to its heart's content and it shouldn't have any
effect on the prosperity of Py3.

The Python community needs to get serious about pushing adoption of Py3 by the
distros, and then we can put this navel gazing to rest and move on with Py3
finally realized as the standard.

~~~
ghshephard
Mercurial is a canary - if they see so much pain involved in moving from Py2
to Py3, and little reward, then that's representative of the overall
ecosystem.

Having people on older versions of the interpreter is a problem, because it
divides the community in half when it comes to knowledge, skillset,
capability, etc.. It also confuses outsiders who are looking at the situation,
and don't understand which version they should move forward with, because it's
not clear which one will by supported by the people they want to hire.

There is a real cost to having the community split - which is why so much
effort is put into trying to pull it back together.

~~~
cookiecaper
Mercurial is a large, complicated project, so it makes sense that they'd delay
a backend upgrade like Py2->Py3 as long as they can get away with it. It's not
representative of the overall ecosystem. That's like arguing that as long as
"canaries" like Facebook don't see any reason to move to Ruby or Python for
web dev, the rest of us might as well stay on PHP too. There's a large
investment in the existing infrastructure that often can't be ignored no
matter how compelling the features of the new platform are.

The differences between Py2 and Py3 are not substantial enough that it
"divides the community in half" in any tangible fashion. Yes, it's possible
new software may not take Py2 compatibility into account and you'll need to
switch to Py3 to get new stuff, but that's been the plan all along, right?

183 of the 200 most-used libraries are Py3 compatible now, including major
systems like SciPy and Django. It's not 2009 anymore, and we should stop
talking like it is. Py3 is the way forward and the Python Foundation needs to
quickly and firmly deconstruct any sign of flapping on that front if they hope
to convert the remaining Py2 holdouts.

I believe it would be very damaging for the community if Python backpedaled
and said "OK, we know you just spent 7 years investing in the Py3 platform and
converting your apps and libs to work on it, but visible projects like
Mercurial still don't like it so we decided it wasn't worth it anymore."
People don't take time out of their day to make their voices heard unless
they're already discontented about something. We shouldn't assume that
everyone hates Py3 just because Py3 users are going about their business
quietly.

~~~
ghshephard
Don't get me wrong - I'm totally on board with Py3, I'm just saying for
newcomers to the language, they hear, "Python 2, if you want to be compatible"
and "Python 3, it's the future" \- and sometimes, they just throw up their
hands and say - "I have no idea what version I should use."

I think things will get better when operating systems start to default to
Python3.

------
vessenes
Python developers would almost all upgrade in a single minute for 30%+ better
performance.

It's interesting that performance wasn't a topic at this rump session as
reported; I moved over to Go about a year ago, and while I miss Python's
expressivity at least once a week, I'm just not willing to slow down all my
programs by 5x.

On the other hand, if Python could double in speed, I'd likely try to rework
it into our workflows. Well, maybe. I really dig Go.

~~~
hurin
>Python developers would almost all upgrade in a single minute for 30%+ better
performance.

But strangely they don't - PyPy has hardly gained traction (albeit the
python2->python3 switch didn't help) and looking at the benchmarks that's more
like 5-7x performance.

I suspect that most often, in places where performance matters enough in a way
that would warranted refactoring a code-base from cpython to PyPy, they
already rewrote the botte-neck parts in C anyways.

~~~
xioxox
A lot of the problem is that PyPy isn't completely compatible with C
extensions.

------
iandanforth
Nothing in that article matters to me and I use Python every single day. I
guess if you know too much about a thing it's easy to lose sight of what
"normal" users care about. So here's my list:

1\. Speed

2\. Language warts (e.g. del, __init__, import *, while: else:)

3\. Lack of a modern UI toolkit

4\. No native support in Android, iOS, or Browsers worth mentioning.

Right now Python is the perfect prototyping, glue, and modest workload
language.

You can make it better for heavy loads by fixing 1. You can make it even more
attractive to novice programmers by fixing 2.

But you really have to get to 3 or 4 before it becomes truly attractive and
you get a mass adoption.

~~~
vacri
I always found it weird that python functions quite happily tell you that
you're missing a colon, but can't 'just run' without it.

Excluding one-liner syntax, why does Python actually need a colon to define a
function, given it's goal of being free of unnecessary syntactic elements? I'm
only an intermediate pythonista, but I'd be interested to know if there was a
particular point to the colon.

~~~
cma
It doesn't, it just is easier to scan. There isn't a goal of being completely
free of syntactic elements, semicolons went because they were excessive and
noisey, colons seem, at least to me (comparing with say, coffeescript) to make
it less noisey.

~~~
vacri
Thanks for the info, all.

------
byuu
> Windows has the CreateFiber() API that creates "fibers", which act like
> threads, but use "cooperative multitasking". For POSIX, using a combination
> of setjmp(), longjmp(), sigaltstack(), and some signal (e.g. SIGUSR2) will
> provide coroutine support though it is "pretty awful". While it is
> "horrible", it does actually work.

I do this, and it works perfectly well. Here's a full implementation
demonstrating this approach:
[https://gitlab.com/higan/higan/blob/master/libco/sjlj.c](https://gitlab.com/higan/higan/blob/master/libco/sjlj.c)

It's been successfully used on x86, amd64, ppc32, ppc64, mips, arm and sparc
in several projects.

However, it still has a good bit of overhead. But you can implement this
concept absolutely trivially on any platform for maximum speed. All you need
to do is save the non-volatile registers, swap the stack pointer, restore the
non-volatile registers from the swapped-in stack, and return from the
function. If you haven't realized, one function can reciprocally save and
restore these contexts. Here's an x86 implementation, for example:

    
    
        co_swap: ;ecx = new thread, edx = old thread
        mov [edx],esp
        mov esp,[ecx]
        pop eax  ;faster than ret (CPU begins caching new opcodes here)
        mov [edx+4],ebp  ;much faster than push/pop on AMD CPUs
        mov [edx+8],esi
        mov [edx+12],edi
        mov [edx+16],ebx
        mov ebp,[ecx+4]
        mov esi,[ecx+8]
        mov edi,[ecx+12]
        mov ebx,[ecx+16]
        jmp eax
    

This turns out to be several times faster than abusing setjmp/longjmp.

I turned this into the simplest possible library called libco (public domain
or ISC, whichever you prefer.) The entire API is four functions, taking 0-2
arguments each: create, delete, active, switch.

The work's already been done for several processors. Plus there's backends for
the setjmp trick, Windows Fibers and even the slow-as-snails makecontext.

If Python does decide to go this route, I'd certainly appreciate if the devs
could be directed at libco for consideration. It'd save them a lot of trouble
making these, and it'd get us some much-needed notoriety so that we could
produce more backends and finally have a definitive cothreading library.

~~~
sanxiyn
FYI, greenlet already includes stack switching code, so it would make more
sense to use that.

[https://github.com/python-
greenlet/greenlet/tree/master/plat...](https://github.com/python-
greenlet/greenlet/tree/master/platform)

~~~
byuu
Ah, neat. Looks like they have even more backends than me. I've never even
_heard_ of S390 before. Okay, nevermind then. Thanks for the link.

That's the downside of these libraries, there's as many libraries as there are
people wanting to do this. I've been hoping for a standard to emerge, even if
it's not mine. This looks a bit too tied into Python to be general purpose,
though.

------
mrweasel
Honestly I thought that we where done with 2 vs. 3. Every new project we start
is Python 3 and I don't hear anyone in the office preferring Python 2.

Dealing with Scandinavian languages having the Python 3 Unicode support is the
killer feature in Python 3, it just make everything so much easier. In terms
of performance it's fine and library support is no longer an issue (for us at
least), everything we use just works.

~~~
coldtea
Yeah, perhaps your experience in your office is not generally representative?

In PIP, Python 3 has like 5% uptake...

~~~
mrweasel
What do you mean with "In PIP"? Are only 5% of the pip downloads from Python
3? That would make sense seeing as pip is included in Python 3.4, so there no
need to download it.

If you mean that only 5% of the packages on pypi.python.org is Python 3, I
would say that sound a bit low. Also unimportant for most, if it's just the
correct 5%. There's also a ton of old cruft on pypi.python.org that would
count against Python 3, but not really be important to anyone.

Quick check, there are 7633 Python 2 in the package index
([https://pypi.python.org/pypi?:action=browse&c=527](https://pypi.python.org/pypi?:action=browse&c=527))
and 8949 for Python 3
([https://pypi.python.org/pypi?:action=browse&c=533](https://pypi.python.org/pypi?:action=browse&c=533))

~~~
coldtea
Neither downloads of PIP, nor packages in PyPI.

I was talking about versions of Python used to download packages off of PyPI
(with pip et al).

As of 2014/1, those accounted for < 5% of the downloads, with 2.X being the
rest:

[https://alexgaynor.net/2014/jan/03/pypi-download-
statistics/](https://alexgaynor.net/2014/jan/03/pypi-download-statistics/)

~~~
mrweasel
Aaah okay, makes sense. I would like to see new stats though. It wasn't until
Python 3.3 we started switching, before to many 3rd party libraries where
missing.

Still I don't see much reason to not switch at this point. I don't believe
that most developers would be missing anything.

Small note: We run our own pypi server for a large number of packages, so our
Python 3 installations aren't counted for everything, but the same should be
true for many Python 2 setups.

------
kylebgorman
This post seems to conflate language and implementation. IMO, Python 3 the
language has tons of improvements and no regressions. The grief on the
internet about Python 3 makes me seriously wonder how many people who don't
like Python 3 have actually tried it yet. (There are legitimate critiques of
Python 3, but they're few on the ground and none are presented here.)

The suggestions in this post are mostly changes to the implementation (i.e.,
make it go faster), not the language itself. While CPython 2.7 and CPython 3.4
(implementations) surely have interesting implementational differences that
don't boil down to just language changes, I'm not aware of them.

~~~
netheril96
The idea that language and its implementation can be separated is only
theoretical. In practice, the adoption of a certain language is highly
influenced by its implementation (and ecosystem).

The language improvements are nice. I know this because I started with Python
3 then switched to Python 2 and missed some of the goodies now and then. But
the language improvements are not enough to overcome the breakage of backwards
compatibility. Only a vastly improved implementation (that is not backported
to Python 2.7) will.

~~~
kylebgorman
Python is notable among its peers in that it has a well-defined spec that is
not just a list of bugs in the reference interpreter. It is thus important to
note that some of the non-reference interpreters do already have the
properties requested in TFA, but for whatever reason they haven't supplanted
the reference implementation. (I suspect that the ability to drop down into
C/C++, link against preexisting libs, or use tools like Cython and taking
advantage of the compiler's optimizer, is actually more important than, say,
the average PyPy speed boost. Certainly it is for me.)

------
abusque
I find it somewhat unfortunate that a LWN subscriber link is being abused like
that. I don't think such a link should be shared on a widely accessible
platform like Hacker News. I find LWN articles to always be of great quality,
and the subscription cost is definitely worth it if you can afford it. Also,
"subscriber-only" content becomes publicly available after only a week. There
is consequently no reason to share a subscriber link like this on Hacker News.
The discussion could have waited a week.

~~~
bryanlarsen
Jonathon Corbet, the editor, encourages occasional subscriber links posted to
Hacker News:
[https://news.ycombinator.com/item?id=5688938](https://news.ycombinator.com/item?id=5688938)

~~~
abusque
I wasn't aware of his stance on this specific matter. I guess I was needlessly
worried, then.

------
danso
This year I've been spending a lot of time learning Python as I can tell, even
being a longtime Rubyist, that Python is the better language for teaching and
for general purpose usage...and since Python 3 is already pretty mature, I
figured I should just pretend that Python 2 doesn't exist...

Well, after a month of studying, using, and teaching the language...all I can
say is, the conflict between 2 and 3 definitely lives up to the hype :)...Most
of the changes make sense to me, and even as a "do-whatever-you-feel-like-
aesthetically" Rubyist, I appreciate what Guido did/attempted to do in the
clean-up. But things like lambda...there obviously was no easy answer...I love
lambdas, but it's so functionally limited and awkward in Python that I also
see Guido's point about just removing it from the language (ultimately, he
gave up on that)...

But what about the built-in reduce()? Again, it's another function that I
instinctively reach for as a Rubyist...and yet it's so awkward in Python that,
again, like lambda, maybe it should die? But in this case, Guido halfway-won,
and now it's been pushed into the functools package. Mmmkay. And so it is with
so many of the 2 to 3 changes at the interface level...as a newbie, it's just
mostly amusing since I have no legacy code to port over, but I definitely
understand the strife.

But the conflict is still hard to avoid as a newbie...many of the most used
guides (LPTHW, Codecademy's Python track) are just done in Python 2...LPTHW
says right up front to stay the fuck away from 3.x. I don't think Codecademy
even bothers to mention what version they're teaching...obviously, beginners
don't need to get into the version wars, but as soon as they get past
Codecademy and start Googling around, they're going to be in for some
surprises.

Hell, the act of Googling is itself affected by the version-wars...everytime I
google for commands/subsections in the official Python docs, the version 2.x
docs are always at top. Sometimes the 3.x docs don't even show up. At least I
know that there's a 3.x and how to manually switch to those docs...imagine all
the novices who are also Googling for references...it's not hard to think that
the cycle of 2.x indoctrination is propped up by the simple fact that 2.x
docs/help are always at the top of the Google results.

------
tdicola
I really wish there was more thought put into allowing code to be compatible
with both Python 2 and 3. The fact that the six library (which supports
writing Python 2 and 3 compatible code) isn't in the core Python library is a
big fail IMHO.

I've written a few things that are meant to support Python 2 and 3 from the
same codebase and it was a bit of a nightmare finding all of the gotchas.
Stupid little stuff like changing dict.iteritems() and replacing it with
dict.items() (which still exists in Python 2 but doesn't do what you expect!)
in 3 are a big pain to deal with when writing code that has to work with
python 2 and 3.

This page has a lot of good advice, but the fact it's so long is just a
testament to how painful the 2 to 3 transition is for people: [http://python-
future.org/compatible_idioms.html](http://python-
future.org/compatible_idioms.html)

------
melling
People are always going to whine about any change. The Python community should
have been more adamant about dropping 2.x updates. Spreading out the pain
doesn't make it easier...more code is still being written in 2.x. Ugh, I think
I said this 5 years ago. Imagine the tens of millions of new 2.x code that's
been written in the last 5 years. Oh well, good luck.

~~~
ghshephard
It's challenging. On the one hand, you don't want to break what is working
(Python 2.x), so you'd like to support it so that community can continue to be
productive. On the other hand, you don't want your braintrust split across two
different versions, and you don't want the confusion that comes from having
two major versions.

For example, the (Epic) Learning Python 5th Edition by Mark Lutz, had to be
written for _two_ simultaneous audiences, 2.7 and 3.3. There are all these
disclaimers noting where things are different between the two versions, and
there is a lot of cognitive overload trying to read a book can't assume you
are on Python3.

In the best case world - people would treat this like a y2k situation, and
realize that if they didn't get with the program, and migrate over to Python3,
they'll end up like Perl, with some other newcomer that isn't so bipolar
charging forward and winning mindshare.

Unfortunately, their are a lot of Python2 people who are happy with Python2,
and we're in the situation we have today.

~~~
melling
There are a lot of Windows users who are happy with XP too. I remember trying
to explain to people 20 years ago why HDTV was better than what they had. Most
people were happy with 480 lines of resolution, or thought they were. It's
human nature. By being accommodating, you've made the problem harder.

Apple is the only company that says screw legacy. Of course users complain but
they just grumble and know to accept it. Apple and their users, as a whole,
benefit greatly by "getting all the wood behind one arrow" strategy.

There's lots of grumbling about Swift but I bet we get a very large developer
adoption rate within 24 months.

~~~
ghshephard
But Swift is an actual example of something with no legacy to worry about.
Instead of being Objective C++2015, it's an entirely new language. And, more
importantly, Apple can give guidance that people have to follow. As Swift
matures, regresses, and has the bugs worked out of it, and library support is
fully developed, Apple can arbitrarily identify a point at which Objective C
just has to die, and the developers (if they want to develop applications for
the App Store, which is 99% of them), will just have to do so.

The nice thing about Windows XP, is that by and large, a Windows XP user could
use Windows 7 with next to zero training. Or, in fact, zero training.

This is why the first version of Windows 8 pissed so many people off -
removing the Start Button all of a sudden made the Operating System unusable
to a _huge_ portion of the Windows user base.

I understand they've brought back the start button.

~~~
qznc
Swift has legacy builtin. It is forced to use Objective-C's OO mechanisms.

------
wiz21z
I've ported a 100KLOC project. Took me a week or so to do it. But it took me
months to iron out the last bugs. Had no problem with libraries (so I just had
to support the language)

Fact is 2to3 is nice but it doesn't give you any guarantee about its code
coverage. So you go almost just as fast working by hand.

But the lack of guarantees, that makes working in production very dangerous.

Tried to support 2 and 3 at the same time, but that's just too exhausting and
error prone (one has to check in both python2 and python3)

Projects with 100% test coverage don't exist, spare time project have even
less test coverage.

For me unicode was the driver to change. And it paid off. And I think that's
the only P3 feature that actually improves expressivity (now I can clearly
express unicode strings). The yield stuf, etc. is fine but nothing /that/
impressive.

For performance, forget PyPy, a 5x/7x improvements is not enough : you still
can't write high perf code with that. If PyPy was 50x faster than CPython,
that would be something.

So basically, after a lot of efforts I'd say write Python3 code because it
helps Python or because you use unicode. Any other reason seems a bit weak to
me. And that's sad, I've bet on Python 4 years ago and it didn't evolve much
(it surely became very stable, which is not funny but damn useful!).

I guess the point of the 2-3 war is precisely that : 2 and 3 are different but
not different enough... So people have hard time to make a choice.

------
kbd
The article mentions stackless Python. My understanding is Guido didn't want
to merge it because it would break backwards compatibility with extensions.

That was a _long_ time ago when Python's userbase was much smaller. I wonder
if Python be in better shape today had they merged stackless back then.

~~~
qznc
I think Python is in great shape. On Unix it is the dominant scripting
language.

[https://www.google.com/trends/explore#q=Python%20programming...](https://www.google.com/trends/explore#q=Python%20programming%2C%20Ruby%20programming%2C%20Perl%20programming&cmpt=q&tz=)

------
rburhum
OK... so from a _practical_ perspective, can someone tell me why I should move
to python 3? None of the arguments presented to switch are strong enough :-/

~~~
qznc
A practical perspective depends very much on your specific use case. Server
programming? Scripting? Scientific computing? Games? Tooling?

~~~
rburhum
I am honestly curious if there is a good argument for any of the examples you
mentioned.

~~~
qznc
Scripting and Tooling becomes more robust wrt Unicode. You write a short
script which works on some ASCII data. Later that ASCII suddenly becomes utf8.
With Python 2 you could easily run into weird issues, when there is a unicode
object instead of the expected str. With Python 3 decode is more explicit and
robust.

Scripting and Tooling are the major uses of Python for me, so I like Python 3.

~~~
rburhum
I always work with utf-8 strings that get passed around from native C code to
python. So far I have never had an issue. The argument for me for unicode
support seems weak. Is there anything else beyond string
encode/decode/manipulation? You mentioned tooling... like what?

------
Animats
It's not about a need for new features. It's about basic quality control. I
recently ported a medium-sized production system (the back end of
"sitetruth.com") from Python 2 to Python 3. It took about a month. Not because
of the language changes, but because several major third-party packages were
discontinued for Python 3, and their replacements were buggy.

Specifically:

\- Python 3 forces you to use CPickle instead of the Python version of Pickle.
In some multi-thread/multiprocess situations, CPickle has some memory
allocation error, Python's memory becomes corrupted, and things go downhill to
a crash. The Python version is fine. I submitted a bug report, but nothing
will happen unless I come up with a simple test case, which is hard. Meanwhile
I found out how to use the Python version, which works, despite Python 3, and
am using that.

\- PyMySQL (a "drop in replacement" for MySQLdb) originally didn't implement
LOAD DATA LOCAL. When it was implemented, it wasn't tested for large data
loads. I kept getting random database disconnects, until I figured out that it
was trying to send the entire bulk data load as one 16MB MySQL connection
packet. This only works if you configure insanely big buffers in your MySQL
server. There's no reason to send a packet that big; LOAD DATA LOCAL will use
multiple packets when necessary. It was just a lame default.

\- HTML parsing uses different packages under Python 3. The HTML5parser/BS4
combination blows up on some bad HTML, usually involving misplaced items that
belong in the HEAD section. The HTML5 parser, obeying the HTML5 spec for
tolerating bad HTML, tries to add to the tree being produced at points other
than after the last item. BS4 is buggy in that area. I wrote a function to
check and fix defective BS4 trees, came up with a simple test case, and
submitted a bug report. I have a workaround for now.

\- Python 3 finally has TLS support in SSL. (That's also been backported to
Python 2.7.9). SSL cert checking is now on by default. It doesn't work for
certain sites, including "verisign.com". This is because of a complicated
interaction between a cross-signed root certificate Verisign created, a
feature of OpenSSL, and how the Python "ssl" package calls OpenSSL. It took
weeks of work to get that fixed. Because it's a core Python package, it will
remain broken until the next release of Python, 3.5, whenever that happens.

\- Running FCGI/WSGI programs from Apache requires a different package than
with Python 2. There are 11 different packages and versions of packages for
doing this. The Python documentation recommended one that hadn't been updated
since 2007, and its SVN repository was gone. There are six forks of it on
Github, three of them abandoned. I finally found a derivative version from
which much of the unnecessary stuff had been stripped out, and it worked.

\- Python's installer program, "pip3", doesn't know which packages work under
Python 3, and tried to install a version of one of them that only worked with
Python 2.5-2.6. You have to know to install "dnspython3", not "dnspython", for
example.

These are all bugs that should have been found by now, and would have if
Python 3 had a more substantial user base. We're six years into Python 3. I
shouldn't be finding beta-version bugs like these at this late date.

Python's Little Tin God's position on third-party library problems is that
it's not Python's problem. His fanboys follow along. (Comment on
comp.lang.python: "You have found yet another poorly-maintained package which
is not at all the responsibility of Python 3. Why are you discussing it as if
Python 3 is at fault?") As a result, PyPi (Python's third-party package list)
has no quality control. Perl's CPAN has reviews, testing, and hosts the actual
packages. Most of Go's key packages are well-exercised within Google and
maintained there. PyPi is just a link farm.

That's why Python 3 isn't getting used. It's not a need for new features. It's
that Python 3 doesn't work out of the box. Its supporters are in heavy denial
about this.

~~~
corford
Re: FCGI/WSGI with Apache, after a lot of frustrating trial & error I found
[https://code.google.com/p/modwsgi/](https://code.google.com/p/modwsgi/) to
work the best.

If you can dump Apache, uwsgi and nginx is (imho) a much nicer way to host
python apps.

------
bad_user
NumPy/SciPy have been the blessing and the curse of Python.

------
hanlec
You've probably read this over and over, but for me there were 2 reasons why
Python 3 wasn't a top priority: 1) lack of support in some libraries --- this
got a lot better; 2) lack of support on Google App Engine.

On the other hand, I didn't find the new features in Python 3 appealing enough
to make me fight the above drawbacks.

Last but not least, while I use most of the tools I've developed in Python on
a daily basis, they are just that: tools meant to make my life nicer.

------
peteretep
If the Perl6 rewrite is validated by it having a larger user base than Python
3 in 5 years' time, part of me that has died will be reborn. Unlikely, though.

~~~
ugexe
What Perl6 rewrite? Perl6 is a specification. Are you referring to various
compilers that have been implemented over the years as rewrites?

~~~
cygx
_What Perl6 rewrite?_

The rewrite of the language.

It was decided that if you need to break backwards compatibility anyway, you
might as well do so in a big way. In contrast, Python3 breaks backwards
compatibility as well, but only offers moderate improvements.

------
buster
Maybe it would have been great to just say "python3 will be based on pypy"..

~~~
mixmastamyk
Definitely could use a another killer feature or two.

------
r00fus
I wonder how much of the Python 2.x stickiness is due to OSX still having 2.x
as the default version. I mean, even Yosemite still defaults to 2.7!

~~~
tarentel
I imagine very little. If that were the case people would be using old
versions of Ruby as well as Python.

------
mahouse
Just make it faster and everybody will move. No need for fancy new stuff...
I'm eager.

~~~
sanxiyn
"Just make it faster and everybody will move" is demonstrably false. See PyPy.

~~~
qznc
Of course, "no regressions" is another condition. With PyPy lots of C
extensions break.

------
willvarfar
The way to make Python 3 more attractive is to backport the best bits to
Python 2.8...

~~~
eiopa
I want Python 2.8.

This is Python 2.7, but is based on PyPy (so it's waaay faster), and has
requests + gevent + lxml built-in, and adds some small niceties from 3.x
(yield from, a, b* = foo()).

~~~
masklinn
Feel free to implement that in Pypy right now?

~~~
eiopa
Haven't hacked on PyPy before. Would be interesting to do someday :)

------
TsukasaUjiie
"For another, conventional wisdom holds that reference counting and "pure
garbage collection" (his term for mark and sweep) are roughly equivalent
performance-wise, but the performance impact wouldn't be known until after the
change was made, which might make it a hard sell."

AFAIK there exists RC GC's which are performance equivalent to MarkSweep, but
these aren't super common out of academia? What is the state of GC performance
in python currently?

------
nness
Python should institute "migration" notices on all of the 2.x documentation,
making it clear that the intent of the community should be to move to Python
3. I know they plan to support Python 2 for a while longer, but no reason they
can't make the statement that everyone should be using the latest versions
available.

------
belorn
I noticed a while back that what really made me want to use python 3 is new
features. Several script I use currently has a bunch of try-except that looks
for functionality, and then monkey patch some python 3 feature into python 2.
This will only get worse until library support allows me to switch.

------
k_bx
I think that Python3 should had got only single big braking change, like
unicode story, for example. Then python4 could get rid of print statement
etc., one breaking change at a time, with a tempo suitable to overall
migration.

------
andrewstuart
These articles that assert low take up of Python never quantify in any
scientific way why they think there is low take up of Python 3.

It's opinion presented as fact that Python 3 has low take up.

Without tangible proof I call bullshit.

------
pc2g4d
Quick impression from the article: CPython holds back the larger Python
language from its potential. Maybe?

------
mark_sz
I would like to learn Python to use it for web apps - what version would you
recommend now to begin with?

~~~
corford
If you're starting now, there's really no reason not to begin with version 3.

------
mpdehaan2
The Python 2/3 problem has created a decision point.

Namely, Python3 is sufficiently different for applications supporting
distributions.

If you are doing Software-as-a-Service hosted webapps, fine, you can choose
your platform, but if you are shipping software, you have to make concious
choices about usually supporting what the distros have.

And the Linux distros are inconsistent.

This problem sounds solveable by saying "users, install a newer Python",
though this seldom is effective -- and long lifetimes of things such as RHEL 5
(yes, still afield - and some folks have to support version 2) ship versions
that are less compatible with Python 3 compatible hacks than newer Pythons.

As a result, this intent to "clean things up", I feel, has massively undercut
Python's growth rate. Maybe it's not declining, but there's been what feels to
be an inflection point.

I suppose looking at download curves for hundreds of long-standing PyPi
projects relative to the growth rates of other systems could provide this is a
thing or not.

Anyway, I do love Python. The problem is not the GIL. Most folks who are
making web services get by far with a pre-forking webserver (mod_wsgi, etc)
and something like celery for backend jobs. multiprocessing is ok enough for
some other cases.

It doesn't matter whether Python 3 is attractive so much, and that's what I
mean about a decision point - the confusion gave people a chance to shop
around, and some people are trying things in other languages now.

For instance, Go seems misapplied - it has a different expressiveness and
domain area. I'm writing a fair amount of clojure, which also feels a bit more
low level (sometimes, just in places). But I felt compelled to look around.

The crux of the theory is this - changing something singificantly will allow
people the opportunity to think is this something they still want to do.

I still believe Python strikes a great balance between expressiveness and
readability, and it's surpression of "clever" in programming makes it ideal
for a lot of problem domains. And it's kind of old enough that people are
going to want to look around.

Still, I begin to feel some of the directions being made in 3 are out of
touch, just as the resistance to some more expressive language features
(crippled lambdas, I vaguely recall) were that way. This happens when those
that write the language don't neccessarily use the language, and the
(percieved) BFDL approach of "fixing regrets" I am not sure it looks after the
good of the whole, the way the 2->3 transition happened. Those should have
been evolved slowly, keeping things compatible, rather than creating what is
essentially a new language.

I'm still pleasantly surprised by how widely deployed Python is to the rate at
which people talk about it (say, vs Rails), I think a lot of that is because
it's _NOT_ complicated, and you don't need to talk about it so much. It's a
quiet workhorse.

But I've also started new projects in Python 2 - because I've needed them to
work everywhere. Python 3 is almost sort of having the Perl 6 stigma to it in
my mind, it's available now, but it's made a compatible break that has shaken
trust.

Since Python 2 is essentially the deployed standard, there's no real reason
for most apps that must be distro installable to work on hybrid support -
until the distros move to a version that makes it easier to be compatible,
it's more important to support where the users are than risk possible bugs and
implementation trouble. Resources are better spent elsewhere.

