
Thoughts on Python 3 - michaelty
http://lucumr.pocoo.org/2011/12/7/thoughts-on-python3/
======
wladimir
"JavaScript is becoming more and more an ubiquitous scripting language that
challenges Python"... The Python dev team cannot change these conditions; even
if they came up with the perfect programming language tomorrow.

Python3 has some nice features and some that could have been better designed,
but personally I don't think it's as bad as this author makes it to be. It's
pretty much a logical progression of the 2.x series. Python 3 is being
adopted, slowly. I still think it's simply a matter of time, as Linux
distributions have plans to move on. No one expected it to go quick.

And I like that Python 3 makes Unicode versus Bytes explicit. There's working
with sequences of symbols (for humans) and working with bytes (for machines).
I regularly hoped this would be done when working with binary data and making
hw interfaces in Python, as there is a lot of confusion regarding
bytes/unicode in Python 2 also in libraries...

It was interesting to read some discussion and arguments for/against 3.0, but
it could have done with a little less "Python is now doomed" attitude...

~~~
mcdonc
Well, the author has direct experience porting and maintaining (very popular,
well-written, well-tested) Python libraries, so I think it bears more weight
than the platitudes I hear from "the other side" of "it will all work out in
the end, it's just a matter of time." I think the latter line of reasoning is
going to fail without any specifics of _how_ it's really meant to "all work
out" if issues like the ones he's given examples of in the blog post aren't
treated seriously (particularly the straddling-2-and-3-issues).

~~~
cdavid
It may just be anecdotal at this point, but several highprofiles libraries
have given up on 2to3 entirely and use the 2/3 common codebase approach:
Jinja, django, ply. For a library, that's the approach that I find more
practical.

I have a hard time seeing how one can use 2to3 on an everyday basis: it makes
testing, distribution, etc... that much more painful because it is so slow
(2to3 takes as much time as compiling all the C code from numpy, for example).
It also removes one of the big plus of a language like python (modify code -
see the result right away).

~~~
mcdonc
Amen, 2to3 in practice for anything but the smallest library is effectively
unusable due to speed. Even if it weren't, I'm not much of a fan of
maintaining generated code. That's an imposition I'm not really willing to put
up with.

------
dmbaggett
Key line from the article: "Python 3 [...] changed just too much that it broke
all our code and not nearly enough that it would warrant upgrading
immediately."

To my thinking, Python, Ruby, and Perl make people productive primarily
because of the availability of tons of high-quality packages that "just work".
The Python Package Index (<http://pypi.python.org>) lists _18 thousand_
packages now. Many are very high quality and require essentially no "impedance
matching" to use with Python 2.7 except "import package". If there's a genuine
issue with a package, you can usually use a several-line monkey-patch and
leave the package source completely untouched. Beauty.

Put simply: there's no way for a language design to make writing code easier
than not writing code. IMO, this is why, despite the warts, these languages
are winning. JavaScript doesn't have a standardized module/import system, so
its packages are fragmented across a dozen frameworks. But this may change if
the world settles on "one framework to rule them all" (or maybe two: jQuery
for UI and node.js server side).

But Python 3 breaks many of the available Python 2.X packages, and in exchange
for improvements that in most cases seem more like tweaks than major design
fixes. Things that should be fixed in both branches (e.g., OpenSSL cert
validation support) are now relegated to ad hoc patches to Python 2.X, because
all the development effort is going into the 3.X series now.

Finally, the biggest improvement to Python IMO hasn't come from the core team
at all: it's the absolutely brilliant work being done by the PyPy team. I
would love to see "Python 4" merge some of the ideas from the 3.X branch in a
fully compatible way with Python 2, and move the standard implementation to
PyPY. Among many other benefits, this would allow the Python community to
start seriously exploring adding static type-checking facilities to the
language, which would make it far more suitable for larger projects. (I'm not
saying make Python into Java, but it would be nice to be able to declare types
as one can in modern Lisp implementations, and have the compiler both check
correctness and optimize using such hints.)

~~~
kamaal
I think the lesson is if you are breaking backwards compatibility better
compress all the deprecation cycles you need to go over the years in one
release.

Perl 6 is doing that. Its is going to be one major incompatible release. But
that like compressing two decades of deprecation cycles in one release.

------
arethuza
"The multimethod based design of the language"

I've only just started looking at Python, but I wasn't aware that it has true
CLOS-style multimethods (or multiple dispatch). I know that there are ways you
can add multiple dispatch to Python - but is it really accurate to say that
the entire language has a design that is based on multiple dispatch?

Note that I'd be rather pleased to find that multimethods are an integral part
of Python - they were one of my favourite features of CLOS and I still miss
them.

~~~
sirclueless
I think "multiple dispatch" is the wrong word. "Dynamic dispatch" is a much
better word for what Python does. It's a ridiculously powerful feature. It
lets you wrap and replace functions at runtime, which gives you incredible
monkey-patching power, which is important to people programming in the real
world.

My favorite example: Suppose you have a naive O(2^n) recursive factorial()
function. I can write "factorial = memoize(factorial)" and suddenly your
recursive calls are to my new wrapped function, which references your original
implementation inside its closure. This is only possible because your
recursive implementation dynamically dispatches by name. I have turned your
O(2^n) implementation into an O(n) implementation.

In an unsafe systems language like C, I would only be able to accomplish the
same thing with some severe memory hacks, and in a language like Java or C# I
don't know how I would be able to do the same thing without some serious
involvement in runtime reflection tools, and maybe even some decompilation.

~~~
Jach
I like the Clojure form of multimethods a lot, it's even neater than CLOS. (
<http://clojure.org/runtime_polymorphism> )

C's dynamic dispatch hack isn't so terrible though--e.g. Microsoft has been
doing it for a long time by sticking in a "useless" instruction at the start
of every function.
[http://blogs.msdn.com/b/oldnewthing/archive/2011/09/21/10214...](http://blogs.msdn.com/b/oldnewthing/archive/2011/09/21/10214405.aspx)

~~~
frio
Just for my own sanity - is "multimethod" another term for a form of pattern
matching? Looking at the clojure code, it strongly reminds me of Haskell:

    
    
      encounter :: Species -> Species -> Result
      encounter (Bunny _) (Lion _) = RunAway
      encounter (Lion _) (Bunny _) = Eat
    

etc. Of course, pseudocode, syntax errors, whatever - but it looks to be the
same idea?

edit - formatting

~~~
thurn
A key difference is that Haskell functions can only have one type signature,
so you need to explicitly make "Bunny" and "Lion" data constructors for
"Species". Multimethods operate on multiple _types_. You can fake multimethods
in Haskell with multi-parameter type classes (with the language extensions to
allow undecidable and overlapping instances).

~~~
frio
I see. Thanks!

------
jnoller
An excellent Python-Ideas post from Nick Coghlan on things we can tweak:
[http://mail.python.org/pipermail/python-
ideas/2011-December/...](http://mail.python.org/pipermail/python-
ideas/2011-December/012993.html)

As well as some other discussion here:
[https://plus.google.com/115662513673837016240/posts/9dLUJxg8...](https://plus.google.com/115662513673837016240/posts/9dLUJxg8MPi)

------
cageface
_In fact if you go back in time and look at some of the first versions of
Python it's a very, very ugly language and it does not come as a surprise that
not too many people took notice of Python in the early days._

This is why I've always found it difficult to love Python. It just didn't seem
to me that Guido was familiar enough with previous language designs or had a
sufficiently refined sense of language esthetics to be a world-class PL
designer. Over time the community has built Python into an extremely practical
and useful tool, but I don't think I'll ever derive the same sense of pleasure
from writing Python code that I do from languages with a stronger unifying
concept like Ruby or Lisp or even OCaml.

~~~
dalke
I don't need to go back in time as I remember the 1990s. Even in 1995, Python
was one of the top choices for high-level "scripting" or embedded languages.
Perl was the hot language, and the other main choice was Tcl.

I used all three non non-trivial projects, and liked Python the best. It was
better at handling complex data structures than the other two. Tcl was an
easier language for my target audience (scientist/non-professional
programmers) and it was easier to embed and extend Tcl, but Python's module
and object system made up for it.

By the late 1990s, others in my field were already shipping Python-based
applications, using Python bindings to Motif.

IMO, people didn't take notice of Python because of the "strange indentation",
because high-level languages are seen as being too slow for real work, and
because people coming from a statically compiled language often want the
assurance that compile-time type checking gives.

~~~
cageface
I started using Python in 1997 and found it profoundly mind-expanding since my
previous experience with languages was limited to C/C++ and Pascal. But over
the years I learned other languages and came to see missed opportunities and
outright mistakes in the original design that I don't think a more experienced
student of programming language design would have made. Many of the original
mistakes have now been corrected but the churn in the language this has
required means that it no longer "fits in my head" the way Ruby or C do.

------
yason
I thought Python 3 was already D.O.A. I haven't seen anybody using it nor have
I seen any compelling reason to start using it myself, or any reason at all to
even keep it on my radar.

When v3 was announced, IIRC even the Python folks themselves actually
suggested that people just continue with v2.x until later when v3 becomes
mainstream and it never did. In fact, I was surprised to see negative
criticism about Python 3. It seems to me that nobody has been using Python 3,
and therefore not complaining about it either.

~~~
sp332
There's a 5-year roadmap, and we're about halfway through it. Many of the most
popular libs are already available in Py3. For example Django just released a
version supporting it. <https://news.ycombinator.com/item?id=3305021>

~~~
briancurtin
They didn't cut a release with it, yet. The work has been done and it is or
will be in a branch, but it still needs to be reviewed and "accepted" in order
to move on.

~~~
jacobian
It is, however, a question of "when", not "if".

~~~
kamaal
I don't have time to wait for "when", especially when there are working
options right now.

I am going to use and encourage other to use things that work right now, than
wait for something in the future which we aren't even clear of.

~~~
jacobian
That's awesome - write about what you do, please? The Python3 web world is
still in its infancy, and I'm really interested to hear how it goes

------
zephyrfalcon
"In fact if you go back in time and look at some of the first versions of
Python it's a very, very ugly language and it does not come as a surprise that
not too many people took notice of Python in the early days."

I don't know... Python in the early 90s looked pretty much the same as it does
now. Unless you mean that some features (or lack of them) required inelegant
workarounds?

I think most machines were just not powerful enough yet in the 90s to make
Python a viable solution for many problems. As computers got faster, that
became less of an issue. Also, there was already a scripting language with a
large following back then (Perl, naturally). Whether it was "ugly" probably
had little to do with it. (Quite the contrary in fact, I recall that Python
was often perceived as clean, elegant, concise and very readable.)

~~~
chalst
No GC, coupled with Guido's ignorant comments about how superior refcounts
were.

Wacky three namespace scoping.

Back in those days, Python seemed cleaner than it was.

~~~
kzrdude
In a way that's still true. I'm thinking of `nonlocal` and attribute lookup
which is all too complicated.

~~~
DrCatbox
And the GIL, in a world of multi-core processors with each capable of several
threads... a single process is meh.

------
nicpottier
I'm actually pretty new to Python, using it daily for the past few years, but
I do have to say I have a real uneasy feeling about Py3.

Adoption seems very slow from the various libraries, and without those people
just won't move over. And if that's the case, then the language will stagnate,
along with the myriad of great libraries that make it so excellent.

Python 2.x suits me just fine right now, it is a pragmatic language that lets
me get things done quickly and predictably. But I would be lying if I didn't
admit to gazing over Ruby's way now and then and thinking that the grass sure
looks green over there.

~~~
briancurtin
> Adoption seems very slow from the various libraries, and without those
> people just won't move over.

It started slow like we expected, but I think it's acceleration lately has
outpaced what a lot of people thought would happen. The number of Python 3
packages on PyPI is steadily rising [0], the number of Python 3 installers
downloaded from python.org is rising with each version [1], and the number of
projects announcing Python 3 support in places like reddit.com/r/Python is
rising every day.

[0] <http://dev.pocoo.org/~gbrandl/py3>

[1] <http://i.imgur.com/SLFDL.png> \- monthly download numbers for Windows
installers for all downloaded versions over the last year (it's a rough draft,
I just threw the download numbers in Excel quickly one day).

------
plq
Yes, porting to Python 3 is more cumbersome than it should be. Yes, some of
the decisions (like crippling the byte types, or implicitly changing behavior
based on environment variables) turn out to be bad decisions, but it still
sounds like there's already some work towards fixing these. As more and more
people gets to work with Python 3, that seems normal to me. As we know, "There
are only two kinds of programming languages: those people always bitch about
and those nobody uses." It seems to me that Python 3 has started to get its
healthy dose of bashing, and that's a good thing.

As for my anectodal experience with 2to3: I've recently been working on
porting rpclib to Python 3. After skimming the diffs it produced for a simple
`2to3 src/rpclib` call, I chose to ignore most of the transformations it
applies.

Replacing commas in except statements by the "as" keyword or adding
parentheses where missing work just fine. But wrapping every call to
dict.keys() inside a list() call? That's bold.

Once 2to3 is tamed[1], I think the code it generates can be maintained.
Certainly beats having to get the current exception from sys.exc_info.

[1]: <https://github.com/plq/rpclib/blob/master/2to3.sh>

~~~
wahnfrieden
Maybe it needs an interactive mode?

------
pitiburi
Are you upvoting it because you agree with the rant, because you think it's
time for a new debate over Python 2.8, because you think Python is losing
space and turning into the future Pascal, because you hate 2to3, just because
you think is nice to have some news about Python... It would be very
interesting if some of you elaborate a little bit on what parts of this
article you agree with.

~~~
pyre
Not everyone up-votes because they agree with something. It's perfectly
reasonable to up-vote something with the hopes that it hits the front page
because you feel it will spawn an interesting discussion that you want to read
and/or participate in.

~~~
pitiburi
Exactly, that is why, when i saw it with more than 20 votes but not a single
comment that i dared to ask, honestly, why were they upvoting; I wanted to
know the reasons. As you say, it can be for many different reasons, and I was
interested in what the community thought about this. Sadly, that meant I could
be misunderstood, and in fact I was downvoted for just asking, in the first
comment on the new, to please know the REASONS the others were upvoting.

------
viraptor
Could someone explain this part?

> Now this all would not be a problem if the bytestring type would still exist
> on Python 3, but it does not. It was replaced by the byte type which does
> not behave like a string.

I was under impression that bytes is just an array of bytes and provides
pretty much what `str` provided. What big thing is missing from that
interface?

~~~
mcdonc
[http://docs.python.org/release/3.2.2/library/stdtypes.html#b...](http://docs.python.org/release/3.2.2/library/stdtypes.html#bytes-
and-byte-array-methods)

And this annoying feature:
[http://docs.python.org/dev/howto/pyporting.html#indexing-
byt...](http://docs.python.org/dev/howto/pyporting.html#indexing-bytes-
objects)

------
zbowling
I agree on the `estr' idea. I agree that it "punishes you" when you want to
try and deal with byte strings. It really gets in the way with handling
decoding and encoding of email.

------
perfunctory
"Python 3 ... does not offer much besides being more “correct”.

Since when correctness is not much?

~~~
fastviper
The context was XHTML. It was more correct but it was wrong (now dead) path.
XHTML created only problems, but was "more correct"

To be precise, XHTML promised that pages render faster, which turned out to be
browsers fault, not markup. PyPy solves "render fast" problem for python2.
What problem Python3 does solve? Unicode? No...

~~~
ricardobeat
XHTML was poised to offer much more than "correctedness". Reliable validation,
custom DTDs, extensions, better interoperability. It just wasn't meant for the
general web, where you have a massive number of non-technical authors. I don't
think this comparison has any place in this discussion.

~~~
dextorious
"""XHTML was poised to offer much more than "correctedness". Reliable
validation, custom DTDs, extensions, better interoperability. It just wasn't
meant for the general web, where you have a massive number of non-technical
authors."""

You just retold the whole of his Python 3 argument in terms of XHTML. P3 was
also poised to offer more than "correctedness", and also "wasn't meant for the
general real world where you have a massive number of different systems".

"""I don't think this comparison has any place in this discussion."""

Actually it's the perfect analogy.

XHTML -> add correctness, some new features, idealistic, unsuitable for the
real world, didn't catch on.

Python 3 -> add correctness, some new features, idealistic, unsuitable for the
real world, didn't catch on.

~~~
perfunctory
"I don't think this comparison has any place in this discussion"

I agree. Comparing markup languages to programming languages is even worse
than comparing js to assembly. But since we are on the topic... I am not quite
sure i undestand what " xhtml is unsuitable for the real world" means. Xhtml
is a contract between a content author and a browser. Why do we call the
browser's failure to implement the contract "unsuitable for the real world"?

------
swdunlop
Python 3 has one really significant problem for me -- many of my dependencies
don't support Python 3 well or at all. That keeps me and my own modules locked
in Python 2. Python used to be the language that bragged about coming with
"batteries included", but it is slowly becoming the language that requires new
batteries.

~~~
briancurtin
I'm also not sure your battery analogy works. The batteries that came with
Python 2 (the standard library) are still charged up in Python 3. Third-party
projects not porting certainly does affect you and others, but it's not the
same thing.

------
dcolish
For me, the strongest point is about which version of Python everyone uses at
work. When you have many commercial users its really difficult to get everyone
to move. Python3 is not currently a target for my code at work because just
writing the features is a full-time job. The difficulties in porting would not
currently be worth the effort and I would have an extremely hard time
justifying the ports business value.

------
kbd
What are the downsides of Ruby's alternate approach of having strings be bytes
that carry an encoding object around?

~~~
grandinj
Makes some string operations more expensive because of the potential of having
to convert between representations.

So you have the choice of converting at source, and paying the price there, or
converting during processing and paying the price there.

~~~
Someone
Lazy evaluation, however, has the benefit that, at least in theory, if
conversion turns out to be unnecessary, one can skip conversion, and never pay
the prize of conversion.

One could have an abstract 'String' type with concrete subclasses (ANSIString,
UTF8String, UTF16String, EBCDICString, etc)

Assuming that any to-be-handled character strings can be round-tripped through
UTF-8 (and that probably is a workable assumption), any function working with
strings could initially be implemented as:

\- convert input strings to some encoding that is known to be able to encode
all strings (UTF8 or UTF16 are obvious candidates)

\- do its work on the converted strings

\- return strings in any format it finds most suitable

Profiling, one would soon discover that certain operations (for example,
computing the length of a string) can be sped up by working on the native
formats. One then could provide specific implementations for the functions
with the largest memory/time overhead.

The end result _could_ be that one can write, say, a grep that can work with
EBCDIC, UTF8 or ISO8859-1, without ever converting strings internally. For
systems working with lots of text, that could decrease memory usage
significantly.

Among the disadvantages of such an approach are:

\- supporting multiple encodings efficiently will take significant time that,
perhaps, is better spent elsewhere.

\- the risk of obscure bugs increases ('string concatenation does not quite
work if string a is EBCDIC, and string b is ISO8859-7, and a ends with rare
character #x; somehow, the first character of b looses its diacritics in the
result')

\- a program/library that has that support will be larger. If a program works
with multiple encodings internally, its working set will be larger.

\- depending on the environment, the work (CPU time and/or programmer time)
needed to call the 'correct for the character encoding' variant of a function
can be too large (in particular, for functions that take multiple strings, it
may be hard to choose the 'best' encoding to work with; if one takes function
chains into account, the problem gets harder)

\- it would not make text handling any easier, as programmers would, forever,
have to keep specifying the encodings for the texts they read from, and write
to, files and the network.

[That last one probably is not that significant, as I doubt we will get at the
ideal world where all text is Unicode soon (and even there, one still has to
choose between UTF8 and UTF16, at the least)]

I am not aware of any system that has attempted to take this approach, but
would like to be educated on them.

------
dpkendal
Might Perl 6 suffer the same problems? Or will it be saved by virtue of its
compatibility mode for Perl 5 code?

~~~
iso8859-1
Funny how people claim Py3k was DOA. I don't know anyone who uses Perl 6 for
anything serious. I'd bet that there are 100 times as many Python 3
programmers as there are Perl 6 programmers.

I think Perl 6 still has a chance, though. No reason to dismiss it just
because it's developing slowly.

The compatibility mode doesn't do Perl 6 much good, if you ask me. There are
already too many ways to write Perl, if you ask a Python programmer. It's like
writing C in C++.

~~~
chromatic
Python 3 has been available and usable for quite some time. Not so Perl 6.

However, CPAN compatibility would be hugely important to a usable Perl 6, by
simple fact that no other language has the breadth and quality and
availability of libraries to rival the CPAN.

------
DrCatbox
It is very difficult for a developer to support two code-bases, for the "same"
language.

The `estr` suggestion is quite welcome.

------
perfunctory
"The multimethod based design of the language"

What is he talking about?

~~~
sirclueless
In a comment here on HN. the author clarified. What he means is that a whole
bunch of builtin functions operate by calling methods on the objects they are
passed.

int(n) --> n.__int__()

format(x, spec) --> x.__format__(spec)

~~~
perfunctory
It's not multimethods. It's just calling methods on the objects you are
passed.

------
ricardobeat
It strikes me that a developer working on the server side doesn't have at
least an idea of what POSIX is.

~~~
masklinn
You misread what he wrote. He knows what POSIX is in absolute, the question
was what "POSIX" is _as a locale_. It's in _the previous sentence_ :

> I had the situation that when I logged into my remote server _the locale was
> set to the string “POSIX”_.

~~~
ricardobeat
Ugh. That's what I get for skimming. Sorry.

