
The Python I Would Like To See - rachbelaid
http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to-see/
======
pixelmonkey
It's an interesting article, as a tour of some Python internals.

But it comes off as a rant without a real suggestion.

Python 3 was an internal interpreter cleanup. That's actually part of its lack
of widespread popularity. The core developers didn't add a ton of unique
functionality (and most of the new stuff was backported to 2.7 anyway), but
they fixed some annoying problems in the CPython codebase and broke some
things in the name of cleanup/unification of concepts. They made Python easier
to maintain and easier to build new features atop. They sharpened the axe.

Yes, they didn't clean up Armin's pet thing -- slots. And they created new
problems for library developers in their bytes vs unicode changes. But they
cleaned up a whole lot of other stuff.

I personally think with the maturity of pypy and the stability of both the 2.7
and 3.4 lines of development, the Python ecosystem has never been more
exciting. The advances in pypy make Python attractive for CPU-bound work and
the inclusion of asyncio in the stdlib will make it more and more attractive
for I/O-bound work over time. Python has long been a winner for mixed
workloads, and the ecosystem around Python, especially pydata utilities like
numpy and pandas, keep getting better. Stop complaining -- let's just build an
awesome future atop this marvelous language.

~~~
the_mitsuhiko
> But it comes off as a rant without a real suggestion.

How exactly is that a rant? It's an exploration about a design decision /
mistake that was carried through Python for 25 years and has left an impact.
There is no complaint and there is no suggestion for Python.

As I said in the early paragraph it's something that's interesting for people
that are interested in language design.

~~~
pixelmonkey
I think I mis-spoke. I didn't really mean "rant", I meant "nit-pick". It's
akin to the frequent complaints I hear about `len()` being a function rather
than a method on `list` and `str` types.

This internal detail you focused on solves real problems; it's also a wart
internally. OK, we get it. It's also not a hugely important wart, IMO. I don't
think it's holding Python back.

But, like I said, I think the article was an interesting walkthrough of the
internals. I enjoyed it overall :)

~~~
hyperpape
I think this is a bad comparison, because len() vs. foo.len() is just syntax,
and isolated syntax at that. It's valuable to get syntax right, but this
particular decision doesn't really impact anything outside of itself.

In contrast, the article suggests that the implementation of method dispatch
slots (not __slots__!) actually constraints the optimizations you can make.

I get that you're trying to suggest that they're both small, but I think that
this analogy only serves to confuse things.

------
nakovet
I follow Ronacher's work, open source and posts and I agree with his
arguments.

This post remember me of Spolsky's one
([http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...](http://www.joelonsoftware.com/articles/LeakyAbstractions.html))
though. When you get to a point where you get to know what's under the hood
and why it's not working the way you are trying to use, however in a majority
of cases you are just fine and don't need to know what's happening under the
hood and maybe if this majority is big enough that's just fine.

~~~
PhantomGremlin
I got a totally different impression from the post. What I thought of was
([http://en.wikipedia.org/wiki/How_many_angels_can_dance_on_th...](http://en.wikipedia.org/wiki/How_many_angels_can_dance_on_the_head_of_a_pin%3F)).

In particular:

    
    
       In modern usage, this question also serves as a
       metaphor for wasting time debating topics of no
       practical value, or questions whose answers hold
       no intellectual consequence.
    

Python will (or won't) be used for a particular application regardless of
whether some contrived test takes 0.158 usec or takes 0.256 usec per
iteration.

This sort of misunderstanding comes up so often that there are a plethora of
cliches for it. Here's another one: "Missing the forest for the trees.".

------
philh

        >>> original = 42
        >>> class FooProxy:
        ...  def __getattr__(self, x):
        ...   return getattr(original, x)
        ...
        >>> proxy = FooProxy()
        >>> proxy
        42
    

For anyone else who found this as confusing as I did ("wtf, how can proxy
actually be 42?"), what's going on here is that it's calling
`proxy.__repr__()` when it attempts to display `proxy`, which in turn calls
`42.__repr__()`. (Similarly, `proxy + 1` calls `42.__add__(1)`.)

------
GnarfGnarf
I'm learning Python and I must say it is a wonderful language. Being able to
concatenate strings by simply saying "a + b" is a great productivity boost
(coming from C++). Python libraries are powerful. I can read an Excel
spreadsheet with one line of code. I can create plots in PDF format with a
half-dozen lines. Amazing!

However, I am disappointed with the difficulty of turning a program into a
Windows EXE. I wrote a small program (couple of thousand lines), tried Py2exe
which failed to handle the Numpy (or Matplotlib, I forget which) imbroglio.

PyInstaller works, except that the EXE is 85MB, and takes one minute to start
up. Not practical for customer distribution. I can't expect my customers to
install the Python runtime. In contrast, my 500KLOC C++ program, with all its
third-party libraries, is 19MB. Yes, I know, Python needs everything including
the kitchen sink. Still, 85MB is not practical.

Too bad. Not all programs run on a server.

~~~
theseoafs
> Being able to concatenate strings by simply saying "a + b" is a great
> productivity boost (coming from C++)

C++ has had this operator overloaded for strings for decades.

With regard to the size of compiled executables, I can't really say much
except "that's not what it's made for". If you need to ship compiled
executables to people, Python is an extraordinarily inappropriate choice;
stick with C++.

~~~
ma2rten
Dropbox seems to be doing ok shipping python to customers.

~~~
koshak
If they were ok why should they had forked python?

------
berdario
I was a bit puzzled by the "lightweight old style classes" remark, so I tried
it out:

    
    
        >>> sys.getsizeof(OldStyleClass), sys.getsizeof(NewStyleClass)
        (104, 904)
        >>> sys.getsizeof(OldStyleClass()), sys.getsizeof(NewStyleClass())
        (72, 64)
    

I agree that OldStyleClasses might be simpler (while less featureful), but I
think I'd care more for the footprint of instances, rather than class objects
themselves

------
orf
I switched to Python 3 this year and I haven't looked back -there are some
niggles, but overall its a great improvement.

A great feature that's not really talked about is the __prepare__ function in
metaclasses: you can supply a custom type that stores all class members. You
could whip up multiple-dispatch using this (it lets you handle classes with
duplicate property names) in conjunction with signature annotations, which I
think is pretty neat.

~~~
Sami_Lehtinen
This has been one of the reasons why I'm not currently doing anything for
Google App Engine, they still lack Python 3 support. It feels awkward to code
in Python 2 after Python 3.

------
hosay123
This post is surprisingly confused, it is phrased as a complaint about the
language, then immediately degrades into CPython implementation specifics that
have little bearing on the usability of the language itself.

Ronacher should also know better than to post microbenchmarks like the one
provided here, especially without corresponding (C) profiler output. At the C
level, slots allow the implementation constant-time access to the most common
code paths for an object, and especially when you have C code calling other C
code via the type system (IMHO the primary use for Python, and still its
strongest use case), "interpreter overhead" is reduced to a few extra memory
indirection operations.

In the alternative world, sure, perhaps some microbenchmark may behave faster,
but now systemically, and for e.g. "reduce(operator.add, range(1000))"
requires more hash table lookups than I can count.

Python is all about providing a lightweight way to compose bits of fast code
(the kernel, network stack, NumPy, MySQL, whatever). Unfortunately somewhere
along the way solutions like Django got popular, which are almost the
antithesis to this old viewpoint. Ronacher seems to be advocating that we
should punish the CPython implementation's traditional strong areas in favour
of.. actually, he didn't even describe any vision that was impacted by his
complaints. He just seems to want the CPython implementation to suffer.

Perhaps his complaint about new-style __getattribute__ would be better suited
as a bug report, it seems the only substantial observation made about the
language itself in this post.

~~~
the_mitsuhiko
> This post is surprisingly confused, it is phrased as a complaint about the
> language, then immediately degrades into CPython implementation-specifics
> that have little bearing on the actual usability of the language itself.

You might think that, but you are very wrong and I should probably make that
point in another blog post. These CPython specifics are enshrined in the
language. While PyPy does not have the actual structs, it needs to implement
the same user exposed API as it slots were used.

PyPy cannot just say "a + b" means "a.__add__(b)", it needs to implement the
exact same dispatch logic that CPython has because people's code depends on
it.

//EDIT:

> Ronacher seems to be advocating that we should punish the CPython
> implementation's traditional strong areas in favour of.. actually, he didn't
> even describe any vision that was impacted by his complaints.

Maybe I did not make my point very clear but the whole last paragraph
advocates about trying a version of Python that abolishes the internal slot
system.

~~~
hosay123
Once again, this sounds like a bug report, not a ranty blog post

~~~
the_mitsuhiko
And what exactly should the bug report be? "I do not agree with design
decision made 25 years ago, please make backwards incompatible change?".

~~~
hosay123
The first might be "Let's replace all use of tp_* with hash lookups", which I
suppose would be closed quickly, amidst a storm of muffled giggles. But at
least by then you'd know why the current implementation works the way it does.

The next might be "__getattribute__ mechanism has surprising behaviour in
common use-case" (which itself would imply having a use case where any of this
mattered). This one might actually result in some productive discussion
regarding a fix.

Your second reply is basically a doc bug, one of "__getattribute__
optimization is not specified in the language reference" or "__getattribute__
optimization causes non-comformance with the language specification", either
way, you would get the attention of the few people who can actually help you,
rather than attention from the many more who can't

~~~
the_mitsuhiko
You're assuming I am not aware why the system works the way it does currently
which is incorrect. I outlined what a Python could look like, not what I want
the CPython guys to implement. I am perfectly aware that this is not going to
happen.

~~~
maxerickson
I think if you had titled the post differently you would have ended up with a
lot less argumentation about how what you propose isn't a good idea for
python.

As it is, the title at least primes the reader to think you are asking for
what you go on to describe.

------
mpdehaan2
I think most of the things I'd want to see first would come in the standard
library and in unicode handling.

I'd probably argue that most uses of metaclasses are a reason to step back and
make the code simpler. There are a few cases where they come in very usefully,
but at least in open source code, they create a barrier to understanding and
increase complexity.

ython starts to get ugly when you start writing code that does "automagical
things", and that's somewhat tolerated because it's not a language for
building automagical things.

Or, to say it another way, if you have to think about how the interpreter
works, your python has jumped off the idiomatic wagon a while ago.

That being said __slots__ as a performance enhancement is crazy useful - and
I'd like to see more things in that area. Though for most people, the thing
that would lead to cleaner code would be a more powerful and elegant standard
library.

~~~
masklinn
1\. __slots__ and TFA have nothing to do with one another

2\. __slots__ are not actually useful, Pypy does what they do by default and
deoptimizes as needed, CPython probably could as well

3\. the only thing __slots__ do is lower memory usage (by not allocating a
dict per instance) and PEP 412 has already improved _that_

__slots__ is not "crazy useful" by any definition of the expression, it is
somewhat useful when you have a huge living number of simple objects.

~~~
PythonicAlpha
Sounds as, that you did not understand, that the author did not mean the
__slots__ system, that allows to avoid the __dict__ on objects, but the
internal slots-system that is intended to speed up the dispatching of special
methods.

~~~
thu
You missed his point 1.

~~~
PythonicAlpha
Ok. Then my answer belongs to the first in the sub-tree. What I actually did
not understand: What means TFA?

~~~
stewartbutler
The Freaking Article.

~~~
PythonicAlpha
Thanks! That helps.

------
alberth
From the OP:

>>"In recent years there is a clear trend of making Python more complex as a
language. I would like to see the inverse of that trend. I would like to see
an internal interpreter design could be based on interpreters that work
independent of each other, with local base types and more, similar to how
JavaScript works."

Sounds like OP should investigate Lua (and LuaJIT in particular).

He might like Lua more than Python.

(Given that OP created Flask, I'd love to see a Flask equivalent developed in
Lua)

Edit: typo

~~~
scrollaway
Good job not reading the article and skipping to the end, I guess?

>> This is in fact how many other dynamic languages work. For instance this is
how lua implementations operate, how javascript engines work etc. The clear
advantage is that you can have two interpreters. What a novel concept.

------
futuredale
Ah nice. I was always curious why __getattribute__ didn't fire for specific
dunder methods. Made subclassing pandas objects much more complicated.

------
xorcist
Armin for BDFL!

(Just a joke, people. Just a joke.) Anyway, I wonder why Armin's not more
involved with core Python things, his perspective should be valuable. As
someone doing some Python stuff on the side his articles are always worth
reading carefully.

The Python community has always been the best part of the language, but the
important subcommunities like NumPy, Twisted and PyPy has always seemed like
they are a bit outside looking in. I don't know why. Perhaps the language
would have evolved differently if these projects were a bit more involved in
the actual language development.

------
rattray
It seems pretty clear to me that we're ready for Python 4 by now. Experienced
Python devs know the problems. We even have some solutions.

So I think it's about time that leaders of the community like Ronacher, Rietz,
Gaynor, etc come together to talk about the next generator of Python.

We have PyPy now; why not use the tremendous advances there in the next
Python?

We have Go and Javascript to compete with and we understand a lot of why
they're winning; it's time Pythonistas fought back instead of just leaving.

Python 3 was supposed to be "the next thing" five years ago; it fizzled. Let's
make the real "next thing" for 2016. Python 4.

~~~
pixelmonkey
Go and Javascript are "winning"? That's news to me. They might be buzzier
right now, but their compiler/interpreters only appeared in 2009 and Python
has been around since 1991. Is it really such a surprise that the hype period
is over for Python as a language spec?

Now people are just getting a lot of stuff done with this productive language
and nobody needs to be convinced any longer how good a language it is. Most
people already know.

Now, we're onto the good work of building a huge ecosystem of open source
modules atop it. Call me when Go and JavaScript have the equivalent of the
PyData stack and rock-solid drivers for every database technology on the
planet. Then maybe I'll look at them for anything other than niche use cases.
(Obviously JavaScript is still the only game in town inside the browser and Go
seems like a pretty interesting alternative to C for UNIX system utilities.)

~~~
coldtea
> _Go and Javascript are "winning"? That's news to me. They might be buzzier
> right now, but their compiler/interpreters only appeared in 2009 and Python
> has been around since 1991. Is it really such a surprise that the hype
> period is over for Python as a language spec?_

It's not about "hype period". It's about moving on with the times to avoid
becoming the next COBOL or TCL. Being even older and problematic than Python
haven't stoped a language like C++ to ease its users problems and make them
happy with C++11. Javascript, almost as old as Python (1995 vs 1991) got
20-100x speed boost with the 2006-era generation of JITs.

A ho-hum remake, like Python 3, that breaks compatibility without much
important to offer, doesn't cut it. If you're gonna break compatibility solve
real problems people have. A large speed boost (entirely possible as V8,
LuaJIT and co have shown even for a most dynamic language) is a great thing to
entice upgrades. A good async/multicore story also. The removal of GIL. Etc.
Those are things people have been nagging Python devs about, not improved
Unicode or print as a function.

> _Now people are just getting a lot of stuff done with this productive
> language and nobody needs to be convinced any longer how good a language it
> is. Most people already know._

Languages that nearly vanished from the job market and current use trends,
like Smalltalk, TCL and Perl also said the same thing...

~~~
orangecat
_A ho-hum remake, like Python 3, that breaks compatibility without much
important to offer, doesn 't cut it. If you're gonna break compatibility solve
real problems people have._

Exactly this. Python 3 was an unfortunate triumph of purity over practicality.
We'd be in a much better place if half of the effort expended in the Python 3
transition was instead focused on the issues you listed, and I'd also include
browser support by compilation to JS.

------
PythonicAlpha
I really would like to see, when the suggested, simpler system would be tried.
It is sometimes amazing, how by simplification gains could be reached
_against_ common wisdom. And even, when no gains or little losses would be
there with the new, simpler system, it could be worthwhile to change, since
the current system has his huddles, that I also already stumbled about, and
which are not clearly communicated, since it is such a deep implementation
detail of CPython. Also it was mentioned, that a simpler system could be even
more powerful.

------
fndrplayer13
Thanks for the very informative post. I've been a full-time Python developer
for only the past 7 or 8 months now. I had not in that time learned about the
slot system or the CPython implementation.

Also, <3 Flask. So good.

------
scardine
tldr; method calling is unnecessarily expensive in current Python due to
debatable design decisions.

------
notastartup
I moved from Javascript to Python 2.7 and haven't looked back since. I now
strictly use Python in all of my projects, Javascript only because browser
doesn't support it.

I wish that Python would introduce a few things from the Javascript world.
Easy async, and a real-time Meteor like framework.

~~~
lmm
Improved async is the biggest new feature in 3.4

