Those changes are necessary for improved performance, but not nearly good enough. PyPy does that and tons and tons of more stuff and it gets dismissed here as "provided only modest speedup on company workload", which means there is far more involved here than just attribute lookup/function call which have been massively speed up by pypy for quite a few years now.
That comment was about something slightly different though - even though richards is 40x faster, pypy still is not faster on some workloads (like instagram in their measurments), which makes me think that there are other forces at play.
> Mark Shannon said that Python 3.7 has added a feature that should provide a similar boost as the method-lookup caching used in the experiment.
As I understand it, class-attribute dicts now carry a version number which increments on every mutation. The first method lookup gets cached and subsequent lookups check the version number to decide whether to invalidate the cache.
People should not use CPython for speed; they should use PyPy for speed.
I agree with you though; one of the interesting and good things about Python is that it's a standard not an implementation. Although CPython is the the most popular by usage, PyPy and Cython are mainstream alternatives (or superset in the case of cython). There's also Jython, IronPython, Unladen Swallow, Grumpy, and others that I can't think of now. Some of those are defunct and others only are Python 2. But the point is, competition is good.
And keep an eye on the (poorly named) Grumpy project (Python compiled to Go.)
But memoizing lookups can be a single branch at the top of a few functions. We should strive to have our cake and eat it too, not simply declare we shouldn't bother.
I’d also add that the comments on the LWN article are good too.
@all if you like this article please pay for a subscription. LWN.net deserves our support.
Agreed. I subscribed a few months ago, in part because LWN.net seems to be the only publication providing significant coverage of the Python Language Summits.
It was mentioned, but in a rather dismissive way
Yeah, I know that’s more work that I can imagine, but I can dream, right?
(PyPy on LLVM has been a thing before. It requires constant upkeep and isn't very fast. I'm sure that the PyPy team would love to hear from prospective maintainers!)
It has the potential to bring an underlying framework which is built on industrial quality VM+JIT and it's primary goal is being compatible with the Scipy ecosystem at least ...Which is reason enough for unlimited optimism.
I'll skip over this mostly, just wondering what exactly you mean about this and whether you consider LLVM not to be "industrial quality" seeing as the failed Unladen Swallow project based itself upon that and it didn't seem to get them anywhere.
> it's primary goal is being compatible with the Scipy ecosystem at least
Well... it's not like PyPy isn't "compatible" with the scipy ecosystem. It just has to use a lower-performing object access mode to use cpyext-based extensions, which I suspect is a compromise any JIT-based implementation will need to make to be able to make use of these more old-school extensions.
Ironically on the subject of "industrial quality" JITs, GraalVM is based upon the same meta-tracing interpreter ideas that were largely pioneered by PyPy.
I'm not sure about your point. There is a new Python version built on Graal, that promises to largely maintain C compatibility (like graal Ruby) and still deliver performance.
I wish Pypy were getting funded by someone and have a lot of respect for what those guys achieved...But the fact remains that it is not bring used. Maybe graal Python can change that.
I used the words "industrial quality" instead of "pioneering" or "innovative". I think it's accepted that the millions of man hours spent on the JVM has made it one of the most incredible VM anywhere - is the defacto foundation on top of which you build big data (spark/hadoop), language theory (scala, closure, kotlin) and a billion mobile phones.
I think even in the frontend, the compiler try to guess the type beforehand, and anotate the real type for the compiler to optimize for the right type.
The problem is that i didnt dive that deep to know if a language as Python would be a good fit, nor im a "compiler guy" myself, so..
But i would love to do this as a backside project. The problem is my time is currently all taken by a big project.
But i would love to try to do this plug.. Thats why im winking here on HN. Maybe others also find a interesting thing to try themselves.
It seems nobody is willing to put big enough money behind making Python much faster. My view is that the limitations are almost purely financial (as in, paying heavily somebody as skillful as e.g. Mike Pall or Lars Bak(1) and his team), not technical.
If Guido would not accept the "faster" Python, the fork would still be more popular if it would be compatible enough. And there are the technical aspects: it's not enough to make Python interpreter alone faster, whoever would take that challenge would have to adapt various important external libraries to be really accepted. Which is AFAIK also doable.
The Python ecosystem in general is severely underfunded despite all big players using it extensively, which makes it really unfair if you compare it to the money poured into JS because of its monopoly on the web.
Remember Unladen shallow ? "Google" attempt to JIT Python ? It was just one guy during his internship (http://qinsb.blogspot.fr/2011/03/unladen-swallow-retrospecti...).
And look at the budget the PSF had in 2011 to help the community: http://pyfound.blogspot.fr/2012/01/psf-grants-over-37000-to-.... I mean, even today they have to go though so many shenanigans for barely 20k (https://www.python.org/psf/donations/2018-q2-drive/).
But at the same time you hear people complaining they yet can't migrate to Python 3 because they have millions of lines of Python. You hear of them when they want to extend the support for free, but never to support the community.
Python needs a sugar daddy. It's used in Mac and Linux. It's used at Microsoft, Google, Facebook, Nasa and so many more.
It is harder to make a fast python than to make a fast JS, but it's not that much harder.
Is a feature not a bug. It makes things like NumPy, SciPy and Pandas possible.
That is, I can understand that there's such a big installed based that people are loath to get rid of the Python/C extension API, but I think that's different than saying those projects are impossible without that extension API.
What are you talking about? The article is about the Python core dev group rejecting speedups that preserve C module compatibility for the sake of, uh, readability or something?
I like what you’re saying, but wonder if we made small incompatible changes over time, would that solve the problem? For example (and please forgive me on this), but there are so many similarities between Python and different languages. Objects are obviously everywhere - C++, Java, .Net, etc; and syntax’s are similar at a cursory glance to things like Fortran. All of the above are much faster.
We took a decade to go from Python 2 to 3, but that had some pretty big changes. Going from 3 to 4 and getting a 50% speedup while making some (hopefully small) incompatible changes would probably be a good motivator for people to migrate faster.
There are obviously pros and cons to this discussion, but i really believe that stagnation is the worst choice. (Ok, Perl made a worse choice, but I’m presuming we learned that the level of change from 2-3 is as far as we can go in a generational update (x.0) ).
That's way too low goal to even matter.
From the article: "The instrumented interpreter found that 70% of objects have all of their attributes set in the object's __init__() method."
As far as I know, V8 approach shines for such objects.
The PyPy team have put a lot of work into building a performant python implementation (there's your "fork" for you...) having to, from what I can tell, work through a lot of these issues with a good deal of ingenuity along the way.
Throwing money and/or "rockstar programmers" at projects isn't as wise as it always seems. Particularly when they don't seem to have significantly investigated the work of a team that's been working on the "fast python" problem for the last 15 years.
"in code like this with a string-valued foo() function:"
for x in mylist:
s += foo(x)
So yes, there's definitely a room to make a faster usable implementation compared to the current PyPy. But it is a hard work, and needs a very focused and knowable leader(s), willing to take the "harder" approaches on every level, when needed. "Harder" than "we'll do everything in Python and only then optimize the whole thing together."
Who knows, maybe releasing their experimental code would prompt some really good discussion and get things moving in a direction of speedy run times.
* The fashionable inner-circle of the current economic winners, making the academic who "works for so-and-so" an immediate authority. Think for yourself! Wealth-makes-leadership leads to some sick outcomes, frankly. Sure, some academic compiler writer knows his function call stats, but that doesnt suddenly make the years and years of participatory work by many hands, less relevent. This is not populist, but rather pragmatic.
* Comparison to yet-another Python 3.x development. Great! Python evolves.. but lets not throw out a stable binary system with well-understood characteristics.. and that is.. Python 2.7
very interesting peek into the phenomenon of this language
The LWN piece says "Guido van Rossum, who loudly objected to Shapiro's tone, which was condescending, he said".
That sounds appropriate.
In your experience, do most people presenting an 'ordinary technical discussion' use a condescending tone? If so, I'm glad I don't work in your organization.
Otherwise, when should people complain when speaker is disparaging most of the people in the audience, even if accidentally?