
Personal thoughts about Pyston's outcome - open-source-ux
http://blog.kevmod.com/2017/02/personal-thoughts-about-pystons-outcome/
======
mhneu
Key takeaway: _" the [speed] difficulties come from Python's extremely rich
object model, not from anything about its dynamic scopes or dynamic types. The
problem is that every operation in Python will typically have multiple points
at which the user can override the behavior, and these features are used,
often very extensively."_

Also, Kevin and Pyston's team are too modest to state this directly in this
article: They made good and steady progress, but it's taken them a long time.
A lot of that time was spent making them experts on Python runtime and Python
JIT options. With several more years of fulltime work we could have a JIT that
we all use instead of CPython.

Pyston showed that certain features of Python make it slow and difficult to
JIT. The best case scenario for Python's future would have been for Kevin and
Pyston to work with the Python core team to decide how to tweak language
features for speed (perhaps marking certain object modifications as being
"slow" in the same way rust has an "unsafe" marker). With Pyston shutting
down, I think the horse of python speedups is out of the barn and is never
coming back.

More discussion:
[https://news.ycombinator.com/item?id=13534992](https://news.ycombinator.com/item?id=13534992)

~~~
tbrowbdidnso
I don't get why it seems like languages are designed for convenience then
later down the road billions of hours are spent trying to get better
performance.

I have seen this happen over and over with PHP, Ruby, js, and now Python.

The best solution is to stop using a slow language for these types of
applications. Personally I don't see a point to investing so much in these
projects, since all it does is encourage developers to continue their bad
behavior.

"Oh don't worry about it, they're working on a JIT if performance is ever an
issue" \-- lead dev deciding to write something that is expected to handle 10k
transactions per second a few years later in Python.

If performance is ever going to be important just use a high performance
language and be done with it. Then you don't have to spend years and millions
reinventing the wheel like Facebook did with PHP.

~~~
gsnedders
Every single company that has invested heavily in JITs for PHP, Ruby, Python
etc. has done so because they have vast codebases, often millions of lines,
written in those languages and rewriting their codebase is infeasible: getting
a 20% performance increase for everything from a dozen man-years is vastly
cheaper than trying to optimize their entire codebase similarly yet alone
rewrite large parts of the application in another language.

Why start writing something in one of those languages? They have high-quality
frameworks that make it quite easy to ship early and ship fast: the vast
majority of web-based startups fail, or at most stagnate with at most tens of
thousands of users, and for them the performance of the common interpreters is
fine. If we want to see a phase shift away from them, we need frameworks that
allow quick prototyping written in languages which are more easily optimised
(and, honestly, JavaScript _really_ isn't bad in terms of writing a JITing
compiler).

~~~
tbrowbdidnso
The point I was trying to make is that the idea that a JIT or compiled version
will come along in a few years drives people to use slow languages for high
performance projects in the first place. This creates a cycle where the
language is used in a way that a JIT coming along is a self-fulfilling
prophesy.

The irony is that the companies investing the most money into eventual
JIT/compilation are the same ones that wrote code believing it would "come
around" one day.

The only way to stop the madness is to avoid using these languages for high
performance code in the first place.

Having used many fast/slow languages over the years I'll say I don't really
notice any increase in productivity one way or another, and no difference in
the richness of the library ecosystem.

Are there any studies that show more dynamic languages are more productive?
Usually the argument is made that the more abstracted the language is from
hardware the better it is to work in, but I don't notice a productivity
difference in C#/Java vs Python/perl

~~~
gsnedders
I'm not sure people have this idea that a JIT or AOT compiled version will
come along; certainly ten years ago nobody really had that impression, has the
rise of the JS VMs really changed this so much?

To take FB as an example: AFAIK, it was written in PHP because Zuckerberg knew
PHP. There was no expectation it would "come around" and it continued to be in
PHP because bills had to be paid and rewriting everything would stop new
features from being launched and hopefully reach profitability.

If you're a small startup your first goal is almost always to hit
profitability, and then your goal is almost always to maintain that. As a
result, language choices are frequently made by what founders know rather than
any technical merit.

------
denfromufa
Michael Kennedy mentioned on podcast that Dropbox is considering Python 3 with
type hints. Pyston is Python 2 only and considerable effort would take to make
it work for Python 3.

PyPy is not that bad as they describe it in incentives to start Pyston. PyPy
now has cpyext and Python 3.5 support is funded by Mozilla.

Another reason is pluggable JIT to CPython such as Pyjion. This is already
possible in Python 3.6.

Currently you can use Cython for hot spots in your CPython apps and even
release GIL.

~~~
mhneu
"the [speed] difficulties come from Python's extremely rich object model, not
from anything about its dynamic scopes or dynamic types. The problem is that
every operation in Python will typically have multiple points at which the
user can override the behavior, and these features are used, often very
extensively. Some examples are inspecting the locals of a frame after the
frame has exited, mutating functions in-place, or even something as banal as
overriding isinstance."

It's not typing that's the problem, it's dynamic behavior like monkeypatchable
methods which require dict lookups.

~~~
denfromufa
Nowhere did I state that typing is the problem. I was explaining potential
move to Python 3 with mypy.

------
BuuQu9hu
"We needed a high level of compatibility and reasonable performance gains on
complex, real-world workloads. I think this is a case that PyPy has not been
able to crack, and in my opinion is why they are not enjoying higher levels of
success."

It's a little tiring, as somebody who regularly puts production workloads onto
PyPy, to hear this bullshit. The truth is that PyPy has been regularly
spanking Pyston on benchmarks[1] and has been adding Python 3 support while
Pyston has struggled to handle its Python 2 commitments.

It's great that you wanted to build a JIT. JITs are fun and cool. But if you
just wanted shit to work, you could have used PyPy _years_ ago and not jumped
through all these hoops.

Yet again, the fact is that if you invested any code into C extension modules,
you picked the wrong path. Good night, Pyston; you were a cool experiment and
I hope that your contributors try sending patches to PyPy next time. (Pyjion,
you're next.)

[1]
[https://pybenchmarks.org/u64q/benchmark.php?test=all&lang=py...](https://pybenchmarks.org/u64q/benchmark.php?test=all&lang=pyston&lang2=pypy&data=u64q)

~~~
richardwhiuk
Not being able to use C extension modules is a killer feature for lots of
large Python code bases.

------
scriptkiddy
I for one applaud Dropbox's devotion to Python. They are really helping to
move the Python community forward.

I also think Kevin hits the nail on the head regarding what makes speeding up
Python difficult. It's the same thing that makes writing Python so enjoyable:
the object model.

------
curtis
This is not specific to just Python, but I have often wondered if it would be
a productive exercise to write a _static_ compiler for a dynamic language
which also supported optional static typing. Presumably this compiler could
generate code that is faster than an interpreter like CPython, but slower than
what you can get out of a high performance JIT, since a JIT can do dynamic
optimizations and function inlining. However, once you add static type
notations, the static compiler can (for some cases) generate code that's
actually faster than a JIT. Typically you'd only need to add type notations to
hot spot code, and you could use profiling from previous runs to figure out
what type annotations to use and where to put them.

Using a JIT to generate very fast code for dynamic (even highly dynamic)
languages is a really interesting topic. But it seems to involve jumping
through a lot of hoops to get performance that is a fraction of what you can
get out of an early 1990's era C compiler (or Pascal compiler, for that
matter).

~~~
munificent
I'm on the Dart team, so this is a topic that's close to my heart.

> Presumably this compiler could generate code that is faster than an
> interpreter like CPython,

People have been writing static compilers for dynamic languages for ages. Lisp
and Scheme have been compiled for a long time.

My understanding is even simple static compilation works tolerably well for
something like Scheme, but object-oriented languages like Python are a
different story.

In object-oriented languages, the majority of "calls" are potentially-
polymorphic method invocations. In Python, even `a + b` may call some user-
defined "+" method which must be looked up at runtime.

If you statically compile that in a straightforward way, almost all of the
resulting "static" code is just going to be calls to dynamic method lookup
functions. Something like:

    
    
        Obj* temp1 = lookup("a");
        Obj* temp2 = lookup("b");
        Obj* temp3 = invoke(temp1, temp2, "+");
    

You won't really _do_ anything useful at the level of native code, so you
don't gain much from eliminating the interpreter's bytecode loop. In fact,
you'll likely lose. That above chunk of C is a pretty verbose encoding of `a +
b`.

You can look at bytecode is a compression format for dynamic code of that
form. The runtime cost of "decompressing it" \-- the bytecode interpret-and-
decode loop may be a net win over the memory wasted by the larger native code
size and the cache misses and stuff caused by that.

In order to statically compile a very highly dynamic language, you really do
need to do some static analysis to eliminate some of the dynamism. That
analysis is either very costly (slow, giant monolithic whole program compiles
that take ages as in MLton) or doesn't work very well (failing to infer good
static types and resorting back to lots of slow dynamic code), or (most often)
both.

With Dart, we have a _very_ mature whole program compiler that does concrete
type inference, and it's still both too slow to run and not effective enough
at inferring good types.

> However, once you add static type notations, the static compiler can (for
> some cases) generate code that's actually faster than a JIT.

You can do that, yes, but it requires a few things:

1\. Your core libraries and idiomatic user code need to actually be fairly
"static" in terms of the types they use at runtime. If the code really does
use a lot of polymorphism and duck typing, all static analysis can do is tell
you, "Yup. You gotta do this at runtime."

Python in particular is well-known for relying heavily on duck typing in its
core libraries.

2\. Your type system needs to be sound. Once you are within the bounds of
static types, the compiler needs to be able to trust that those types are
_correct_ , with no soundness holes. Otherwise, it all falls apart.

3\. You need to deal with the boundary between dynamic and typed code. Because
of 1, you need to do validation when an object flows from dynamic-land into
the typed area. That validation occurs at runtime and has its own cost. If you
aren't careful, that runtime cost outweighs the perf benefits you get once you
are in static code.

The "gradual typing" folks have been beating on this for years and it turns
out to be _really_ hard. Google "is gradual typing dead" to see some of the
latest soul-searching.

With Dart, we had 1 since the language has always had an optional static type
so typical Dart code is fairly Java-esque. We did not have 2 and that turned
out to be a big problem. We have since come up with a new sound type system
[1] to address that.

3 is hard, but our strategy has been to be more restrictive on what kinds of
dynamic code we allow to flow into typed code. For example, you simply can't
pass a List<dynamic> into a function that expects a List<int>.

By the time you do all of these things, I think you discover that you don't
really have much dynamism left in your system. It is handy having a dynamic
type for some places, like deserialization, JSON, etc. But if you care about
static compilation and perf, actually _being_ statically typed is just so much
easier, and dynamic typing doesn't really offer that much value as an
alternative.

[1]: [http://news.dartlang.org/2017/01/sound-dart-and-strong-
mode....](http://news.dartlang.org/2017/01/sound-dart-and-strong-mode.html)

~~~
pg314
> In order to statically compile a very highly dynamic language, you really do
> need to do some static analysis to eliminate some of the dynamism.

Or you can require the user to add type annotations, if/when he wants fast
code.

In SBCL (a compiler for Common Lisp):

    
    
      (defun count-even (array)
        (declare (optimize speed (safety 0)))   ; <- these are the optimisation settings for this function
        (declare ((simple-array fixnum) array)) ; <- the argument to the function is an array of fixed sized integers
        (loop for x across array count (evenp x)))
    

This compiles down to quite efficient assembly:

    
    
       > (disassemble 'count-even)
       ; disassembly for COUNT-EVEN
       ; Size: 65 bytes
       ; 05A7ABC2:       BB17001020       MOV EBX, 537919511            ; no-arg-parsing entry point
       ;      BC7:       498BF8           MOV RDI, R8
       ;      BCA:       31C9             XOR ECX, ECX
       ;      BCC:       31F6             XOR ESI, ESI
       ;      BCE:       31D2             XOR EDX, EDX
       ;      BD0:       498B70F9         MOV RSI, [R8-7]
       ;      BD4:       660F1F840000000000 NOP
       ;      BDD:       0F1F00           NOP
       ;      BE0: L0:   4839F1           CMP RCX, RSI
       ;      BE3:       7D18             JNL L1
       ;      BE5:       488B5C8F01       MOV RBX, [RDI+RCX*4+1]
       ;      BEA:       4883C102         ADD RCX, 2
       ;      BEE:       4883E302         AND RBX, 2
       ;      BF2:       4885DB           TEST RBX, RBX
       ;      BF5:       75E9             JNE L0
       ;      BF7:       4883C202         ADD RDX, 2
       ;      BFB:       EBE3             JMP L0
       ;      BFD: L1:   488BE5           MOV RSP, RBP
       ;      C00:       F8               CLC
       ;      C01:       5D               POP RBP
       ;      C02:       C3               RET
    

If you don't add the type annotation (the second declare form), the compiler
will tell you that it can't optimise the code because of type uncertainty.
Optimising a program becomes a dialogue between the user and the compiler.

~~~
int_19h
The entire second half of the comment you've replied to addresses this
approach at great length.

------
LeanderK
It's hard for me to believe that it's easier (and not only more comfortable
for the engineers) to start pyston rather then rewrite their backend in go or
whatever. Is dropbox really so big (not in sales, in the size of their
backend) that they should manage their own python implementation? Also
choosing python for their backend really seems like the wrong language.

Compilers (especially JIT-compilers) are hard to get right and even harder to
get performant. Since i am interested in these areas i would evaluate
Graal/truffle or Jython. I only looked briefly at it, but Jython seems very
un-optimised and the JVM really improved it scripting-language support in the
recent years. Is there really no way to solve the native-extension problem of
Jython?

But i maybe wrong, any comments?

~~~
mhneu
_It 's hard for me to believe that it's easier (and not only more comfortable
for the engineers) to start pyston rather then rewrite their backend in go or
whatever._

You are correct, it is easier to rewrite in Go. That's what Dropbox learned
after putting two FTEs on Pyston for a few years.

But the larger Python community outside of just Dropbox would have really
benefited from these two guys continuing to work on Pyston.

~~~
LeanderK
Of course, getting a fast JIT-Compiler for Python would be awesome! I have
been doing some python lately and it's a nice language. I would guess there is
such a broad usage in the industry that a committee funded by various
companies could take ownership of the project and greatly increase the
assigned resources.

I just question whether it's the right move for dropbox. Seems like not
actually tackling the problem and instead trying everything so that it just
disappears.

------
raverbashing
"Some examples are inspecting the locals of a frame after the frame has
exited, mutating functions in-place, or even something as banal as overriding
isinstance"

Ok, they're doing all of this in their codebase?

This seems like "trying too hard". Yes, they can do those. It doesn't mean
it's a good idea

~~~
BuuQu9hu
If you don't do this, then it's not Python.

~~~
raverbashing
I've yet to see (real-life) code that needs these.

~~~
icebraining
It's useful when hacking around someone else's code that you aren't supposed
to change. Not sure why Dropbox would need it, though.

------
my123
IronPython uses the Microsoft DLR, which uses a JIT, so it's definitely
possible that it would work.

~~~
mhneu
They can't support C extensions, which are basically required for any real
world Python.

Pyjion is a Microsoft CoreCLR JIT that can support C extensions. That's
Python's next great hope for speedup via JIT.

~~~
denfromufa
Pyjion is C Extension based JIT for CPython. Pyjion only uses CoreCLR RyuJIT
as a JIT compiler backend. It is NOT aware of .NET types at all.

Note that you can use pythonnet for interop between CPython and .NET for both
embedding and extending.

