On: Nuitka: a Python compiler

[jarcane]

It is somewhat faster than CPython already, but currently it doesn't make all the optimizations possible, but a 258% factor on pystone is a good start (number is from version 0.3.11).

Holy cow. And here I'd set aside Python in part because of the difficulty in getting performant executables (I was getting visible lag in a turn-based SDL game ...)

[syllogism → jarcane]

Use Cython.

It's a hassle getting your head around it, and getting it set up, but you'll never look back once you do.

If you rely on PyPy, you get what you're given and then you're stuck. You can't really guess how to rewrite your code to make it faster.

A small blog post I wrote on this: https://honnibal.wordpress.com/2014/10/21/writing-c-in-cytho...

Here's a non-trivial example: https://github.com/honnibal/thinc/blob/master/thinc/learner....

This code is driving a library of very fast NLP tools that I'm writing.

[shadowmint → syllogism]

Unforntuately cython is even harder to package up than normal python because it generates one shared library per module.

...not really suitable for games.

[TillE → shadowmint]

You may need to fiddle around with how modules are loaded by Python, but you can do whatever you want with the C files that Cython generates.

kivy-ios should have all the details for building everything into one big Python blob, if that's what you need.

[gcr → syllogism]

Amen to Cython. When used well (eg. with C types for variables and a few compiler directives, which is not very painful at all), Cython emits code that is extremely similar to idiomatic C for my scientific computing workloads.

[loqi → jarcane]

Might want to take "somewhat faster" with a grain of salt. I just tried it on a smallish program (CPU-bound, mostly string operations) and got a 20x slowdown.

[towelguy → jarcane]

They have a site showing benchmarks http://speedcenter.nuitka.net/ (down at the moment), but here's a google cache of one, showing also the generated C code: https://webcache.googleusercontent.com/search?q=cache:speedc...

[bhouston → jarcane]

But how does it compare to PyPy?

[jarcane → bhouston]

To be honest, I found that in my use case (game dev, supported by bindings to a C library), PyPy was actually slower than CPython. I'd be curious to dig up my code and compare performance to a Nuitka-compiled executable.

[detaro → jarcane]

out of curiosity, which bindings did you use?

[jarcane → detaro]

libtcod. It was a roguelike.

[ptx → jarcane]

Those Python bindings seem to use ctypes, which is indeed slower[1] on PyPy. They recommend[2] using cffi instead.

[1] http://pypy.org/performance.html

[2] http://pypy.readthedocs.org/en/latest/extending.html

[illumen → bhouston]

For games, other soft realtime, and run once code, pypy is slower. The newer GC is better, but the JIT still causes pauses. The slow warm up of the jit, no type hint saving, and slower interpreter is why it is not good on run once code. Of course if your python extensions are calling optimized assembly routines, or hardware, then it will be faster than pypy code written in python.

Also, using a JIT is not possible on some platforms (iOS, or if the CPU architecture isn't supported by pypy).

Not to say that pypy isn't better for certain tasks of course.

[bhouston → illumen]

The solution is easy then, PyPy should have an AOT compilation option, like ART does.

[jfeser → bhouston]

That's pretty much what Nuitka is. Unfortunately, PyPy is only fast because it can take advantage of runtime information in the JIT. AOT compilation loses that advantage.

[ris → bhouston]

Not easy at all. ART generally compiles quite static languages (Java). Python is an incredibly dynamic language. The challenges for AOT compilation are vastly different.