
Nuitka: a Python compiler - woadwarrior01
http://nuitka.net/
======
misaelm
I built a small app that retrieves data from a remote server, builds a summary
and then pushes it to a local html file.

Number of people that tried it when I told them that to make it work they had
to download python, pip, paramiko and install the pycrypto binary? _Zero_.
Number of people that tried it after I just gave them a zip file with an
executable? No longer zero.

I tried different libraries but Nuitka was the only one that made everything
work seamlessly. I owe this guy a beer.

Edit: I no longer owe Kay a beer. I found the donation link on his website.

~~~
fernly
This issue has been addressed multiple times with considerable success by
pyinstaller, py2app, cxfreeze. You can get your single-zip-file distribution
package without nuitka, although it may have other advantages.

------
ntrepid8
_This project looks completely misguided. The talk focused on the trivialities
of mapping Python to C++ rather than on the interesting problems to be
encountered when trying to optimize Python while maintaining its extremely
dynamic semantics. Also the benchmarking effort is laughable; pystone is not
to be taken seriously (only exercises a tiny part of the language) and pybench
does microbenchmarks, which are optimized away. You should try the "real-
world" benchmarks from the PyPy and Unladen Swallow projects. And what is the
size of the generated code? (E.g. how big would the binary for the entire
standard library be?) In your blog, please use less boring subjects than
"version x.y.z released"._ ~ Guido van Rossum

Find Guido's quote in the first comment.

[https://ep2013.europython.eu/conference/talks/nuitka-the-
pyt...](https://ep2013.europython.eu/conference/talks/nuitka-the-python-
compiler)

~~~
sauere
For me, this is not about execution speed AT ALL. I wouldnt even mind if it
was slower.

It is about being able to have a easy, dead-simple way to provide Windows
users with a executable. Without changing my code, without weird build
systems. Like it or not, Windows users are still the majority out there.

~~~
TazeTSchnitzel
Is that not py2exe's job?

~~~
AnkhMorporkian
In theory. In reality, python freezers have a lot of problems. It can be a
real hassle to get a single file, and even if you do they turn out massive.
I've had nothing but problems with them in the past.

~~~
Semiapies
I've had good experiences with PyInstaller, if you haven't tried it yet (or
recently).

~~~
bcj
Seconded. I recently moved a work project from py2exe to pyInstaller and was
very happy with the results.

------
pbiggar
I think this is a really solid approach. It's the same thing I was trying to
do for PHP with phc ([http://phpcompiler.org/](http://phpcompiler.org/)) -
compiling down to C using the built in stdlib and C API, then using full
program optimization.

Long term, it won't be as fast as a full JIT, as the Facebook HPHP team
showed, but Python doesn't have a JIT of the same caliber, so this is probably
useful for tons of people.

Dunno if the author is around, but they might find some of the stuff from my
PhD relevant, especially how the static analysis worked and some of the
challenges of compiling using the C API: [http://paulbiggar.com/research/#phd-
dissertation](http://paulbiggar.com/research/#phd-dissertation)

------
untothebreach
This is probably a better link, for those of us (like me) who had never heard
of Nuitka:
[http://nuitka.net/pages/overview.html](http://nuitka.net/pages/overview.html)

~~~
ntrepid8
Cached:
[http://webcache.googleusercontent.com/search?q=cache:0v-pht1...](http://webcache.googleusercontent.com/search?q=cache:0v-pht19f18J:nuitka.net/pages/overview.html)

------
avinassh
Checked contributors list on Github[0] and most of the code is written by a
single person, Kay. Isn't that amazing? A project of this large and value,
done by single person in his own time. I wish this project gets more
popularity.

Bought a beer to Kay :-)

[0] -
[https://github.com/kayhayen/Nuitka/graphs/contributors](https://github.com/kayhayen/Nuitka/graphs/contributors)

------
tfeldmann
Great project. Compiling a simple PySide application worked fine on the first
try. If you bundle all dependencies you get a binary about the same size as a
binary produced by py2exe. I will do some speed comparisons next.

------
jarcane
_It is somewhat faster than CPython already, but currently it doesn 't make
all the optimizations possible, but a 258% factor on pystone is a good start
(number is from version 0.3.11)._

Holy cow. And here I'd set aside Python in part because of the difficulty in
getting performant executables (I was getting visible lag in a turn-based SDL
game ...)

~~~
syllogism
Use Cython.

It's a hassle getting your head around it, and getting it set up, but you'll
never look back once you do.

If you rely on PyPy, you get what you're given and then you're stuck. You
can't really guess how to rewrite your code to make it faster.

A small blog post I wrote on this:
[https://honnibal.wordpress.com/2014/10/21/writing-c-in-
cytho...](https://honnibal.wordpress.com/2014/10/21/writing-c-in-cython/)

Here's a non-trivial example:
[https://github.com/honnibal/thinc/blob/master/thinc/learner....](https://github.com/honnibal/thinc/blob/master/thinc/learner.pyx)

This code is driving a library of very fast NLP tools that I'm writing.

~~~
shadowmint
Unforntuately cython is even harder to package up than normal python because
it generates one shared library per module.

...not really suitable for games.

~~~
TillE
You may need to fiddle around with how modules are loaded by Python, but you
can do whatever you want with the C files that Cython generates.

kivy-ios should have all the details for building everything into one big
Python blob, if that's what you need.

------
simonpantzare
It seems to be, by and large, the work of a single person on his spare time.
Very impressive!

------
HerrMonnezza
On a tangentially related note: here is a comparison of various Python
runtimes (interpreters and compilers including Nuitka and PyPy) on a fairly
complex scientific code:
[http://arxiv.org/abs/1404.6388](http://arxiv.org/abs/1404.6388)

The shootout, however, is from August 2013 so getting a bit old by now.

------
agentultra
I would love to make distributing games built on Python easier. Jessica
McKeller suggested this should be a priority[0] to make the language more
accessible by getting kids involved in making games and being able to share
them, easily, with their friends. These days Javascript is kicking our butt in
this area.

[0] [https://www.youtube.com/watch?v=d1a4Jbjc-
vU](https://www.youtube.com/watch?v=d1a4Jbjc-vU)

I've been writing little helper libraries on top of Python 3.4 + pysdl2 as
I've been working on games and demos for my (unfortunately cancelled) Pycon
talk. One area I've only played with, unsuccessfully, is getting packaging
going on Nuitka or some other compiler. If anyone wants to get together to
make it awesome get in touch.

------
insertion
I noticed that PyPy does not show any real speed improvement over CPython in
these benchmarks:
[https://www.techempower.com/benchmarks/](https://www.techempower.com/benchmarks/)

If you filter by Python, you can compare some results running on both CPython
and PyPy. I would be curious to know what it is about these benchmarks that
makes PyPy perform poorly. I would also be interested to see how Nuitka
performs.

At the moment I'm also very excited about Pyston from Dropbox:
[https://github.com/dropbox/pyston](https://github.com/dropbox/pyston)

~~~
Beltiras
I'm supremely sceptical about those benchmarks. Go take a look at the code
being tested. I would welcome a serious look at developmental time versus
resource used, using code that is probable in production. I read the code used
to test Django. It's not reasonable code.

~~~
saym
It's an open sourced benchmarking comparison. If you can improve them, shoot
them a pull request!

[https://github.com/TechEmpower/FrameworkBenchmarks](https://github.com/TechEmpower/FrameworkBenchmarks)

------
3am
Why are so people so fast to trust a new compiler?

Have you audited that compiler's source and building the compiler from it with
a known good compiler? Are you inspecting the resulting .pyc, and in this
case, the resulting PEs? It's a super easy way to inject a compromise into a
package that will probably get widely distributed.

full disclosure: link is down for me, so haven't read the article. Been a
comment that has been building up for a while for me, and not specific to
Nuitka. Same goes for new frameworks/languages/etc.

~~~
coldtea
> _Have you audited that compiler 's source and building the compiler from it
> with a known good compiler? Are you inspecting the resulting .pyc, and in
> this case, the resulting PEs? It's a super easy way to inject a compromise
> into a package that will probably get widely distributed._

Because nobody is that paranoid?

~~~
hughes
And if they are, they probably aren't in the business of distributing
precompiled binaries.

------
bhouston
Mirror:
[http://web.archive.org/web/20141218012211/http://nuitka.net/](http://web.archive.org/web/20141218012211/http://nuitka.net/)

BTW how much faster is this than PyPy or CPython? That is the question that I
think is most on people's minds.

~~~
lucashn
Seems to be 2x faster than CPython, but as I understood it is working on
optimizations now.

~~~
mkesper
As others have pointed out, these claims have to be taken very carefully and
the main value (at least for now) seems to be the easy packaging and
deploying.

------
Beltiras
I program Django websites and work in an environment where we like to deploy
often. I'm not sure we could tolerate the extra wait in compile time for
deployment. At least we would need to take a serious look at the
speed/resource gain before considering it.

~~~
robertfw
Deployment in a controlled environment is not the ideal use case here - the
benefits come from when you are shipping to non-technical users.

------
shadowmint
For all that people may think this isnt a good idea, this is a very similar
idea to what Unity3d is persuing with IL2CPP
([http://blogs.unity3d.com/2014/05/20/the-future-of-
scripting-...](http://blogs.unity3d.com/2014/05/20/the-future-of-scripting-in-
unity/)).

This seems immidately relevant to address pythons biggest problems; its slow,
and its hard to distribute. Relevant quote:

    
    
        All code generation is done to C++ rather than architecture 
        specific machine code. The cost of porting and maintenance 
        of architecture specific code generation is now more amortised

------
Beltiras
Git repo is unresponsive. I think we killed the poor guys box.

EDIT: [https://github.com/kayhayen/Nuitka](https://github.com/kayhayen/Nuitka)

------
bane
I'm _very_ interested in some benchmarks on this and various other
compilers/packagers/runtimes for Python.

Sidenote: A very long time ago I had some great luck with Perl's packagers:
PAR, PerlApp, etc. but they all basically just bundle in the basic Perl
runtime with the guts of your program so there's no real benchmark difference.
I suspect the same is not true of efforts like this for Python.

------
dannyroberts1
This is awesome! I tested it on some code that uses some not-compeltely-
trivial python features like metaclasses, etc. and it worked like a charm.

I couldn't get it to work against this obfuscated hello world though:
[http://benkurtovic.com/2014/06/01/obfuscating-hello-
world.ht...](http://benkurtovic.com/2014/06/01/obfuscating-hello-world.html)
:)

~~~
mkesper
That one's relying on CPython implementation details, the author says so in
his detailed description.

------
PythonicAlpha
I really like the idea, that it compiles Python but is (99,9%) compatible to
CPython -- since so you can still use the many C extensions (I don't want to
do my projects any more without my own Python extensions written in C).

------
zongitsrinzler
I don't know much about compilers and runtimes but how come there are SO MANY
of them for Python. Is there something about Python that makes it so easy to
write runtimes for it?

~~~
bane
I think it's because lots of people write lots of code in python, but then end
up with performance problems at some point and try to tackle those. Have a
nice regular language makes it _relative_ easy to rewrite a runtime to scratch
your itch.

I suspect we would have seen this more with languages like Perl, except Perl
is virtually impossible to parse.

------
kedean
My immediate question, which I don't see an answer to on the site or Github
page: is it self-hosting yet?

------
est
Python is a fun language, it has as many web frameworks as its
runtime/interpreter.

