

Numba: NumPy-aware optimizing compiler for Python - pash
https://github.com/numba/numba

======
fijal
So maybe to clarify few things. This is just sensationalism - numba is not a
python compiler. It's a compiler of some restricted subset of python into
LLVM. I don't think the subset is very well defined by now, but I would expect
it to be at some point in the future. It's just that you can choose what to
compile and what not, which is very convinient for a lot of numerics.

The whole approach might be viable and it definitely has use cases, however
the sensational headline makes it so bad :/ Hackernews - you let me down.

~~~
pash
You mean sort of like how PyPy is a JIT for only a restricted subset of
Python?

~~~
fijal
No, PyPy JITs all the Python by design. It also runs all the Python and JITs
all the constructs. It does not make all the possible python magically faster
however, but that does not change my point. A JIT for only a restricted subset
of Python would mean that it segfaults/crashes/gives a wrong answer if you
don't provide a correct subset. This is not true.

~~~
pash
So basically you're saying that PyPy knows what code it can optimize and Numba
requires the programmer to specify it. That probably has more to do with
Numba's being version 0.2 than anything else. And it's hardly reason to moan
that it's not a "real" compiler.

~~~
fijal
I did not call it not a "real" compiler (with or without quotes), I did call
it not a python compiler. It has nothing to do with 0.2 - supporting all of
Python is probably not even within goals of the numba project. I'm not talking
about "all the bytecodes", but making sure all the corner cases work perfectly
well etc. Also supporting all cases would mean that you get rid of a lot of
speed benefits that you see right now, precisely because you don't care about
all the corner cases. Predicting the future is obviously hard and I can't
speak on behalf of numba authors, but I don't think there are plans to support
all of Python.

~~~
pash
Python code goes in, LLVM IR comes out. Sounds like a compiler to me.

As you know, the basic idea of the project is to make writing fast, vectorized
code for numerical and scientific computations as easy as writing native
Python. And to do it in a way that maintains compatibility with the extensive
CPython universe.

I think the criticism you're fishing for is that it is not an _implementation_
of Python. No, it's not that. But, yes, it is a compiler.

~~~
fijal
No, it's not. There is one more requirement - the LLVM IR has to work exactly
the same as intended for python (in this case as cpython does). This is not
the case - it's not full and what's implemented does not support all the
corner cases of the language, starting with overflow-to-long.

------
juanlu001
Here is the post at Travis Oliphant's blog about numba

[http://technicaldiscovery.blogspot.com.es/2012/08/numba-
and-...](http://technicaldiscovery.blogspot.com.es/2012/08/numba-and-
llvmpy.html)

As it is said there, it's still early software and its "road-map is being
defined right now by the people involved in the project". Sure the subset is
not well defined now and there are no docs, but hey - let's give the thing a
few months.

Haven't tested thoroughly yet, but I think no NumPy calls can be made from
inside a numba-compiled function - so this might be the case for other Python
modules as well.

~~~
travisoliphant
Actually some NumPy calls can be made inside the numba-compiled function as
there is nascent support for the NumPy C-API --- it translates the call to a
C-call.

In the future, we want to support all Python, but not necessarily optimized
--- just using the Python Object C-API.

------
kkuduk
here is a talk form SciPy 2012 about Numba
<http://www.youtube.com/watch?v=WYi1cymszqY>

------
montecarl
Here is a mailing list thread with the results of a simple benchmark:
[https://groups.google.com/a/continuum.io/forum/m/#!msg/numba...](https://groups.google.com/a/continuum.io/forum/m/#!msg/numba-
users/AGNO2maDVkk/E2A4mFy2bJsJ)

Looks promising!

------
buster
How good does this work with other python modules? Can i use this to compile
_every_ python program to native executables?

~~~
andreasvc
No.

In any case, doing that wouldn't be difficult, you just wrap a Python
interpreter and the Python code in a single executable and you're done. What
would make it worthwhile is when it would be an _optmizing_ compiler, which is
of course difficult for such a dynamic language.

------
bravura
The github README (<https://github.com/numba/numba>) explains how to install
Numba, but not how to use it.

Or is it used _automatically_ for all Python byte code? If so, can it be
disabled?

~~~
pash
The main way to use it right now is by decorating the functions you want it to
compile. See the examples directory [0] in the Github repository.

0\. <https://github.com/numba/numba/tree/master/examples>

------
textminer
Big user of Scipy Sparse. Anyone know if this is compatible?

~~~
alexenko
... It uses the remarkable LLVM compiler infrastructure to compile Python
byte-code to machine code especially for use in the NumPy run-time and SciPy
modules... from the README on github

------
fosap
And the PyPy authors told us that Python would be to dynamic for ahead of time
compilation. This is awesome.

~~~
fijal
this is not a python compiler. it's a subset of python compiler (this exercise
has been done already quite a few times)

