

Bitey: Import LLVM bitcode directly into Python - mace
https://github.com/dabeaz/bitey

======
zokier
I very much like the concept. But I dislike the "magic" feel of overloading
"import". Especially as it creates namespace collisions, eg when you have
python module and object file with same name. Imho something like
"bitey.import_obj(name)" would have been nice and more clear.

I wonder how complicated it would be to parse header files to populate field
names of structs automatically? Maintaining separate .pre.py and .h files
seems like recipe for trouble.

~~~
wzdd
Neither do I, but to be fair to the author this particular magic is encouraged
in Python (see <http://www.python.org/dev/peps/pep-0302>).

To parse headers for field names would require a complete C preprocessing and
parser. That wouldn't be a problem for this author (who wrote a very popular
parser generator for Python), but it still wouldn't be perfect until it
completely replicated the system compiler's behaviour with respect to system
headers (consider conditional compilation). It is particularly annoying if the
host and target systems are different, i.e. in cross compilation. I've tried
this exact thing (header parsing to get type information) and it is quite a
pain to get it right.

~~~
zokier
Maybe you could use LLVM/Clang to parse the headers. That way at least the
parsing would be consistent with the compilation. I don't know how difficult
it would be to access such type information, but I thought the whole point of
LLVM/Clang was to be usable as a library.

------
synparb
I'll be interested to see how this compares to numba and how easy it is to fit
it into the scientific python ecosystem. I'm also curious about performance
versus numba and my current go to solution, cython.

------
bbayer
What about performance? Is there any benchmark for a function compared with
pure python and C implementation?

~~~
osener
You can find one in the mandelbrot example.

<https://github.com/dabeaz/bitey/tree/master/examples/mandel>

~~~
zokier
It would be nice if it was compared to pure c implementation too, to measure
the overhead.

------
jrockway
"I call the big one Bitey."

~~~
king_magic
First thing that came to my mind :-)

------
madlag
Very cool ! Can't wait to test it ! Why exactly no C++ support ?

~~~
koenigdavidmj
It would be hard in the presence of parametric polymorphism (multiple
functions having the same name but different numbers and types of parameters),
as well as the difference between C++'s and Python's method resolution
semantics (in Python, for instance, everything is virtual).

Also, templates.

Also, a bunch of other things.

~~~
mamcx
And obj-c? Also hard?

~~~
kennywinker
You'd have to shoehorn the obj-c runtime in there somehow. Almost definitely
not worth the effort. What would you want to run in this way?

------
zoobert
It seems cool but why would you use it instead of having a c lib (or c obj
file) interfaced with ctypes or swig ... ? Maybe I miss what LLVM brings ?
Thanks

~~~
wzdd
I addition to cdavid's comment: LLVM IR is system independent. So if you had a
deployment over heterogeneous machines you could write the extension code once
and have it run everywhere without recompilation.

~~~
horsetail
LLVM IR is not system independent.

[http://llvm.org/docs/FAQ.html#can-i-compile-c-or-c-code-
to-p...](http://llvm.org/docs/FAQ.html#can-i-compile-c-or-c-code-to-platform-
independent-llvm-bitcode)

~~~
wzdd
Fair enough, that was too strong a claim. But it is more portable than an
extension module compiled with the system compiler.

~~~
tungstentim
It's not portable enough to let me take IR generated for my x86 laptop and run
it on my arm board, in general, even though both platforms are ILP32, little-
endian, and running the same OS.

~~~
wzdd
Yes, but this is a problem with the source language and/or the host system
libraries, not with LLVM IR itself. There is a broad domain of applications
for which it is portable.

------
tocomment
So can you use this with C functions compiled normally with gcc? It seems like
that would be even more useful.

~~~
winter_blue
What difference would it make? To my understanding, Clang is fully compatible
with gcc.

~~~
tocomment
I guess I mean can you use this with C instead of LLVM?

~~~
darkarmani
Your terms are off. C is compiled using LLVM (clang) into an intermediate form
that is compatible across platforms (LLVM IR).

(AFAIK, please correct me if I'm wrong)

------
tvdw
I could be completely wrong here, but doesn't this completely bypass the
optimizer that makes C so fast?

~~~
kibwen
LLVM performs its optimization passes on the LLVM bitcode itself (the "middle-
end" of the compiler), before finally translating the optimized bitcode into
machine-specific binary code.

Not an LLVM expert though, I could be glossing over a few details.

~~~
benbjohnson
You're correct. LLVM includes a pass manager that performs extensive
optimizations.

