
Pythran: A Python subset to C++ compiler that takes advantage of SIMD - maxmouchet
https://github.com/serge-sans-paille/pythran
======
gamesbrainiac
There are quite a few attempts to make faster python implementation these days
from pypy (A JIT built on top of rpython), pyston (yet another JIT that which
uses llvm as a backend), pyparallel, nukita and I'm pretty sure there are a
lot more.

Either the python community is being very prolific, or has no clue what will
actually solve the problem, hence its trying everything.

~~~
_yosefk
Well, these things are fun to hack on and then each might solve a part of the
generally near-unsolvable problem, and for each such part there are people
caring about _that_ part the most so the solution has its uses. (Incidentally,
my own preference to date has been to only use CPython and fall back to C
extension modules when speed starts to matter... But I could have a different
preference in a different environment working on different things.)

Kind of like making C or C++ safe is a near-unsolvable problem for most values
of "safe" and there's an ever-growing set of static and dynamic analysis tools
detecting things like uninitialized memory access, out of bounds access, data
races, integer overflow, etc. etc. etc. And each of these tools gives you a
different mix of false positives, false negatives, build time overhead
(including getting the program to build with the tool at all) and runtime
overhead (including getting the program to run), not to mention availability
on a given platform and pricing. And different people have a different
preference with respect to what this mix should be for their tool of choice.
But the problem is unsolvable which is why it's _worth_ to be "trying
everything."

~~~
jboy
As a longtime programmer of Python, C & C++ (and a longtime appreciator of
your FQA Lite, by the way), my preference until recently was also to use
CPython & fall back on extension modules for speed.

But I grew weary of writing all the Python C-API boilerplate -- especially
checking parameter types and managing reference counts properly.

So I created (and recently open-sourced) Pymod, a Nim+Python project that
auto-generates all the Python C-API boilerplate & auto-compiles a Python C
extension module, to wrap the functions in a Nim module:
[https://github.com/jboy/nim-pymod](https://github.com/jboy/nim-pymod)

As a pragmatic Python programmer, Pymod may be of interest to you too...

(As someone who learned C++ in 1999 and spent most of 200x working in C++, I
also grew weary of staying on top of C++0x's new features & new gotchas. So
about a year ago, I switched from C++ to Nim-lang and haven't looked back. To
me, Nim combines all the best features of Python & C++, while avoiding most of
the worst: C++ speed, C++ static types, C++ generics, C++ operator
overloading; combined with a clean Pythonic syntax & Lispy macros.)

~~~
maxerickson
Cython (and before it Pyrex) occupies a similar space between C and Python.

Was any of your design inspired by them? I also wonder how practical it would
be to reuse parts of it when making a bridge for another language.

~~~
jboy
Actually, Pymod was designed to be almost an anti-Cython. :)

My issue with Cython is that it's a limited sub-language within Python, where
you add Cython elements incrementally & iteratively (diverging from Python in
the process) until the code runs "fast enough". I'd rather work directly in a
full-featured, internally-self-consistent language from the start. Nim has a
clean Pythonic syntax, with all the best parts of C++ (including its runtime
speed).

Hence, Pymod takes the form of an `exportpy` annotation (a user-defined Nim
pragma) onto existing Nim functions, which are then auto-compiled into a
Python extension module.

So there's no gradual divergence of my Python code (as it becomes more
"Cythonized"); rather, the high-performance code is written directly in pure
Nim. :)

~~~
maxerickson
Right, but Cython also supports the external library use case:

[http://docs.cython.org/src/tutorial/clibraries.html](http://docs.cython.org/src/tutorial/clibraries.html)

~~~
jboy
Interesting! Thanks for the link.

After browsing that page, I observe that even when Cython is wrapping an
external C library, there's still a notable difference between the way Cython
does things & the way Pymod does things.

The explanation on that Cython page begins: "To get started, the first step is
to redefine the C API in a .pxd file, say, cqueue.pxd". So to wrap an external
C library using Cython, you still need to define an entirely new header-like
definition file in Cython's intermediate language.

In contrast, Pymod only requires an `exportpy` pragma annotation at the end of
an existing Nim procedure function header -- you don't need to create any
intermediate definition files. And the `exportpy` pragma is _inert_ unless you
invoke the "pmgen.py" script (a Python script in Pymod that determines the
Python C-API system configuration & automates all the compilation), so your
Nim code is still completely valid Nim code after you apply the `exportpy`
annotation.

~~~
maxerickson
I'm now largely out of my depth, but much of that will just come down to what
is available from Nim and what is available from C? And perhaps some of it
from the complexity of supporting arbitrary C code in a flexible way?

The Cython wrapper code is still pretty trivial compared to the C code (
[https://github.com/fragglet/c-algorithms/blob/master/src/que...](https://github.com/fragglet/c-algorithms/blob/master/src/queue.c)
), and this is a pretty small piece of C.

~~~
jboy
Oh absolutely, Nim should definitely get the vast majority of the credit.

It's not just because Nim compiles to C; it's also due to Nim's powerful macro
system, which: (1) allows me to define my own first-class pragmas such as
`exportpy`; (2) supplies my pragma with a detailed & expressive AST of the Nim
function onto which my pragma was annotated; (3) enables my pragma to auto-
generate the C-API boilerplate code, and write it to disk as a newly-created C
source file, all within the Nim compilation pass. Nim is a fantastic language,
perfectly suited to this scenario.

I merely spotted an opportunity. :)

------
nickcw
I just wondered if anyone had any insight in how this differs from shedskin -
[https://code.google.com/p/shedskin/](https://code.google.com/p/shedskin/)

From the shedskin website: Shed Skin is an experimental compiler, that can
translate pure, but implicitly statically typed Python (2.4-2.6) programs into
optimized C++. It can generate stand-alone programs or extension modules that
can be imported and used in larger Python programs.

~~~
drdaeman
Don't know about the differences, but I think shedskin had died some years
ago.

~~~
stuaxo
Nope, they are still responsive on the bug list, I think it lives - just very
slowly.

------
toolslive
But does it retain the python semantics? for example:

    
    
      def fac(n):
          if n == 0 :
              return 1
          else:
              return n * fac(n-1)
    
      print fac(100)
    

will yield a big string, while any naive C++ translation will print the string
"0" due to overflow.

~~~
e12e
Good question (and I don't know the answer, as I'm not familiar with this
project). Note that a naive c++ translation will also error out on simply i=2
* * 524. Automatic extension/promotion of numbers is pretty fundamental to
python.

