
How Numba and Cython speed up Python code - BerislavLopac
https://rushter.com/blog/numba-cython-python-optimization/
======
metalliqaz
I like Cython, but I'm always surprised at how willing programmers seem to be
to use the cython syntax. In so many cases I look at the optimized program and
think to myself, "I would understand that better if was just written in C."

~~~
pebers
I agree, my vastly preferred option now is Python code and C code with cffi to
glue them together. I find that much preferable to Cython which is basically a
third language altogether (and IMO not an especially nice one, although that
is a bit of a value judgement).

~~~
dagw
The huge advantage with cython compared to C is that it is trivial to call
python libraries and get the same behaviour as you would in python.

------
peheje
Nim is a language that mimicks python syntax but compiles down to c. Really
cool and easily gives you 'fast code'. If you are into this kind of stuff I
would recommend you take a look.

[https://nim-lang.org/](https://nim-lang.org/)

~~~
coleifer
How's the standard library in nim? What's the 3rd party package ecosystem
like? If I have questions will I be able to find a good answer on Google? How
about jobs?

There's a handful of languages out there you could consider "faster python". I
just use cython though.

~~~
peheje
Python is awesome don't get me wrong.

I have limited knowledge about it, but mainly it isn't version 1.0 yet so the
std lib stills changes and has some quirks/missing pieces or pieces that are
not quite aligned yet.

The package system is Nimble, seems easy enough, fetches packages from github
given a version, though I'm not sure how secure that is.

Not sure about Google as its relatively new, but in general the Nim forum is
pretty active. I have asked around there a few times and got answers even from
the language creators themselves.

------
jebej
Just use julia instead!

~~~
f311a
Julia lacks ecosystem outside of academic stuff.

~~~
cultus
It's quite complete for anything numerical/machine learning/statistical at
this point, with almost 2000 packages. This is in addition to the very
comprehensive standard library, which includes everything you get in
Numpy/Scipy. Of course, outside of numerics, you're pretty much out of luck,
which is not the case for Python.

It also has multiple dispatch like Common Lisp, as well as gradual typing. All
functions are multimethods, actually. This is an enormous abstraction
advantage over Python. It's much easier to do complicated things without
pulling your hair out in comparison to Python.

I use it a fair amount in my job as a data scientist. It's also what I reach
for if I need to write some custom algorithm myself that needs to be high
performance, rather than doing it in C.

~~~
f311a
Does it have a tooling for deployment? Is there a good way to deploy a machine
learning model in production?

------
tasty_freeze
I haven't used it, but nuitka ([http://nuitka.net/](http://nuitka.net/))
translates python (2.6, 2.7, 3.3 to 3.7) to C and compiles that. It claims to
be highly compliant and performant without any extra pragmas.

How does is compare to the two technologies in the article?

------
abakus
They usually cannot improve carefully optimized numpy operations, nor GPU
operations. The use cases are thus limited.

~~~
mehrdadn
IIRC, this is false regarding NumPy. The fundamental problem NumPy has is that
it only vectorizes 1 operation for each pass through the data (or occasionally
2-3, like dot product). It can't do arbitrarily hybrid operations (like idk,
maybe a[i] * b[i] / (|a[i]| + |b[i]|)) in one pass through your data. This is
an inherent limitation in NumPy, and the lack of it is inherent in Numba. So
you very much _can_ expect speedups in some cases -- it just depends on what
operations you are performing and how big your data is.

~~~
jzwinck
You're right, apart from np.einsum() and numexpr which is a separate package
(albeit less drastic to use than Numba, because with numexpr you don't write
your own loops).

~~~
mehrdadn
I meant "occasionally 2-3, like dot product" to include stuff like einsum.
FWIW I found einsum was actually slower than tensordot last time I tried, so
you may want to use that instead if this is still the case.

------
mlevental
is cython compatible with all python code or is it only a subset of libraries
that you can use?

~~~
deathanatos
It depends. This is the only deviation I've ever found:
[https://github.com/cython/cython/issues/1936](https://github.com/cython/cython/issues/1936)
; that is, Cython will execute __prepare__ in Python 2; but __prepare__
doesn't exist in Python 2, so the normal interpreter (CPython) won't execute
it. This can lead to deviations; in my case, the code crashes if run under
Cython, and executes fine under CPython.

The Cython maintainers disagree with me that this is a bug, so, if you're
under Python 2, I would say it is "very nearly" compatible. If you're in a
recent version 3, AFAICT, it just makes Python code faster.

