
Why Python Is Slow: Looking Under the Hood (2014) - bshanks
https://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/
======
WestCoastJustin
Not because I want to limit discussions here, but this has been posted before
on HN and there are many great comments in these threads. So, this comment is
more of a useful suggestion, that if you are interested, check these comments
out too (both 2+ years old).

[https://news.ycombinator.com/item?id=8625280](https://news.ycombinator.com/item?id=8625280)

[https://news.ycombinator.com/item?id=7721096](https://news.ycombinator.com/item?id=7721096)

~~~
blaze33
Also, 4 pays ago there was a "why is python slow" post with many interesting
comments:
[https://news.ycombinator.com/item?id=12025309](https://news.ycombinator.com/item?id=12025309)

~~~
0xmohit
And
[https://news.ycombinator.com/item?id=12025529](https://news.ycombinator.com/item?id=12025529)
therein specified these:

The following may also be of interest:

\- Why Python is Slow: Looking Under the Hood [0]

\- Fast Python, Slow Python by Alex Gaynor [1]

[0] [https://jakevdp.github.io/blog/2014/05/09/why-python-is-
slow...](https://jakevdp.github.io/blog/2014/05/09/why-python-is-slow..).

[1]
[https://www.youtube.com/watch?v=7eeEf_rAJds](https://www.youtube.com/watch?v=7eeEf_rAJds)

------
Symmetry
Generally good but it makes it sound like what it calls the dynamic typing
penalty is a matter of more instructions. If that was true Python would only
be a factor of 2 slower or something like that. Instead the problem is that
the cost of getting the value of an integer is first of all that after reading
the type of the variable Python has to execute a fairly high entropy branch
which will probably cost the branch delay penalty of 20 cycles or so. And then
the extra memory lookup will cause a slowdown as well.

------
jstimpfle
Is there a reason why Python must be slower than Javascript? I.e. given
economic and cultural incentives, couldn't there be something like V8, for
python?

~~~
weberc2
There are faster Python implementations, namely Pypy. I think one of the
reasons Pypy isn't more popular is because it's C interoperability story is
weaker than the reference Python implementation (CPython--not to be confused
with Cython), or at least it's not compatible with many important C-based
Python libraries (e.g., Pandas).

~~~
Animats
_" I think one of the reasons Pypy isn't more popular is because its C
interoperability story is weaker than the reference Python implementation"_

The CPython crowd apparently made some change between CPython 3.4 and CPython
3.5 which changed how Windows DLLs are called. This broke Py2exe, which no
longer works for the current release of CPython. [1]

CPython's approach to C extensions usually requires having the same Microsoft
C compiler version used to build Python. This is now a problem for Python 2.7,
because the Microsoft compiler version used to build it is no longer
available. Microsoft recently changed their API for the C runtime, and CPython
has been changed accordingly.

CPython's "C interoperability story" isn't really that good on Windows.

[1] [http://stevedower.id.au/blog/building-for-
python-3-5/](http://stevedower.id.au/blog/building-for-python-3-5/)

~~~
ihnorton
> This is now a problem for Python 2.7, because the Microsoft compiler version
> used to build it is no longer available.

There is a CMake build system for 2.7 that works great with VS2013-2015:

[https://github.com/python-cmake-buildsystem/python-cmake-
bui...](https://github.com/python-cmake-buildsystem/python-cmake-buildsystem)

------
foota
There was someone a while ago in a group I was in making the claim that python
was no inherently slower than other languages that were more static like Java.
I might have to point them at this now. Great read! Reminds me of:
[http://blog.kevmod.com/2016/07/why-is-python-
slow/](http://blog.kevmod.com/2016/07/why-is-python-slow/) which was discussed
a few days ago at
[https://news.ycombinator.com/item?id=12025309](https://news.ycombinator.com/item?id=12025309)

~~~
qznc
I remember another article, but cannot find it. It had very different
arguments what is slow about Python and would have disagreed with those two
articles.

Python is slow because it is dynamically typed rather than statically typed?
No. In most cases, type inference is good enough. See Javascript. Python is
slow because it is interpreted rather than compiled? That is not really a
limit in the long run. Insert a JIT compiler. Java was interpreted as well
initially.

Python's object model can lead to inefficient memory access? No we are getting
somewhere. Scripting languages in general tend to overuse hashmaps. That gives
you a lot of flexibility and monkey patching. It is also a lot harder to
optimize than dynamic dispatch.

Another problem of scripting languages is the inability to control your
memory. Python forces you to make a lot of memory allocations and there is
little control over the garbage collector. Even Java suffers from this, but
the JVM engineers invested a lot to fix it. There are no value record types
(struct) in Java.

~~~
telotortium
Why Python, Ruby, and Javascript Are Slow:
[https://news.ycombinator.com/item?id=5304873](https://news.ycombinator.com/item?id=5304873)

~~~
qznc
That might be it. I remember it as an article. Maybe
[http://lukauskas.co.uk/articles/2014/02/13/why-your-
python-r...](http://lukauskas.co.uk/articles/2014/02/13/why-your-python-runs-
slow-part-1-data-structures/)

------
BuckRogers
Python is slow because the core dev team has long concerned with Python3.
Priority is not on speed. CPython actually got slower with 3, and it still is.
They say it's "a wash" but I haven't seen that in anything I've done. Since
Microsoft got its claws into Python through the core dev team, we've seen a
shift from the Linux (POSIX) text model in Python2 to the Windows text model
in Python3.

It now matches how the .Net CLR handles unicode. Which is a shame since Linux
is kind of increasingly popular.

There are some recent developments where Microsoft is investing through the
core dev surrogates into Python3 and speed with Pyjion. That is promising but
it's a framework where it requires CPython3 to use a JIT, which strikes me as
a way to enforce control- a response to PyPy and alternatives taking the
mantle when they abdicated the throne with Python3. But it remains to be seen
how it works out, and PyPy already is likely to remain the leader on faster-
Python.

------
cm2187
What's so great with dynamic languages that justifies the performance hit?
Most statically typed languages have type inferrence, which reduce the syntax
to something very close to a dynamic language. And is it such a good idea to
only catch mistyping at runtime, and require to write twice as much code with
testing to compensate for the lack of typing?

~~~
jerf
"Most statically typed languages have type inferrence, which reduce the syntax
to something very close to a dynamic language."

To understand the appeal of dynamic languages, it's critical to consider the
era they came from. Your statement is _not_ true of the general programming
landscape ten years ago. It's not really true of the general programming
landscape today, for that matter, which is still by sheer mass C++ and Java.
But 10 years ago, there weren't hardly any choices of statically-typed
languages that weren't a bit of a bear to use. 1990s C++ is a monstrous
nightmare. A lot of this monstrous nightmare was arguably the programming
environment (Windows, Xwindows, UNIX, everything was a bear to program in back
then), but when you took battle-scarred C++ Windows programmers and showed
them

    
    
        print "Hello world!"
    

they weren't in a mood to analyze exactly which differences between
Python/Perl and C++ were essential and which were accidental; they were too
busy drinking in the joy.

I think there's a general trend towards languages that either make the static
typing easy enough to use that you don't feel like you're losing out on much,
or make the static typing pay for itself better (Rust), and I think that will
continue to be the story for the next ten years on the developing language
front. I think it may be the case that if they weren't already entrenched,
what I call the 1990s-style dynamic languages (because it's not just that
they're "dynamic", Perl, Python, Javascript, and Ruby are almost the _same
language_ , compare with the "dynamic" Erlang to see a major difference)
wouldn't succeed if someone wrote one for the first time this year. In fact
they'd probably take a beating for the performance issues it would have. But
they are entrenched and have over a decade of refined libraries, support
documentation, tutorials, and of course, the big hitter, existing code. So
they aren't going anywhere any time soon.

~~~
cm2187
From that point of view I agree.

The other thing is tooling. There are lots of hardcore developers to whom even
syntax highlighting is too much distraction, let alone error checking, auto
completion, etc.

Well... To say the least I am not one of them. In fact to me, a large part of
the simplicity of a language resides in the interaction with the IDE, the
feedback it gives you on what's legal and what is not, where you may have a
bug or an ambiguity, what can I do from there, etc. Dynamic languages defeat
advanced and proactive tooling. And to me it makes them actually quite hard to
work with, not self-discoverable, and all-in slower to develop. The time not
spent writing a class or a return type ends up being spent on the reference
page of the library (after you checked this is the correct version) checking
what are the arguments of a function, etc.

------
amyjess
I have to wonder how much this is CPython specific and how much applies to all
implementations of Python (including PyPy, which is noticeably faster than
CPython).

~~~
sevensor
I've found that PyPy is _very_ fast in most cases. While I've seen hand-
optimized C code beat PyPy by a mile, I've seen more cases where the gains
were in the 10-20% range. And PyPy keeps getting faster.

------
bshanks
Thank you 0xmohit for this link (via
[https://news.ycombinator.com/item?id=12025529](https://news.ycombinator.com/item?id=12025529)
)

------
module0000
Thanks for taking the time to publish this, I've always been interested in
tearing at CPython, but this pushed me over the edge. Looking forward to new
and exciting ways to crash the runtime :)

------
amenod
It is easy to miss the point with such articles. OP touched upon it in chapter
"So Why Use Python?". Raw speed, while sexy, is not a deciding factor in most
cases - not everybody is working on HFT. And when it _is_ important to be
really fast, you can always write an extension in C (or Rust I imagine?) just
for that single tight loop and get raw speed without sacrificing
maintainability. Seriously, most bottlenecks (in other people's software of
course ;) I came across were either due to bad architecture or unreadable
code, which lead to not noticing that too much work was being done. So yeah,
Python might be slower than alternatives (though I have a hard time believing
that it is slower than Java), but it is fast enough.

~~~
curried_haskell
The only reason people might think of Java as slow, is because of desktop
applications like LibreOffice, IntelliJ, or Eclipse. Java on the server blows
Python out of the water in terms of speed.

~~~
Narishma
AFAIK LibreOffice is written in C++, not Java.

