

A guide to analyzing Python performance - rbanffy
http://www.huyng.com/posts/python-performance-analysis/

======
shoyer
This guide is a nice start, but it misses the easiest way to analyze Python
performance -- using the %time, %timeit and %prun magic commands in IPython,
the better python command line interface (and which also has a killer web
notebook interface).

The line and memory profiler modules he mentions can be installed as IPython
extensions, too, and you don't even have to modify your code with decorators
to make them work.

------
IgorPartola
Good. Having spent months (a while back) analyzing where memory leaks were
occurring in a fairly big code base I can say that it is no fun without good
tools. In that case the issue was that the underlying homegrown library was
based heavily on SQLAlchemy. Our SQLA objects referenced each other and when
we freed the session, they were not decoupled automatically.

Also beware crappy C extensions. After much debugging I found a quad tree
implementation that leaked like a sieve. From what I understand these tools
will not help with that, though it is rare.

~~~
gizmo686
>Also beware crappy C extensions. After much debugging I found a quad tree
implementation that leaked like a sieve. From what I understand these tools
will not help with that, though it is rare.

As someone who has had to find and resolve memory issues with C extensions, I
would suggest using Valgrind [0]. It is not a python tool, but (after
compiling Python with to play nice with memory debuggers), it works well for
debugging memory in C-extension. It basicly runs your program in a VM, where
it monitors memory access and reports leaks and other potential errors (such
using unitialized, unallocated or deallocated memory) along with a stack
trace.

[0] [http://valgrind.org/](http://valgrind.org/)

~~~
IgorPartola
Yup. Valgrind is the bees knees when it comes to debugging any C program. I do
not remember if I used it or not for this particular purpose, but it
definitely would have helped.

------
gere
There is also the nice CProfile [0] that comes with the Python Standard
Library. It does more or less what the mentioned line profile does, with the
bonus that you don't have to install another package.

[http://docs.python.org/3.3/library/profile.html](http://docs.python.org/3.3/library/profile.html)

------
shuzchen
There is also Guppy/Heapy, which I like in particular because you can spawn a
monitor that you can connect to externally. Sadly the docs aren't very
complete.

