
Python performance is a fool's errand - ntoshev
http://whoosh.ca/blog/fools_errand?repost
======
timwiseman
I find this an interesting post, but I must partially disagree.

Performance is not always the the primary issue in programming. Using python I
am much more worried about code readability, maintainability, and programmer
speed than I am code speed. And Python certainly is designed in a way that
makes true high performance somewhat challenge. If I were ever truly concerned
about maximizing performance, I would use C++ or similar.

On the other hand, performance is always a consideration. It should never be
totally forgotten about. For instance, if you will be using a single regex
pattern multiple times in a program (or putting it in a loop) the performance
increase from compiling it once is noticeable and does not subtract from any
of the other attributes. Similarly, in many cases a list comprehension will be
faster than an equivalent for loop.

Python performance is not a fool's errand, but I do think that people
programming in python should realize they are making something of a trade off
(unladen swallow and future research may partially change this in the future.)

(Edited to fix spelling typo.)

~~~
krakensden
_Performance is not always the the primary issue in programming_

Except when it is, python rapidly becomes depressing. Try writing a game in
python/pygame, for instance, and watch how quickly things get out of hand.

~~~
l0nwlf
Well, I code python for ease of use and readability. Every language has its
pros and cons. To be frank speed was never a forte of python but it was never
meant to be as such. If you need speed there are options available, use them.
However C have a faster run-time and slow development time. There are issues
in all languages. However there is always a favourite language. Mine is python
:)

------
icey
What's changed since you submitted this a few weeks ago?

<http://news.ycombinator.com/item?id=684017>

~~~
ntoshev
I felt it received undeservedly little attention due to the time of the day it
was submitted. Sorry for the redundancy.

------
zokier
Python is designed to be easily extended with C. If you need performance then
write those performance-critical parts in C and write lightweight wrapper to
python.

~~~
ivan_i
Or write speed-critical Python iterator directly in C. Python source is the
best reference:
[http://svn.python.org/view/python/trunk/Objects/dictobject.c...](http://svn.python.org/view/python/trunk/Objects/dictobject.c?view=markup)
(look at the end).

Python/C API rules.

------
algorias
Optimizing like this in python is clearly the wrong approach. Python will
never be fast in the traditional sense.

The only optimization you should be worrying about is avoiding quadratic
behaviour and such, which usually accounts for 99% of all inefficiencies.

If you really need speed, move the code to C and call it from python.

------
ambulatorybird
I haven't kept up with Python developments, but the fact that almost
everything is a hash table (even stack frames for interpreted function calls,
I believe) would seem to place a relatively low upper-bound on performance.
And this is in addition to the usual overhead of a straightforward
interpreter.

~~~
viraptor
I'm not sure if that's a low upper-bound. He's complaining about an attribute
access speed - but he's not trying to fix it... missed a great opportunity for
__dict__ usage.

From a quick py2.5 benchmark:

    
    
        noop 0.3s
        class_create 0.76s
        with_dict 1.68s
        without_dict 2.06s
    

Functions tested 1,000,000 times - noop is an empty function for comparison;
class_create is just x=A(); with_dict and without_dict create a class each
time, assign 3 attributes, then read the same 3 attributes. with_... does it
on an object that lists the attributes in __dict__, the other one doesn't use
__dict__ and assigns attributes in the constructor.

So... 1s (without the constructor and loop time) for 3,000,000 attribute reads
and 3,000,000 attribute writes. Slow? Is he sure that it's the attribute read
and write that's causing the problems? I'm not sure if I believe that without
some more details - especially if he works with some documents that look like
they're on some external storage (disk / db / whatever).

------
billjings
First, I second the notion that switching to another tool is a good idea when
performance becomes important. I was doing this by diving down into C years
ago, and I have to imagine the toolset has only gotten better since then.

Second, you can make your generator method a coroutine and send it data as it
iterates.

