
A few things to remember while coding in Python - satyajitranjeev
http://satyajit.ranjeev.in/2012/05/17/python-a-few-things-to-remember.html
======
halayli
It's worth explaining why mutable defaults are bad.

The problem with mutable defaults is that they are evaluated once only when
the function is defined. Each time the function is called you'll be using the
same mutable variable that was created during function definition.

~~~
tzs
> It's worth explaining why mutable defaults are bad

They can also be good. Here's an example from the Reddit discussion, showing
how a mutable default can be used to very neatly and cleanly add memorization
to a function:

    
    
       def fib(n, m={}):
          if n not in m:
             m[n] = 1 if n < 2 else fib(n-1) + fib(n-2)
          return m[n]

~~~
ZenPsycho
Is that clean and neat, or weird and inscrutable? Will it be clear to /anyone/
reading that code what it's doing?

~~~
tzs
I'm a Python newbie, and that code was immediately clear and obvious to me
when I read it.

I won't say it would be clear _anyone_ that reads it, because we live in a
world where people who claim to be programmers can't do fizz buzz.

~~~
StavrosK
I'm a Python veteran, and, if you do it like this, I'll shoot you.

Add a @memoize decorator and do it there, you need to always be as obvious as
possible. Compare:

    
    
        @memoize
        def fibonacci(n): pass
        
        def fibonacci(n, memory=[]): pass
    

You don't even need documentation for the first example.

~~~
paulmcpazzi
Starting from Python 3.2 there is a builtin decorator:
[http://docs.python.org/dev/library/functools.html#functools....](http://docs.python.org/dev/library/functools.html#functools.lru_cache)

~~~
StavrosK
That's fantastic, thanks for the link.

------
stcredzero
_Generally most use this:_

    
    
        freqs = {}
        for c in "abracadabra":
            try:
                freqs[c] += 1
            except:
                freqs[c] = 1
    

If this is really the common idiom, I'd say this is a sign that professional
programming has yet to fully mature as a field.

 _Some may say a better solution would be:_

    
    
        freqs = {}
        for c in "abracadabra":
            freqs[c] = freqs.get(c, 0) + 1
    

Okay, so I understood immediately what was going on with the 2nd bit of code.

 _Rather go for the collection type defaultdict_

    
    
        from collections import defaultdict
        freqs = defaultdict(int)
        for c in "abracadabra":
            freqs[c] += 1
    
    

As a non-pythonista, the 3rd bit of code, I had to Google "defaultdict" to
figure out. It's only a couple of seconds to Google, and a professional should
know this tidbit, but it seems like premature optimization to me. This brings
to mind this post:

<http://news.ycombinator.com/item?id=3995185>

As a programmer, one's most valuable resource is brainpower. Supposedly, a
programmer's most important goal is writing clear code. Look around at what
goes on in our industry. There's a lot of our most valuable resource spent on
showing off our cleverness, not directed towards the clearest code. To me this
is like spending money to show one can spend money or playing an instrument to
show off dexterity instead of producing gorgeous sounds.

(I think this starts in school and other environments where one is motivated
to show off one's coding chops.)

Most of the complexity in our field accrues like litter: a bit here and a bit
there. I think it says something about the culture of the folks who live
there.

~~~
feral
I agree with the spirit of your argument.

In fact, I almost came here to write a parallel comment: I'm really not sure
that reversing the list 'a' with 'a[::-1]' is better than 'reversed(a)', which
usually effectively does the same thing, but whose meaning is much more
obvious.

But, while I agree with your general point, in the specific case of
'defaultdict', I differ.

I use 'defaultdict' all the time and I'm glad its there. It feels cleaner than
'freqs.get(c,0)'. I define the default value in one place, and then the
interface to my datastructure is simpler; hence as I continue writing, I can
spend more of my brainpower in the problem domain.

Its a small detail, but its one less thing to think about when writing a
complex algorithm.

~~~
jsprinkles
reversed(a) and a[::-1] are _not_ equivalent. The former produces an iterator
over the given list (with all the mutability dangers that come with it), while
the latter produces a copied list. For plain iteration, you're correct,
reversed() is better (similar to how xrange vs. range was back in the day);
however, for reversing something and keeping it around, the slice syntax is
better.

    
    
        >>> reversed([1, 2, 3])
        <listreverseiterator object at 0x107265650>
        >>> [1, 2, 3][::-1]
        [3, 2, 1]
    

To clarify your point, list(reversed(a)) and a[::-1] are equivalent. It's a
slightly subtle point, but _extremely_ important if you're keeping the result
of reversed() around for any length of time. If you're just iterating at the
moment that you use it, yes, they're effectively equivalent.

~~~
njharman
list(reversed(a))

------
raymondh
Overall, this is a nice post. There are two quibbles though.

1). For the most part, "c = collections.Counter()" is almost always better
than "c = defaultdict(int)"

* Counter only supplies missing values rather than automatically inserting them upon lookup.

* The Counter version is much clearer about what it is trying to do. The defaultdict version is cryptic to the uninitiated (understanding it entails knowing that it has a __missing__ method to insert values computed by a factory function and that int() with no arguments returns zero).

* The Counter version provides helpful methods such as "most_common(n)".

2). An ellipsis in Python is normally used in a much different way than shown
in the article (it's used for an extended slice notation in NumPy).

~~~
Camillo
Unfortunately, Counter is not available before 2.7, so for many people it's a
bit too early to require it.

~~~
njharman
Two years old is a long enough.

~~~
sitkack
OSX 10.6.8 still has Python 2.6 as the system Python.

------
lihaoyi

        halve_evens_only = lambda nums: map(lambda i: i/2, filter(lambda i: not i%2, nums))
    

I still find it rather silly that python doesn't supper a nice list
map/filter; it could be so much nicer

    
    
        nums.filter(lambda i: i%2 == 0).map(lambda i: i/2)
    

If they did, even including the annoyingly long-to-type "lambda". List
comprehensions are cool and all, but do not really scale visually (i.e. get
rather messy) when you have more than one map and filter step.

These arbitrary break-away from OO method style into module+data style (len(L)
is another!) are one of the things I hate most about Python. There are some
reasons for doing so, but a pure-OO (like Scala) or pure-method+data (like F#)
would have saved me many a runtime error.

~~~
darkstalker
the D language supports that kind of syntax

nums.filter!(i => i%2 == 0).map!(i => i/2);

~~~
fferen
To elaborate, this is because of D's Uniform Function Call Syntax
(<http://www.drdobbs.com/blogs/cpp/232700394>) - any function that takes an
object as first argument can be called as though it were a method of that
object; "map" and "filter" are actually functions in the std.algorithm module.
It's a pretty neat trick and although in principle it could make things harder
to reason about I haven't had any problems with it so far.

------
Estragon
Another handy one I saw recently:

    
    
      varname, = [x for x in l if predicate_with_single_truth_value(x)]
    

The comma after varname is an implicit assert that the list comprehension only
contains one element.

~~~
tikhonj
Trailing commas are really easy to miss. When reading this line of code, I did
not notice it immediately; I originally assumed that varname was being
assigned a list.

This sort of code would be very confusing when I'm just quickly reading
through a procedure trying to find the potential bug.

~~~
almost
Agreed. It could be written much more clearly in my opinion like this:

    
    
        [varname] = [x for x in l if predicate_with_single_truth_value(x)]

~~~
euccastro
I had to check that on the REPL. I'm surprised that even works and I can't
think of a good reason why should list syntax be allowed as a lvalue, in
addition to tuple syntax.

~~~
Estragon
It's called destructuring assignment. It's been around for a while.

<http://dunsmor.com/lisp/onlisp/onlisp_22.html>

~~~
euccastro
I mean why would you want to allow _both_ list and tuple syntax for exactly
the same semantics, when either of them would be enough.

~~~
almost
I just wish we had destructing assignment for other types as well. And maybe a
proper pattern matching!

------
ajitk
Till now I had never seen Ellipsis. It seems very similar to slice notation
[:]. Found an StackOverflow comment [1] that has more details about usage of
Ellipsis in slicing higher dimensional array numpy.

[1] [http://stackoverflow.com/questions/118370/how-do-you-use-
the...](http://stackoverflow.com/questions/118370/how-do-you-use-the-ellipsis-
slicing-syntax-in-python)

~~~
grifaton
Here's a rare example of obfuscated python, deceptively called "Python's
Ellipsis Explained": <http://blog.brush.co.nz/2009/05/ellipsis/>

------
euccastro
There is a builtin function called `reversed`. You'd better remember that than
the "useful" [::-1] idiom.

The recommendation on `iteritems` had better be generalized to include
`iterkeys`, `itervalues`, and other opportunities for using iterators rather
than building lists. A note that the 'iter...' versions are removed in Python
3 (because iterator behaviour becomes the default) would be appropriate here.

In relation to collections, itertools is a great module to get familiarized
with. I import * from this module. I consider functions there as if they were
builtins.

"Conditional assignment" is a weak and misleading name. "Conditional
expressions" is more descriptive. There is no assignment in

    
    
      print "yes" if some_condition else "no"
    

as the article acknowledges later.

Using Ellipsis for getting all items is a violation of the Only One Way To Do
It principle. The standard notation is [:].

I commend the good intentions of the writer, but I'm surprised that this
article got 144 upvotes in HN.

~~~
jongraehl
list(reversed(xs)) to make a copy rather than a view, though

------
jc4p

      Generally, most use this:
        freqs = {}
        for c in "abracadabra":
            try:
                freqs[c] += 1
            except:
                freqs[c] = 1
    

Who does this?!

~~~
makecheck
Not only is it overly-verbose and ultimately unnecessary (as shown by the
alternative that follows in the article) but this mechanism is actually
broken. For instance if "freqs" were data given to you from somewhere else and
"freqs[c]" happened to have a type that cannot legally have 1 added to it,
you'd want to see this error. The "except:" however will absorb any and all
exception types and respond to all of them by setting the value to 1.

~~~
Estragon
Also, raising exceptions used to be quite slow, which could hurt for a sparse
counting set. Don't know whether that's still the case.

~~~
underwater
Also Exceptions should be used in "exceptional" circumstances and not as part
of normal flow.

~~~
koenigdavidmj
One exception (teehee!) to the rule: file operations and other things where
atomicity matters. Example code:

    
    
      if not os.path.exists("foo"):
          os.mkdir("foo")
    

That introduces a race condition. If foo does not exist on the first line but
is created by something else on the second line then this will raise an
exception. The proper code is:

    
    
      import errno
      try:
          os.mkdir("foo")
      except OSError as exc:
          if exc.errno != errno.EEXIST:
              raise
    

That doesn't have the race condition.

(Yes, that's wordy. Eventually they plan to add to Python 3 a fancier
exception hierarchy described at <http://www.python.org/dev/peps/pep-3151/> ,
which lets you filter at a more fine-grained level, as below.)

    
    
      try:
          os.mkdir("foo")
      except FileExistsError:
          pass

~~~
makecheck
It's good to point out the race condition.

To solve this particular problem in future code more compactly however, note
that Python 3.2 (finally) adds an "exist_ok" Boolean keyword parameter to the
multi-directory variant, os.makedirs(). In other words, calling
os.makedirs("mydir", exist_ok=True) will silently ignore existing directories
and only raise if other errors occur.

------
bcl
My only complaint with this list is conditional assignments. They make it
harder for me to understand the code. I am used to the if being at the start
and when it isn't it takes more time for me to parse what the real meaning of
the line is.

~~~
mcguire
One of the reasons Python stopped being my favorite programming language was
the appearance of what seemed to me to be Perl envy. I hadn't seen conditional
assignments before, but they don't exactly make me regret that choice.

------
harlowja
Probably best not to use iteritems, in python 3, it won't be there, "the
dict.iterkeys(), dict.iteritems() and dict.itervalues() methods are no longer
supported."

~~~
andreasvc
The iter* methods appear in Python 3, just without the iter prefix. Therefore
it is good to use them in Python 2 because that makes it possible to translate
the code automatically with the 2to3 script. Otherwise possibly superfluous
list conversion might get added, e.g., list(d.keys()).

~~~
DasIch
.keys(), .views() and .items() in Python3 return a memoryview which happen to
be iterators but do far more.

------
Terretta
Satyajit Ranjeev:

Since this is posted by the you as the author, I'll comment here: some
JavaScript is running on page load that blanks the entire page in Safari and
iCab on iPad, making the page turn white except for the bullet symbols, and
making the article unreadable.

I was able to read it only by disabling JavaScript or parsing it with
Readability. (Both disable my ability to comment about this bug there.)

------
shenberg
There is one slight mistake there - saying that [::-1] is a special case. An
empty value in a slice implies the beginning or the end, and when the stride
is negative, the beginning is the last index, while the end is 0 - making
[::-2] for example start from the last element and go down in jumps of two.

------
FiloSottile
You should think of adding the use of `with` context managers.

------
gcb
isn't items more efficient than iteritems when you are going to go over all
the items for sure on a short list?

~~~
Estragon
No, never. The items method generates an intermediate list.

------
phpfanboi
PHP: Facebook.

Python: ?

Ruby: ?

~~~
mappu
Python: Reddit.

Ruby: Twitter.

