

The Zen of Python by Example - pjo
http://artifex.org/~hblanks/talks/2011/pep20_by_example.py.txt

======
hblanks
By way of an afterword, I'm both pleased and surprised with the thought and
attention people have given to these slides, which I made for a perhaps too
hastily prepared, 10 minute talk at this Tuesday's joint PhillyPUG / philly.rb
meeting. (Many thanks to the organizers, other speakers, and attendees -- it
was a great turnout and a good time, too.)

I agree with several commenters that, like any collection of aphorisms, the
lines in PEP 20 are not real rules but rather rules of thumb. Nevertheless,
rules of thumb, or heuristics, are surprisingly important to the practice of
engineering,* and perhaps even more so to the work of writing software. As
such, nothing in PEP 20 should be followed hard and fast, but almost all of it
is worth considering. Also as such, there's nothing (except for the comment on
Guido) that makes PEP 20 relevant only to Python. It happens that these rules
have a particular currency in the Python community (i.e., when people talk
about being Pythonic, they're often talking about something described in the
Zen of Python). But if you work in a different language, there's probably
something worthwhile in it there, too.

On a different note, it is flattering to find this code, hacked out such as it
was, has been reviewed by a far larger number of peers than the (comparably)
few, albeit exceptional, engineers I work with at Monetate. Although it was
hardly intended to be read on its own, or to run beyond the feeble purpose of
generating its own slides, I have tried to correct what errors people have
found. Ambiguities, of course, remain, but you can find an updated document at
the original URL, or a diff of the changes at:

    
    
        http://artifex.org/~hblanks/talks/2011/pep20_by_example_2011_02_10_edits.patch
    

To respond to a few specific or general points:

\- Yes, of course, importing from within in a function is almost always a bad
idea (the only exception I've ever seen is Google App Engine, and that's
obviously just a performance hack). Nor would you ever want to actually
convert your floats to strings and back -- my only point there was to
illustrate the fact that yes, those are the rules of floating point, and so
you can't expect Python to break them for repr().

\- The order of examples is not necessarily bad first, good second. (Alas for
inconsistency!) So, for instance, ORMs may be complex, but they're often much
better than writing your own SQL, which even in this trivial example starts to
get complicated. Similarly, doctests often make code more readable than unit
tests hiding in some other file -- although in honesty the doctest/unittest
split is probably a false dichotomy; for some things, doctest is much cleaner,
for others (like things that require setting up a DB schema first), unit tests
are.

\- Finally, I still didn't have the wherewithal to call pygments directly
(it's certainly easy and better; I just knew the command already). But I did
add HTML output, since this the web, after all, to:

    
    
        http://artifex.org/~hblanks/talks/2011/pep20_by_example.html
    
    

* Cf. Billy Koen's enjoyable discussion of engineering heuristics, _Discussion of the Method_ or _Discussion of the Engineering Method_. (Some chronicle is available at <http://www.me.utexas.edu/~koen/OUP/HeuristicHistory.html>)

~~~
Locke1689
Do you by any chance know Jack Lange? That's the only other time I've ever
seen an artifex.org domain.

~~~
hblanks
Yes, I do. Jack's an old friend an another one of the admins for Artifex.org.

~~~
Locke1689
Oh, cool! He was an advisor of mine during a summer REU a couple years ago
when I was working on Palacios. Welcome to HN, by the way!

------
d0mine
There is a bug in #1. The condition should be `i % 2 == 0` to get even
numbers:

    
    
      def halves_of_evens(nums):
          for i in nums:
              div, mod = divmod(i, 2) #NOTE: it might be slower
              if mod == 0: #NOTE: `not mod` is not explicit enough
                 yield div
    

#2: Function level import is a bad practice though it might be justified in
this case.

#3: `users` is not defined. `sqlalchemy` names are not defined. The file
should be opened for writing in the json example.

#5 It is simpler to allow the animal to know its kind:

    
    
      def identify(animal):
          return animal.__class__.__name__
    

or just:

    
    
        animal.kind
    

#6 `elem.xpath('./a/@href')` is simpler than

    
    
      [a.attrib[’href’]
       for a in elem.find(’./a’)
       if ’href’ in a.attrib]
    

Also `elem.iterlinks()` could be used
<http://codespeak.net/lxml/lxmlhtml.html#working-with-links>

There is no `lxml.CSSSelector` (python-lxml 2.2.6-1). Use `elem.cssselect`

#7 You don't need to add `ifmain` stub in every module:

    
    
      $ python -mdoctest module_with_doctests.py
      $ python -munittest module_with_unittests.py
    

# 8 & 9: The first variant of `make_counter()` won't work without `nonlocal i`
-- it leads to `UnboundLocalError` otherwise.

Despite the fact that it works on my machine on all available Python
implementations such code should be avoided:

    
    
      assert float(’0.20000000000000007’) == 1.1 - 0.9 #XXX horror!
    

# 10 & 11: Use stdlib's version:

    
    
      try: import json
      except ImportError:
           import simplejson as json
    

It is a matter of preference but it makes testing more predictable.

#13 `__builtins__` is not defined on jython:

    
    
      try: import __builtin__ as builtins
      except ImportError:
           import builtins # py3k

~~~
siika2000
Why is function level import considered bad practice?

~~~
d0mine
Deadlocking on the import lock might be introduced by an innocent code that
uses a library with function level imports.

In general globals should be written at a global level.

------
j_baker
You know, the Zen of Python makes a for a really cute thing to read, but even
Guido himself has admitted that it requires quite a bit of interpretation and
doesn't apply in all cases. Generally, I feel that applying it directly to
code is tricky (except for trivial pieces of code like these). I think it's
better to treat the Zen of Python as a broad vision for how Python code should
look rather than as a concrete set of rules for code.

For example, explicit isn't _always_ better than implicit. I mean, by that
standard alone, the idea of a garbage collector going around implicitly
freeing memory would be terrible.

~~~
pyre
I think that the issue is that some of the statements can come into conflict.
You can easily make something that is more complex (i.e. less simple) in an
attempt to make something that is more explicit and less implicit.

------
KirinDave
I couldn't help but feel irritated by this document. It's cute and all, but it
implies Python is somehow unique—culturally or technologically—in grasping at
these concepts. You could easily have written this as a Ruby or Scheme
manifesto with nearly identical wording and had a valid message.

And then, in our very first example, there is a use of lambda, a misnomer of a
concept Python doesn't really support? My brow furrowed slightly more.

I can't help but feel like these people are celebrating their investment in
their tools rather than the virtues of the tools themselves.

~~~
TheBoff
The first section is pointing out list comprehensions as a better syntax than
lambda, I believe. Also, the zen of python has been around for a long time:
<http://www.python.org/dev/peps/pep-0020/>.

There's an import statement that will print it I believe, but I'm not sure
exactly what it is.

~~~
KirinDave
It's a better syntax than _Python's 10-year-out-of-date notion_ of lambda. For
example, this is what a more modern language does with lambdas:

    
    
        numbers.filter( _ % 2 == 0 )
    

This document is old, so we can probably excuse it... but how many Python
folks know that?

~~~
d0mine
What is the language?

    
    
      numbers.filter( _ % 2 == 0 ).map( _ / 2 )
    

Zen of Python doesn't contain any lambdas (the link from the grandparent
<http://www.python.org/dev/peps/pep-0020/> ). List Comprehensions are 10 years
old <http://www.python.org/dev/peps/pep-0202/>

How do you write:

    
    
      lambda x, y=1: x+y
      lambda *args: reduce(lcm, args)
      lambda **kwargs: kwargs.get('a') or kwargs.get('b')
    
      def lcm(a, b):
          return a*b // gcd(a, b)
    
      def gcd(a, b):
          while b:
              a, b = b, a % b
          return a

~~~
KirinDave
The link for the thread:
[http://artifex.org/~hblanks/talks/2011/pep20_by_example.py.t...](http://artifex.org/~hblanks/talks/2011/pep20_by_example.py.txt)

It does indeed mention a lambda in the very first example, if only as a
competition with list comprehension syntax. But even Python's list
comprehensions look a little dated now.

~~~
eru
Python's list comprehension is actually inspired by Haskell (or was it the
other way around)?

What do you mean by dated? What new ideas have come up for List comprehension
syntax?

(I agree that Lambda should be call fun or fn or \\. An implicit _ might also
work, though that's somewhat against Python's spirit---c.f. the explicit self
in methods.)

~~~
sausagefeet
Like most good ideas, it came from Haskell :)

The part that bothers me about the list comprehension section is that
conflating list comprehension and lambda. Is it telling me that list
comprehension is better than map/filter or better than lambda? I don't know.
Quiet honestly, in many situations I find:

halves_evens = lambda nums : [i/2 for i in nums if nums % 2 == 0]

to be the cleanest.

------
dorkitude
I love this document. This stuff is my porn.

That said, I'm having quite a hard time grappling with section 4.

First, what is the 'users' keyword all about in the "bad" area?

Second, why is ORM the "bad" approach, and manual construction of SQL strings
the "good" one? It seems to me that the abstraction and centralization of
query language is paramount (examples: to afford oneself an easy one-day
migration to GQL/BigTable, or to a document-oriented database like MongoDB or
the upcoming Couchbase).

~~~
Legion
Just like how he reached for PyUnit for "readability counts", I definitely
think of ORMs over raw SQL when it comes to "readability counts". Especially
when you're actually working with data, not just the table creation step.

~~~
endtime
Agreed completely. The difference between a good ORM and raw SQL is like the
difference between, well, Python and assembly, IMO. Perhaps the difference is
a little less extreme, but it's similar in spirit.

~~~
coolgeek
er, as someone who did much database access in assembler, I'm going to have to
disagree with your analogy.

You could argue "between Python and C"... but most certainly NOT assembler

~~~
endtime
Wow, I'm sorry you had to do that, haha. Python and C is surely a more
reasonable comparison in terms of actual ease of use; I was thinking in terms
of "raw instructions" versus "high level abstraction".

------
ff0066mote
Forgive me if I am wrong, but I believe that section 1 contains mistakes. In
the implementations of evens_only, shouldn't both of the places where modulus
is used be preceded by "not"?

This isn't meant to detract from the truth that "Beautiful [code] is better
than ugly," I merely thought the implementation should be correct.

~~~
rodion_89
or better yet i % 2 == 0

~~~
highfreq
I agree. Using the truthiness of non-boolean values has always seemed
questionable in any language (Javascript is possibly the worst offender). For
many statically typed languages this is caught as a syntax error, which is
nice for those of use prone to occasionally typos.

~~~
baddox
The boolean equivalents of non-boolean values are well-defined and fairly
simple in Python, and I believe it's encouraged (or at least extremely common)
to use them as such.

------
singingwolfboy
A quarrel -- Pygments is written in Python, and is appropriately modular. Why
did the author use subprocess and create new Python interpreter to run
Pygments, when he could have just imported it and run it directly, without
kicking off a new interpreter?

------
baddox
Awesome. Here it is in a paste bin with syntax highlighting:

<http://paste.pocoo.org/show/336126/>

------
l0nwlf
Here is the history of "Zen of Python", aka ">>>import this" :

[http://www.wefearchange.org/2010/06/import-this-and-zen-
of-p...](http://www.wefearchange.org/2010/06/import-this-and-zen-of-
python.html) , a blog post written by Barry warsaw.

On a side note, other eggs are :

>>> from __future__ import braces

>>> import antigravity

------
baddox
This is a bit off topic, but what are your opinions on the validity of the 80
character width restriction? I think the 80 character limitation is
increasingly rarer to actually encounter these days. Even people developing
via a shell will almost certainly have much more than 80 characters of width
to play with.

We've finally accepted that web developers can expect visitors to have a
higher resolution than VGA, and those are potentially non-tech-savvy users.
Can't we expect developers to have more than 80 character width editors? Plus,
I think splitting lines is pretty ugly.

~~~
j_baker
On the other hand, I prefer not to have all of my desktop taken up by my
editor. Usually, I dedicate half of it to my editor and half to a command-
line. Now, 80 characters might be a bit _too_ small, but I think it's a good
goal to shoot for as long as you bear in mind that it's a guideline and not a
hard rule.

------
sausagefeet
I have a hard time agreeing with this document. In many cases it is hard to
tell which part of the wrong solution they are actually correcting. For
example on the one where they write measurements out to a file, in the first
case they use sqlalchemy to write to a sqlite db and in the 'correct' case
they write json to a file. So what is the issue that is be corrected here? Is
using sqlite without sqlalchemy ok? Both?

------
suraj
I do not like example 4 "Complex is better than complicated". Use of string
concatenation is going to lead to SQL injection bugs and SQLAlchemy version
actually looks better to me.

Can somebody please enlighten me on how using raw sql is preferable?

~~~
Luyt
You could use dbapi's query parametrisation instead of string concatenation,
or maybe use sqlalchemy's SQL Expression if you don't want to write SQL.

ORMs are not a panacea. Especially when the database schema gets more
complicated they require more setup and tuning, and there's always the problem
of how much data the ORM should suck in vs. how lazy it can behave.

------
coolgeek
Hey, I was at the RedSnake meetup where this was presented on Tuesday night!

Go Hunter!

~~~
phillytom
It was a great event - there is a short writeup of the other talks here:
[http://trevmex.com/post/3198570101/redsnake-1st-annual-
phill...](http://trevmex.com/post/3198570101/redsnake-1st-annual-philly-ruby-
python-meetup-notes)

------
nmb
For the evens_only implementation, is there a reason to use map/filter vs a
generator? eg:

    
    
       evens_only = lambda nums: [n/2 for n in nums if n%2==0]

seems to work just as well.

~~~
l0nwlf
I think that is List Comprehension and not Generator. 'Return' would be
replaced by 'Yield' in case of a generator.

------
ludwigvan
PDF version is easier to follow:

<http://artifex.org/~hblanks/talks/2011/pep20_by_example.pdf>

------
drstrangevibes
I find all this discussion about the rules slightly amusing, as it is Zen. The
rules are there to instruct the apprentice, to guide adept, but there are no
rules for a master _. shu ha ri.

<http://en.wikipedia.org/wiki/Shuhari>

_ I do not claim to be a master

