
Python iteration - joshbaptiste
http://nedbatchelder.com/text/iter.html
======
dustingetz
while > for > foreach - increasingly higher level abstractions.

article stops at iteration (foreach) but misses an even higher level
abstraction: filter, map, reduce. each of these can be implemented on top of
foreach, and i haven't yet seen a loop that can't be decomposed into a
combination of map, filter, reduce. also note that filter/map/reduce are
expressions, not statements, so they are more "mathy" and thus easier to
reason about.

its interesting that these are not necessarily idiomatic python despite being
a slightly higher level abstraction. note they are idiomatic in all "real"
functional languages, which python is decidedly not. also list comprehensions
are faster than map/filter in python. (this is probably python's fault and is
really dumb because you could just implement map/filter as a list
comprehension. some people have criticized Guido for some of his design
choices[1].)

[1] [http://www.quora.com/What-are-the-main-weaknesses-of-
Python-...](http://www.quora.com/What-are-the-main-weaknesses-of-Python-as-a-
programming-language)

also on topic: <http://docs.python.org/library/itertools.html>

~~~
ggchappell
He does do a "map", although without using the function of that name:

    
    
      answers = [f(x) for x in iterable]
    

Also, the "zip" operation gives loops that seem to me to be unable to be
decomposed into map, filter, reduce. E.g., how would you print a file with
line numbers using only map, filter, reduce?

Regardless, I agree with your basic point: that thinking of loops in terms of
high-level loop primitives, is a Good Thing.

~~~
dustingetz

        def zip(*seqs):
          return map(lambda *items: items, *seqs)
    
        assert [(1, 'a'), (2, 'b'), (3, 'c')] == zip([1,2,3],['a','b','c'])
    
    
        def count(start=0):
          while True:
            yield start
            start += 1
    
        from itertools import izip, islice # lazy versions
        with open('/etc/passwd', 'r') as f:
          zipped = izip(count(), f.readlines())
          for numbered in zipped: print numbered
    

izip and islice are trivial to implement:
[https://github.com/dustingetz/sandbox/blob/master/etc/lazy.p...](https://github.com/dustingetz/sandbox/blob/master/etc/lazy.py)

~~~
ggchappell
Well, strictly speaking, you have answered my objection. But I guess I was
thinking in Haskell. If Python's map takes multiple iterables, then, yes, zip
is a special case. But writing zip by mapping a function without side effects,
using a "map" that works only on a single iterable (as in Haskell) strikes me
as a rather different thing.

So I suppose that the truth of "this loop can be written using map" depends on
exactly what one means by "map".

~~~
dustingetz
interesting. how about

    
    
        def head(seq): return seq[0]
        def rest(seq): return seq[1:]
        def transpose(mtx):
            a = map(head, mtx)
            b = map(rest, mtx)
            return [a] + ([transpose(b)] if b[0] else [])
    
        def zip(xs, ys):
            return transpose((xs, ys))
    
        def zip2(*seqs):
            return transpose(seqs)
    

this works but is sort of icky in python, any ideas? this is also where
functional python starts to fall apart - no tail call recursion, and default
data structures aren't persistent. based on a haskell solution which is nicer.
[http://stackoverflow.com/questions/2578930/understanding-
thi...](http://stackoverflow.com/questions/2578930/understanding-this-matrix-
transposition-function-in-haskell)

------
benhoyt
Simple and useful overview. I find enumerate() quite useful. I should probably
define generators more often.

One built-in function I basically _never_ use is zip(). I know how it's used
and what it does, and I grok the names/ages types of examples that are always
used to explain it. But I've simply never needed it in real code. Anyone else
use it often, and in what contexts?

~~~
grifaton
I sometimes use zip as a poor man's matrix transposition operator:

    
    
        >>> m = [(1,2,3), (4,5,6), (7,8,9)]
        >>> zip(*m)
        [(1, 4, 7), (2, 5, 8), (3, 6, 9)]

~~~
mickeyp
If you want to transpose a matrix-like list of lists, but the sizes don't all
match up, you can use izip_longest from itertools and give it a fillvalue to
"fill in" for the missing elements:

    
    
      >>> m = [(1,2,3), (4,5,6), (7, 8)]
      >>> zip(*m)
      [(1, 4, 7), (2, 5, 8)] # Wrong
      >>> from itertools import izip_longest
      >>> list(izip_longest(*m, fillvalue=None))
      [(1, 4, 7), (2, 5, 8), (3, 6, None)]

------
krupan
First examples of generators that I've seen that made them look useful. I'd
never thought of using a generator as a filter, or for turning nested for-
loops into one for-loop. I have some code I can go clean up now.

~~~
ff0066mote
Here's a pile more to blow your mind with: <http://www.dabeaz.com/coroutines/>
[2009]

------
ff0066mote
I would have liked to have seen mention of this while loop instead of the
index-based one.

    
    
        while my_list:
            v = my_list.pop(0)
            print v
    

Although it doesn't fit the theme of the others because it consumes the list,
it is certainly a more natural use of "while" to loop over a list.

