
Python List Comprehensions Explained Visually - ingve
http://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/
======
tsumnia
Maybe its me, but "Explained Visually" sounds more like it would have some
'visual' aspect to it besides a GIF of code (don't get me wrong, I think most
code should be made an image to limit copy/pasting).

A website like VisuAlgo ([http://visualgo.net/](http://visualgo.net/)) is a
great example of exampling something visually should mean

~~~
th
I agree. I was using "List Comprehensions: Now In Color" as a working title
but I thought that was too nonsensical. It was actually the colorized code and
not the gifs in particular that made me think "visual".

Sorry for the misleading title!

~~~
x0
What's the font you're using in that gif? It looks good.

~~~
th
Inconsolata:
[https://en.wikipedia.org/wiki/Inconsolata](https://en.wikipedia.org/wiki/Inconsolata)

~~~
x0
Ah, cheers :)

------
danso
Coming from Ruby to Python...I could not believe how damn hard it was to
comprehend the beautifully explained and simple code (just 21 lines!) in Peter
Norvig's writeup of how to create a spelling corrector [1], purely because of
the list comprehensions (and a few other visual differences, such as partial
list notation)

For example:

    
    
         inserts = [a + c + b for a, b in splits for c in alphabet]
    

It's no different (conceptually, at least) than doing:

    
    
         inserts = []
         for a, b in splits:
            for c in alphabet:
                inserts.append(a + c + b)
    

Or in Ruby (OK, maybe this isn't perfect Ruby):

    
    
         inserts = splits.reduce([]) do |x, (a, b)|
             alphabet.each{|c| x << a + c + b }
             x
         end
      
    

But the list comprehension was so difficult for me to read that it might as
well have been a brand new concept. But I found I got quickly over it after
rewriting all of his comprehensions as for-loops, and then trying to convert
them back into comprehensions. It was a good reminder that sometimes the
barrier is just plain muscle/memory reflex (and probably much more so for
beginning programmers...)

[1] [http://norvig.com/spell-correct.html](http://norvig.com/spell-
correct.html)

~~~
mjevans
I most frequently use List Comprehensions in a nested manor, whenever I do
that I /ignore/ the normal python syntax and do this for readability.

    
    
        inserts = [ a + c + b
                    for a, b in splits
                    for c in alphabet ]
    

It has the effect of showing the nested loops stacked like they would be in
code, but without intending.

~~~
nomel
For the projects I manage, I would have the dev rewrite that as a proper for
loop.

That's way too confusing and hard to parse. List comprehensions are great for
simple operations, but once you start nesting, you're just being a jerk to the
next dev that has to read your code.

Code should be clear first, elegant second.

~~~
troym
> Code should be clear first, elegant second.

I disagree: code should be correct before it is clear. And because it's so
easy to mess up a for loop (for me at least), I choose list comprehensions
where reasonable.

> you're just being a jerk to the next dev that has to read your code

That's not very charitable to either party. You're assuming that the motive of
the author is to be a jerk, and you're assuming that the reader won't
understand. If I had a dev on my team who couldn't read a list comprehension,
I'd (a) wonder how they were hired, and then (b) teach them.

~~~
ebola1717
I've written a fair share of Haskell, and I still get tripped up by list
comprehensions in python - even my own. It's not that it's impossible to
understand, and when there's only one predicate, I think it's fine:

    
    
      for row in [[i*j for i in range(1, 8)] for j in some_list if j % 2 == 0]:
        some_op(row)
    

vs

    
    
      for j in some_list:
        if j % 2 == 0:
          row = [ i * j for i in range(1,8) ]
          some_op(row)
    

I would compare it to sentences and paragraphs. The former feels like a run-on
sentence, while the latter is more obvious, cause it has one predicate per
line. Also, list comprehensions are a bit like yoda-speak - it introduces the
verb before the subject. You have to untangle the order of operations, rather
than having the order read top-down and left-right.

Of course, I'm a rubyist, so I'd prefer:

    
    
      some_list.select { |j|   j % 2 == 0 }
               .map    { |j|   (1..8).map { |i| i * j } }
               .map    { |row| some_op(row) }
    

Though definitely need to do something about that second line.

Haskell list comprehensions are a bit easier to parse because they have
symbolic delimiters, the fact that Haskell is naturally more terse, and
because you can always check the type of the list

    
    
      [ [i*j | i <-[1..8]] | j <- [1..4], j % 2 == 0 ]

------
chestervonwinch
Cool. As a math guy, I enjoy the similarity between comprehensions and set-
builder notation:

[https://en.wikipedia.org/wiki/Set-
builder_notation](https://en.wikipedia.org/wiki/Set-builder_notation)

~~~
Gnewt
In the opposite direction, having knowledge of Python's list comprehensions
(and to some degree, an understanding of lazy evaluation) helped me a ton when
set-builder notation was introduced in my first discrete math class.

------
iamsohungry
Are Python List Comprehensions really hard for people? I feel like my memory
of learning them was I saw one in production code I was modifying, intuited
what it did, pulled open a REPL and played with it for a bit, and then moved
on.

~~~
brbsix
I agree. It really surprised me to see this getting so much traffic. I
sometimes get hung up with nested comprehensions when I haven't been using
them in a while, but other than that, they are quite mundane.

------
coherentpony
Is it just me or were there zero visualisations in this post?

~~~
bgilroy26
It might not be what you were expecting, but there's an animation if you
ctrl-f "Here’s the transformation animated:"

~~~
coherentpony
:) I did see that one and indeed it wasn't what I was expecting.

------
ebola1717
That translation is kind of why i don't like python list comprehensions
though. It's often clearer as a for-loop or map/filter, cause the different
predicates are broken out into separate lines, rather than lumped together in
one long line.

~~~
d0mine
It is not much easier to understand the explicit loop:

    
    
      doubled_odds = []
      for n in numbers:
           if n % 2 == 1:
               doubled_odds.append(n * 2)
    

compared to the list comprehension:

    
    
      doubled_odds = [n * 2 for n in numbers if n % 2 == 1]
    

Here's the explicit loop without newlines for comparison:

    
    
      for n in numbers: if n % 2 == 1: doubled_odds.append(n * 2)
    

Notice that the for-loop and the if-statement are in the same order in the
loop and in the list comprehension. Only the value expression (n * 2) is out-
of-order. OPs jumping around is often unnecessary in the gif.

The one line list comprehension is as much readable for people familiar with
Python as the variant with the explicit loop.

It is not uncommon to use several list comprehensions in a function. If you
rewrite them as explicit loops then it may increase the line count 4 times
e.g., it can convert a simple 3-lines function that could be understood at a
glance into 12 lines function that you have to read line by line to understand
-- it is easy to keep track of 3 values on 3 lines but it is not clear whether
it is true for 12 lines.

~~~
iamsohungry
I think the problem here is that nested comprehensions and filter
comprehensions are almost always too complex. If you are doing something that
requires that, you should break it up and give the pieces names:

    
    
        odd_numbers = range(1, limit, 2)
        doubled_odd_numbers = [n * 2 for n in odd_numbers]

------
c3534l
Frankly, comprehensions are one of Python's worst features. Some properly
implemented functional syntax would be both more readable and concise. Plus
double-nested (I'm not sure what you'd call it) comrpehensions seem to be
backwards. Rather than [ingredient for ingredient in recipe for recipe in
cookbook] it's [ingredient for recipe in cookbook for ingredient in recipe]
which makes it sound as if there's only going len(recipe) number of
ingredients in your list. Plus it's so verbose and horizontal, half the time
if you want descriptive names you're gonna write something so verbose it
doesn't even make it that much more readable.

~~~
has2k1
When reading code I find _comprehensions_ easier to comprehend than
combinations of _map_ , _filter_ and _reduce_. That is considering that I
played around with some Common-Lisp before getting into Python.

The parts in the comprehension

    
    
        [*wanted* *iteration(s?)* *condition?*]
    

compares well to

    
    
        *iterations(s?)*
            *condition?*
                *append wanted*
    

for normal looping behaviour without changing order of the iterations.

One of my favourite comprehensions is of the sort

    
    
        [item for item in items in for i in (0, 1)]
    

of course this can be the same as

    
    
        list(items) * 2
    

but the comprehension is more versatile

    
    
        [item for item in items in for i in (0, 1) if items[0] != 'a']
    

For most once the mental fog clears comprehensions become natural very quickly
as they are not that far removed from normal loops.

------
cballard
> Sometimes a programming design pattern becomes common enough to warrant its
> own special syntax. Python’s list comprehensions are a prime example of such
> a syntactic sugar.

I disagree with this thesis, although the post is otherwise a very nice guide
to list comprehensions. Special syntax for specific types indicates a failure
of the language to be generic enough.

List comprehensions anoint lists as a special _thing_. You don't get to play
with list comprehensions, unless you're using the type that the language has
decided to let you play with. If you decide to use a different type for some
reason, you... can't.

map and filter can be part of a typeclass/interface/protocol (in Python these
would just be informal), so you can use them on arbitrary types. If you want
to switch your list type, it should just work.

I write a lot of Swift, and I'm constantly frustrated that optional chaining
(?.) and exceptions are special _things_. I can't implement ?. for my type
(say, a Result). Only the language creators can use it. Somewhat confusingly,
?? _is_ a thing I can implement.

In Python's case, I think that list comprehensions are necessary because the
language's support for first-class closures is poor.

~~~
dragonwriter
> List comprehensions anoint lists as a special thing. You don't get to play
> with list comprehensions, unless you're using the type that the language has
> decided to let you play with.

That's one reason why I like Scala's comprehensions; they have the conciseness
of list comprehensions, but are more generic.

~~~
whateveracct
aka monads ;)

------
sdrothrock
One basic misstep here for me is a comparison of why list comprehensions are
better than loops -- speed, ease of reading, etc. There are a lot of
comparisons, but no discussion of WHY list comprehensions.

I work on a project that's written mostly in Python; on occasion, other
coworkers have to edit/work with my code and they're not used to Python at
all, so I find myself writing for loops instead of list comprehensions for
clarity.

~~~
TheCowboy
I find the utility of list comprehensions is maximized when I'm in a Python
shell and dealing with indentation when writing a loop can become obnoxious to
edit. I tend to have a Python shell tab open in my terminal throughout the
day, and it's nice to bang out a quick loop in a one-liner.

When writing actual code, I tend to stick with regular for loops.

------
tsurantino
For those confused - the visual part is done in a GIF (i.e.
[http://treyhunner.com/images/list-comprehension-
condition.gi...](http://treyhunner.com/images/list-comprehension-
condition.gif)).

I think animating the transformation from a normal loop to a list
comprehension is a great way to show how the syntax translates between the two
forms. Very awesome and comprehensive post.

~~~
hatmatrix
Maybe I'm an FP snob but it would have been nice to mention the analogy to set
builder notation (though lists aren't sets - maybe multisets or bags) to
indicate the existence of higher abstraction; not just syntactic sugar on the
for-loop.

However, from a practical point of view I think it is great introduction for
procedural programmers to begin using list comprehensions.

~~~
dragonwriter
> though lists aren't sets - maybe multisets or bags

arrays/lists (like maps/hashes, though with a more narrow restriction on the
indices) are (or at least are isomorphic to) sets of pairs of (index,
element).

They carry information about order which does not exist in multisets/bags.

------
keepitsurreal
I've been reviewing Python this week and this was a great brush up. Thanks!

------
jredwards
nested comprehensions are kind of a nightmare to read, but basic list
comprehensions seem incredibly straightforward to me. So much so that the
explanation is more complex.

------
skadamat
Not sure how this is visual, neat post either way!

------
y04nn
When I started to learn Python, I was quite lost with this feature, but now I
love using it.

------
plusquamperfekt
Is it really that hard to grasp?

