

The scope of index variables in Python's for loops - zoidb
http://eli.thegreenplace.net/2015/the-scope-of-index-variables-in-pythons-for-loops/

======
Animats
One problem with implicit declaration is that it's sometimes hard to decide
whether some construct should create a new variable or not. Python's basic
rule, that everything has function scope, was at least simple. I'm surprised
that was changed for list comprehensions.

Go has worse problems. The ":=" operator in Go indicates the creation of a new
variable. Except when there are multiple variables on the left side of the :=.

    
    
       a, b := 1,2
    

can declare a, b, or a and b, but not neither of a and b. (See
[https://play.golang.org/p/zE7AIMhleo](https://play.golang.org/p/zE7AIMhleo)).
This can create some hard to find shadowing bugs.

Declaration-free language are hard. It's impressive how far Python got without
declarations. Now there's an advisory typing proposal for Python. It's not a
very good proposal; some of the type info goes in _comments_ because it didn't
fit the syntax.

~~~
aroberge
> It's not a very good proposal; some of the type info goes in comments
> because it didn't fit the syntax.

Actually, the reason why it is put in the comments is that it is implemented
as an experimental feature so that its usefulness can be evaluated without
having to (possibly only temporarily) change the syntax.

------
acadien
Yes another super useful trick you can use with this scope trick is to get the
length of a file (or its last line) just do:

for i,v in enumerate(open(f)): pass

now 'i' is the number of lines in your file and 'v' is the last line. This is
also blazingly fast, especially useful when you have ~several GB large log
files to parse.

Great article!

~~~
potash
Note that your code still forces python to read the entire file! If all you
want is the last line of a file, use the UNIX tail:

    
    
      tail -n 1 $f
    

Tail seeks backwards so it will only read the one line. Of course, this won't
give you a line count.

EDIT: I haven't tested it but you might be interested in this implementation
of tail in python:
[http://stackoverflow.com/a/136368](http://stackoverflow.com/a/136368)

~~~
acadien
I've done timing tests and it is typically on par with tail & wc. The bulk of
the time is wasted reading your file into ram, the time it takes to count the
lines is essentially 0.

Edit: Of course I misspoke, yes tail is much faster for getting the last line
of the file! I meant for getting the line count the loop methods is typically
just ~5% slower than wc on sufficiently large files.

------
nhaehnle
Somewhat tangentially related, the for-else-statement of Python is something
that I occasionally miss in every other language that I seriously use.

And sometimes I really wish that the scope of variables inside a do-while-loop
in C-ish languages would extend into the loop condition of the final while...

------
JeremyBanks
The Python 2 list comprehension behaviour could be used to assign local
variables within lambda expressions. Disgusting abuse, but fun:
[http://stackoverflow.com/a/14617232/1114](http://stackoverflow.com/a/14617232/1114)

------
leoh
I found this example most surprising:

    
    
        >>> def foo():
        ...         lst = []
        ...         for i in range(4):
        ...             lst.append(lambda: i)
        ...         print([f() for f in lst])
        ...
        >>> foo()
        [3, 3, 3, 3]
    

...So I looked into this a bit more.

It appears that the lambda creates a closure for i, but i is defined in the
outer scope and we capture the reference for i, rather than i's value.

So we can get around this:

    
    
        >>> def foo():
        ...     lst = []
        ...     for i in range(4):
        ...         lst.append((lambda a: lambda: a)(i))
        ...     print([f() for f in lst])
        ...
        >>> foo()
        [0, 1, 2, 3]
    

It appears that i is dereferenced when passed to the first lambda constructor,
which provides i's dereferenced value as a in a scope provided for the inner
lambda. Please correct me if I am wrong, anyone?!

\--------Edit 2--------

Out of curiosity, I looked at this in Clojure and the way it creates lambdas
is similar to Python (captures reference as opposed to value).

    
    
        user=> (def a 1)
        #'user/a
        user=> (def f (fn [] a))
        #'user/f
        user=> (f)
        1
        user=> (def a 2)
        #'user/a
        user=> (f)
        2
    

You could do something like this:

    
    
        user=> (def f ((fn [x] (fn [] x)) a))
        #'user/f
        user=> (def a 1)
        #'user/a
        user=> (def f ((fn [x] (fn [] x)) a))
        #'user/f
        user=> (f)
        1
        user=> (def a 10)
        #'user/a
        user=> (f)
        1
    

But perhaps there is another way as well.

~~~
rhizome31
You can capture the index with a keyword argument default value:

    
    
            def foo():
                lst = []
                for i in range(4):
                    lst.append(lambda i=i: i)
                print([f() for f in lst])
    

FWIW I played around with this issue in various languages in this blog post:
[https://my.smeuh.org/al/blog/lost-in-
scope](https://my.smeuh.org/al/blog/lost-in-scope) Once you get the difference
between block scope and function scope, it's quite easy to see what's
happening.

------
cnvogel
Probably this stems from the typical C-way of error-reporting out of a for-
loop, which is to terminate earlier than the "regular loop condition", and
hence a second test for the loop-condition can easily be used to check for
abnormal termination...

    
    
        int i;
        for (i=0; i<length; i++) {
            if (is_broken(element[i]))
                break;
            process(element[i]);
        }
        /* if everything went fine, i == length, else we broke out early */
        if (i < length) {
            fprintf(stderr,"Something was broken.");
            return -1;
        }
    

Whereas the Python version obviously doesn't make sense in a pythonesque
program iterating over sets of elements.

    
    
        for i in [1,2,3] :
            do_something(i)
        # ..now i is 3, not 4... obviously

