
A collection of not so obvious Python stuff - adamnemecek
http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/not_so_obvious_python_stuff.ipynb
======
andolanra
I'm being a bit pedantic, but the last snippet under _About lambda and
closures-in-a-loop pitfall_ is arguably misleading when it says, "This,
however, does not apply to generators…" Which is true in the example given,
but not true _in general_. Generators don't actually provide a new environment
on each evaluation, they just lazily construct their contents, which assuages
the problem in certain situations. This snippet

    
    
        for l in (lambda: n for n in range(3)):
            print(l()) # prints 0, 1, 2
    

will alternately request an argument from the generator, return a lambda, call
it, and _only then_ request the next argument from the generator, which means
at the time of the funtion call, _n_ will not have been incremented up to its
final value. We can see the previous closures-in-a-loop behavior return if we
keep the closures around to execute again, at which point the loop environment
contains the final value of _n_ :

    
    
        funcs = []
        for x in (lambda: n for n in range(3)):
            funcs.append(x)
            print(x()) # prints 0, 1, 2
        for y in funcs:
            print(y()) # prints 2, 2, 2
    

So generators don't 'fix' the problem by constructing distinct environments
for each loop iteration, but by deferring evaluation of their body until
necessary.

~~~
MereInterest
Yeah, that part stood out to me as well. The only way I've found to get
different values for each of the functions is to use a different closure for
each functions.

    
    
        funcs = [(lambda i: (lambda :i))(n) for n in range(3)]
        for f in funcs:
            print(f())

------
acjohnson55
What I love about Python is that these hangups are mostly relatively obscure,
due to the puritanical adherence to principle of least surprise by Guido and
the other language maintainers. No language is completely without quirks
though, especially one as old as Python.

------
jsmeaton
This bit bothered me slightly:

>But what if we put a mutable object into the immutable tuple? Well,
modification works, but we also get a TypeError at the same time.

    
    
        tup = ([],)
        tup[0] += [1]
    

Of course that's an error. It's equivalent to:

    
    
        temp = tup[0]
        temp += [1]
        tup[0] = temp
    

But the given explanation is way too obtuse. This shouldn't be a mystery at
all:

> If we try to extend the list via += "then the statement executes
> STORE_SUBSCR, which calls the C function PyObject_SetItem, which checks if
> the object supports item assignment. In our case the object is a tuple, so
> PyObject_SetItem throws the TypeError. Mystery solved."

~~~
rix0r
It's not ENTIRELY obvious. For example, tup[0].extend([1]) would work fine,
and the behaviour of __iadd__() seems like it should be the same as extend().
In fact, the reference[1] has the following thing to say:

> For instance, to execute the statement x += y, where x is an instance of a
> class that has an __iadd__() method, x.__iadd__(y) is called.

However, this is patently not true, apparently:

    
    
        tup = ([],)
        tup[0].__iadd__([1])  # Works
        print tup
    
        tup = ([],)
        tup[0] += [1]  # TypeError
        print tup
    
    

[1]
[https://docs.python.org/2/reference/datamodel.html#emulating...](https://docs.python.org/2/reference/datamodel.html#emulating-
numeric-types)

------
zwegner
The "consuming generator" problem is a good point, but their workaround misses
a better solution. In Python 3, range() returns a range object, which has a
__contains__ method, so you can use "in" just fine without consuming any
generators. It's also much faster (it would be constant time/memory except for
the fact that integers can have arbitrary size).

    
    
        l = range(5)
        print('2 in l,', 2 in l)
        print('3 in l,', 3 in l)
        print('1 in l,', 1 in l)
    

prints...

    
    
        2 in l, True
        3 in l, True
        1 in l, True
    

Also, the lambda/closure example is something that really irritates me about
Python. Both dynamic and lexical scoping have this problem: they always
reevaluate the variable inside the closure whenever it's called, the only
difference is which parent scope the value is pulled from (the parent scope or
the calling scope). It's not that the "last lambda is being reused", as the
article says, but rather, there are five different lambdas, but they each
refer to the same variable, which is set to 4 after the list comprehension.
This sucks for metaprogramming, where you'd like to create a bunch of copies
of a function with different parameterizations. There's two workarounds that I
know of for this for dynamic scoping: creating another nested scope, or using
keyword arguments. I'm not sure what you'd do for lexical scoping (but who
uses that anyways?).

    
    
        for x in [(lambda i: lambda: i)(i) for i in range(5)]:
            print(x())
    

...or...

    
    
        for x in [lambda i=i: i for i in range(5)]:
            print(x())
    

In my purely-functional Python-like programming language, Mutagen, closures
are _static_ , so that they capture the value of variables from the enclosing
scope at the point that they're defined. So this works as you'd expect,
printing 0 and 1 (no list comprehensions yet, and lambdas are full functions,
and of course lists are immutable so no +=):

    
    
        lambdas = []
        for x in [0, 1]:
            lambdas = lambdas + [lambda: return x;]
        for l in lambdas:
            print(l())

~~~
emidln
The first workaround is actually the same as what you do for a map in 2.x
(that always works):

    
    
        for x in map(lambda i: lambda: i, range(5)):
            print(x())

------
d64f396930663ee
Even after reading this list, I'm comfortable forgetting everything (with
maybe one exception) because these surprises really only come up when you use
horrible coding practices. But it's nice to know it really takes some serious
effort to find these kinds of things in Python.

------
allendoerfer
Midnight is not False in Python 3.5 anymore:

[https://docs.python.org/3.5/whatsnew/3.5.html](https://docs.python.org/3.5/whatsnew/3.5.html)

------
emehrkay
I somehow didn't know about nonlocal. With that, isn't python 100% comparable
with JavaScript?

~~~
Grue3
Python didn't even have a nonlocal statement until 2.7. Making closures was
(and still is, since I have to work with 2.6) a total nightmare.

~~~
random_coder
nonlocal is only in python 3.

~~~
Grue3
Oh, that's even worse.

