
C# to take a breaking change in version 5 - mwsherman
http://stackoverflow.com/questions/8898925/is-there-a-reason-for-cs-reuse-of-the-variable-in-a-foreach
======
ot
Very interesting, I've been bitten by the same problem with Python once
(capturing the iteration variable in a closure). To make things even worse, in
Python the behaviour is inconsistent between list comprehensions and generator
comprehensions:

    
    
      In [10]: lambdas = [(lambda: i) for i in xrange(10)]
      In [11]: [f() for f in lambdas]
      Out[11]: [9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
      
      In [12]: lambdas_gen = ((lambda: i) for i in xrange(10))
      In [13]: [f() for f in lambdas_gen]
      Out[13]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    

The problem was not fixed even in Python 3. I don't know if there has been any
discussion about it.

~~~
lloeki
This is because the following is simply _not_ a closure in Python:

    
    
        lambdas = [(lambda: i) for i in xrange(10)]
    

This works, because it is a proper closure:

    
    
        def build_closure(i):
            return (lambda: i)
    
        lambdas = [build_closure(i) for i in range(10)]
        print("%s" % [f() for f in lambdas])
    

See [http://stackoverflow.com/questions/233673/lexical-
closures-i...](http://stackoverflow.com/questions/233673/lexical-closures-in-
python)

You can write something close only with _lambda_ this way:

    
    
        lambdas = [(lambda j=i: j) for i in range(10)]
    

The behavior is actually consistent between generators and list
comprehensions, but it's giving the same result as a closure because
generators are lazily evaluated (so f() is evaluated right on time, but i is
still _not_ enclosed), and work only once: running it twice will have the
generator exhausted:

    
    
      In [12]: lambdas_gen = ((lambda: i) for i in xrange(10))
      In [13]: [f() for f in lambdas_gen]
      Out[13]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
      In [14]: [f() for f in lambdas_gen]
      Out[14]: []
    

That's why forcing the generation with list() causes evaluation of the
generator, and only then do you evaluate the f()s, hence the same result as
the list comprehension case.

Again, changing it to the following properly encloses i:

    
    
        lambdas_genlist = list((lambda j=i: j) for i in range(10))

~~~
wisty
> This is because the following is simply not a closure in Python

Well, they should be closures, or they shouldn't be there.

This isn't a question of not understanding Python's semantic rules, it's a
question of those rules being screwed. I understand why it's not consistent
with generators (as you say - i isn't generated yet). I don't understand why
it's not consistent with what you'd expect, namely lambdas not being closures.

It's an even weirder gotcha than:

    
    
        def f(x = []):
            x.append(1)
            return x
    

and we know how many people get hit with that one ;)

~~~
lloeki
Actually, you know, I lied (for the sake of simplicity). They are closures,
else how would the lambda evaluate 'i'? The difference is in the binding.

How closures work depend wildly on the language. With lexical closures it all
comes down to how scopes are handled [0] and how and when variable binding is
done [1] (notably §8). The fact that 'i' can be either bound late (giving the
'outer scope' effect) or bound early (giving the 'inner scope closure' you
expect) is actually a quite useful feature (and I assure you both cases are
equally useful), although admittedly a bit surprising when coming from other
languages.

Default argument value evaluation is a nice gotcha, but it's a trade-off I'm
more than willing to accept [2].

Anyway I would definitely not qualify this as 'screwed'.

[0] <http://stackoverflow.com/a/292502/368409>

[1] <http://docs.python.org/reference/executionmodel.html>

[2] <http://stackoverflow.com/a/1651284/368409>

------
MartinCron
It is encouraging to see the C# team learn from real-world scenarios and
improve the language.

~~~
tlb
It is discouraging to see them make the same mistake that early Lisps did, the
solution to which was well understood in the Scheme community since 1975.

~~~
solutionyogi
Well, they are human after all.

Overall, I feel that C# team has definitely achieved their goal of being a
'Pit of Success'. [http://www.codinghorror.com/blog/2007/08/falling-into-the-
pi...](http://www.codinghorror.com/blog/2007/08/falling-into-the-pit-of-
success.html)

E.g. Order of evaluation is strictly defined as left to right in C#.

[http://blogs.msdn.com/b/ericlippert/archive/2008/05/23/prece...](http://blogs.msdn.com/b/ericlippert/archive/2008/05/23/precedence-
vs-associativity-vs-order.aspx)

Disallowing use of uninitialized local variable.

[http://stackoverflow.com/questions/7797424/why-arent-
unassig...](http://stackoverflow.com/questions/7797424/why-arent-unassigned-
local-variables-automatically-initialized)

C# warns you if you do

if (i = 1) //notice the '=' instead of '==' {

}

warning CS0665: Assignment in conditional expression is always constant; did
you mean to use == instead of = ?

Obviously, they failed in the case of 'foreach' but I feel great knowing that
they will be fixing it in the next release.

------
pixie_
It's really only a breaking change if you designed your program to have the
closure from inside the loop purposely use the last element of the collection
outside the loop. Who would do this? I've hit this issue lots of times as have
others, and it's a bug that is fixed by copying the iterator inside the loop.
It's nice I won't need to worry about that anymore.

~~~
bunderbunder
The bigger issue will be the confusion when people who are used to the C# 5
semantics work on a C# 4 project.

~~~
noblethrasher
Yep, that's why it's a breaking change (or why breaking changes are bad). On
the other hand, if they're using the C# 5 compiler to output C# 4 projects
then maybe there will be a new warning (which, of course, is another breaking
change if you treat warnings as errors).

------
hetman
I've mentioned this elsewhere but I think the following needs to be
emphasised: "The "for" loop will not be changed."[1]

I would argue it's a terrible idea to break consistency between how this is
handled by the for and the foreach loop. The for loop can't be changed because
the initialiser expression is general and not restricted to variables. I
expect the result will be many bemused developers surprised closures behave
one way in a for and another way in a foreach.

The end result is that the problem hasn't really been eliminated, just made
even more obscure. My personal preference would be for consistency; one can
still achieve the desired effect simply by explicitly defining a variable
inside the loop scope for the closure to close over.

[1]
[http://blogs.msdn.com/b/ericlippert/archive/2009/11/12/closi...](http://blogs.msdn.com/b/ericlippert/archive/2009/11/12/closing-
over-the-loop-variable-considered-harmful.aspx)

~~~
bunderbunder
Closures behave the same way in either case. The closure semantics in C#5 will
be unchanged.

The difference is that the for loop's variable is scoped to the entire loop,
whereas the foreach loop's variable is only scoped to the embedded statement
(i.e., an iteration through the loop). That difference is perfectly
justifiable.

In the semantics of a for loop, the loop variable is very clearly (and
necessarily) a persistent value which can be changed according to rules
described in the loop statement, by code within its embedded statement, or
both.

In the semantics of a foreach loop, on the other hand, the implication is that
you're successively retrieving otherwise independent values from a collection
and operating on each one in succession. There is no reason to need or expect
that the variable will have a scope that extends outside the embedded
statement. It is simply replaced on each iteration through the loop. The loop
statement doesn't offer any opportunity to perform any logic on the variable
in between iterations, since all you can do between the parentheses is bind a
variable. You cannot use a varaible from the outer scope as the loop variable,
and the loop variable goes out of scope as soon as the loop exits.

In fact, there is really only one place where this scoping difference is
visible: When the loop contains a closure. In that case, the semantics that
C#5 will use are indisputably superior. Even in cases where that sort of
behavior is desired (which is exceedingly rare), it is much better to require
that the behavior be accomplished by using a variable from the outer scope,
for the sake of readability. Being a language in which it is easy to write
obfuscated code was never high in C#'s priority list.

If the cost of doing it this way is that there is a(nother) difference between
the semantics of the for-loop and the foreach-loop, I have no problem with
that. I can see no demonstrable need for consistency there; it is a foolish
consistency.

~~~
hetman
I never implied closure behaviour would change, only the variable scoping is
inconsistent. I agree it makes sense, but it's still going to cause surprises.

