
Python has closures - samratjp
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/67666
======
lispm
Unfortunately in the example there is not a single closure shown. A closure is
a function and its bindings. But there are no function objects created.

What he shows is some attempt to describe some kind of lexical binding. But
even that looks very suspicious.

    
    
        def foo():
             global y
             x = 0
             def bar():
                 print x, y
             bar()
             x = y = 1
             bar()
             x = y = 2
             bar()
             x = y = 3
             bar()
             x = y = 4
             bar()
             x = y = 5
    
        foo()
    
        0 0
        1 1
        2 2
        3 3
        4 4
    

Above is really trivial. The problem is that the assignments change a variable
and the function BAR sees the changes. So there is only one variable binding
for X. In a variant where each assignments were really bindings, BAR would
only see the binding in its scope and the number would not change.

Above would be in Lisp:

    
    
        (defvar *y* 0)
    
        (defun foo ()
          (let ((x 0))
            (flet ((bar ()
                     (print (list x *y*))))
              (bar)
              (setf *y* 1 x *y*)
              (bar)
              (setf *y* 2 x *y*)
              (bar)
              (setf *y* 3 x *y*)
              (bar)
              (setf *y* 4 x *y*)
              (bar)
              (setf *y* 5 x *y*))))
    

Again, no closure. Just a function with a local function that gets called
several times.

He then describes that in a 'subfunction' a variable from an outside scope
can't be set, but that is almost a side issue. It also may be that this is not
often found in his code, but it is found in a lot of Lisp code that actually
uses closures.

The most trivial example is a counter closure:

    
    
        (let ((fn (let ((x 0))
                    (lambda () (setf x (+ x 1))))))
           (print (funcall fn))
           (print (funcall fn))
           (print (funcall fn)))
    

1 2 3

In above code FN gets bound to a closure. This closure is called three times.
Each time it updates the variable X in its lexical scope.

~~~
btilly
No, they absolutely are full closures. He just didn't demonstrate that fact to
your satisfaction. Here is a code example that may be more convincing to you.

    
    
      def make_counter(name):
          position = [0]
          def counter():
              position[0] += 1
              return (name, "incr", position[0])
          def show_counter():
              return (name, "show", position[0])
          return counter, show_counter
    
      foo, see_foo = make_counter("foo")
      bar, see_bar = make_counter("bar")
    
      for i in range(5):
          print foo()
          print see_foo()
    
      for i in range(5):
          print bar()
          print see_bar()
    
      for i in range(5):
          print foo()
          print foo()
          print see_foo()
          print bar()
          print see_bar()
    

If you run that you'll see that the returned closures retain lexical bindings
to the scope they were defined in. Closures generated at different times do
not interfere with each other. And closures generated in the same scope are
pointing at the same variables.

What more do you want?

~~~
lispm
Nothing. I never said that Python doesn't provide closures. I just pointed out
that Mr. Prescod did not give an example for those, even though he writes
about closures. He really was talking about lexical scope. He did not
demonstrate closures for my satisfaction, he didn't demonstrate them at all.

This makes me guess that Mr. Prescod never understood what a closure is, since
he failed to give any example. That one can set closed over variables in Lisp
is a useful feature. He demonstrates that Python 2 can't set a variable of an
enclosing scope and claims that he does not need that. Maybe if he were really
using closures, he would have found it useful.

~~~
btilly
But he _did_ give an example. In his example, bar is a function object that is
a closure. He just didn't make it do anything that would demonstrate it had to
be a closure.

As for Pascal, you need to look again. From the beginning Pascal had the
ability to pass functions and procedures as parameters to other functions and
procedures. If those functions and procedures are defined in an inner scope,
they will carry the original lexical scope, and are in fact full closures. OK,
they are clunky to use, but they work. Skip to the end of the section starting
at
[http://en.wikipedia.org/wiki/Comparison_of_Pascal_and_C#Func...](http://en.wikipedia.org/wiki/Comparison_of_Pascal_and_C#Functions.2Fprocedures)
to see the necessary syntax demonstrated.

Don't quote me on this, but I believe that Pascal was actually the very first
language with full closures. All of the necessary pieces are in ALGOL-68, but
I think that Pascal was implemented first and so wins.

Incidentally Sussman claims that he got the idea of lexical scope for Scheme
from studying ALGOL. Of course Scheme realized the possibilities of the
construct as no language before it had, and a lot of credit is due for that.
But you shouldn't lightly dismiss the contributions to Lisp from languages out
of the Lisp family.

~~~
lispm
No, bar is not a closure. It is basically just a local procedure.

The original Pascal did not have full closures. Functions did only exist as
long as their surrounding function was called. They could not be returned from
functions. Pascal lacked also the necessary Garbage Collection. Pascal had
lexical scope, but not full closures.

All you could do was pass functions to other functions. But that's trivial,
since each time one passed function X, it gets its active environment, which
always is on the stack. There was no way to have closures, where functions
have their own set of variable bindings, even when the surrounding function is
no longer active. Being able to pass a function downwards is not sufficient to
require full closures.

Basically the same applies for Algol. Functions can't be returned and are not
first class objects.

What Algol contributed to the evolution of Lisp was lexical scope, but not
closures. Also note that it was not Scheme that introduced closures in Lisp.

That's what I fear, many people think they know what a closure is, but in
reality there is a lot of confusion.

~~~
omaranto
> No, bar is not a closure. It is basically just a local procedure.

bar is a variable whose value is a closure. In python, defining a named
function just creates a variable whose value is a closure.

~~~
lispm
Python the language or Python the implementation?

~~~
omaranto
I do mean the main implementation, CPython. I guess an implementation is free
to avoid unnecessary closures and it would be nice to know if any others do.

------
andolanra
Python 3 already has the nonlocal keyword which does exactly what it sounds
like. It's explained in mind-numbing detail by PEP 3104 at
<http://www.python.org/dev/peps/pep-3104/> but it's really what you'd expect.

    
    
      def make_counter():
          n = 0
          def incr():
              nonlocal n
              temp, n = n, n+1
              return temp
          return incr

~~~
tkahn6
I think `nonlocal` is a prime example of a leaky abstraction. Can anyone
explain the advantages of using references for normally primitive variables?

~~~
kgo
There are no primitive variables in python. Even ints, strings, floats, etc
point to a reference of an object. All nonlocal does is point the variables
reference to a different object.

As to the advantages...

Well for one you can use the variables in closures...

------
supersillyus
I like the part where he notes that this isn't a real problem except that it
creates centithreads of argument from people who believe it to be a real
problem. As I grow older and less idealistic, I'm more sympathetic to the
notion that problems in theory but not in practice aren't really problems.

~~~
alec
I write "real code" in Common Lisp, and I used something like what he
dismissed as "useless in practice" twice today to make my code more readable
and maintainable. I do not consider myself a Lisp advocate when I bemoan the
lack of sane scoping semantics most similar languages just get right; I just
want to write better code.

~~~
jerf
Yes, but Lisp code and Python code aren't the same when program in the _native
idioms_.

In Python if your function references are buried within each other deeply
enough that this is even an issue you're _already_ doing it wrong. This is not
because Python is lacking power, but because you're not using the power Python
actually has. With the generators, comprehensions, objects that implement
__call_, bound method references, and all of the other tools available to you,
you just shouldn't be defining functions in functions in functions. In Haskell
where it's all functions everywhere, a function that is actually ten layer
nested functions calling each other makes sense. (When using the "do" syntax
sugar, every line that has <\- is another nested function, so if you see ten
<-s, you've got a ten-layer deep nested function. (Or eleven, if I've
committed a fencepost error.))

If you're still fighting with Python by trying to jam functions in three
layers deep, take a moment and try to learn to be more idiomatic. If you're in
Haskell or Lisp and you _never_ have functions three layers deep, take a
moment and try to be more idiomatic. In practice, you do not need this in
Python.

------
abstractbill
__But_ the addition would be useless in practice because this problem almost
never arises in real code. Its only purpose would be to answer Lisp and Ruby
advocates who want to say that Python doesn't really have closures._

A great example of the Blub paradox in action ;)

~~~
ubernostrum
Yeah, and Haskell programmers love having it pointed out that their language
makes them stupid by not allowing them to do this ;)

Oh wait. Nobody says that about Haskell programmers? They only say it about
Python? Huh. When they both have this same shortcoming with their so-called
"closures"? Wonder why that is?

~~~
chromatic
Anyone who's reached that point with Haskell is comfortable with the idea of
immutable state.

Anyone who's reached that point with Python is very comfortable with the idea
of mutable state... everywhere else.

~~~
ubernostrum
Which is funny, because in both cases they're symptoms of trade-offs made in
the language design.

But a Haskell programmer who understands this and decides to accept it is just
considered normal. A Python programmer who understands this and decides to
accept it (as in the linked post) is declared to be a Blub user, which is
essentially a polite-enough-for-HN way to say "LOL STOOPID NOOOB".

------
dfox
As I understand CPython, problem with this, is that python's "closure" does
not contain reference to parent environment frame, but it's copy, so this is
not only syntactic issue. Also putting reference to parent environment would
create reference cycle, which was clearly undesirable in times when python did
only reference counting.

~~~
stcredzero
One of the most powerful features of Smalltalk Blocks is the ability to
reference everything in the parent context. This gives one the ability to
Don't Repeat Yourself in even the stickiest of situations. A lot of tricks for
not repeating yourself involve passing a Block from the current context into a
function, where it is executed, possibly inside an exception handling block.
99% of everything you'd be hard pressed to avoid repeating yourself with can
be taken care of using tricks based on this idea.

EDIT: Come to think of it, this is just a good-enough substitute for a
fraction of the stuff you can do in Lisp with Macros.

~~~
jules
Smalltalk's blocks came from Lisp. It's called lambda in Lisp and is arguably
the most important idea in Lisp.

~~~
lispm
Right, and Smalltalk had for a long time no lexical binding, thus returned
blocks were never closures. Just as in early Lisp dialects.

------
m0th87
Instead of adding an "intermediate" keyword, I think JavaScript's approach is
preferable. That is,

> var x = 0

Creates a new variable.

> x = 0

Assigns a variable if it has already been declared in a parent scope, or
creates a new global variable otherwise.

Of course, this work break all existing Python code, so it would never happen.

~~~
orangecat
_or creates a new global variable otherwise_

That is the worst possible result. Requiring an explicit local or global
keyword is reasonable, defaulting to local is reasonable, defaulting to global
is insane.

~~~
m0th87
Good point. Though defaulting to any sort of variable declaration could be
problematic. You might be creating a new variable when you think you're
updating one. I like AndyKelley's crash-on-default proposal best. This would
also help the compiler in optimizing.

------
KirinDave
Does it actually reference the parent, or does it just make a copy? One is a
closure, the other is an implementation artifact.

~~~
gxti
It doesn't make a copy. It creates a special object (which you don't see
unless you're digging in the internals) called a "cell" which holds a
reference to the object for as long as the function exists. To see for
yourself, look at `somefunc.func_closure` -- it'll be `None` on regular
functions, and a tuple of cells on a closure function.

------
jimbokun
"I don't know whether Guido will ever add it for, er, closure, but I do know
that I have _never_ _once_ needed this keyword in hundreds of thousands of
lines of real Python code I have written."

This is the blubbiest non-ironic statement I can remember reading.

~~~
cdavid
Could you give examples of useful closures (actually multi-line lambda) ? I am
relatively proeficient in python, and am learning javascript, and I really
don't like how non trivial anonymous functions are pervasive in the later. I
understand it may just be habit, though....

OTOH, one nice aspect in say ruby is the combo "real" anonymous functions +
the syntax f a equivalent of f(a), which is what makes DSL possible in ruby
(and impractical in python in general). I think it comes at a price in
readability, though.

------
rlpb
I always wondered what closures actually were. I'd read up on them, but
apparently Python didn't have them so I always felt that I was missing
something.

Thanks for clearing this up.

~~~
Hemospectrum
A lot of people don't seem to understand the difference between closures and
lambdas, and a lot of people get it backwards as soon as someone points it out
to them. Good to know there's one clear demonstration out there.

Actually, if you could pass this on to people when you hear them refer to
lambdas as closures, that would be pretty awesome.

------
crizCraig
The crappy thing about Python closures is that you have to define the function
above where you call it. I think this kind of puts the cart before the horse
and messes of readability.

JavaScript, on the other hand, allows you to define the inner function
anywhere in the local scope and makes closures much more attractive, IMO.

Changing this may mean big changes to the interpreter and so may not be worth
it. I wonder if lambda created closures are any different...probably not.

~~~
Jach
You can of course define single line lambdas inline, and I think those are
quite attractive, but when you want to get multiline lambdas you get in quite
a sticky situation with Python. For a simple example, see the top answer here:
[http://stackoverflow.com/questions/1233448/no-multiline-
lamb...](http://stackoverflow.com/questions/1233448/no-multiline-lambda-in-
python-why-not)

In order to make it work Python would have to introduce a scope-blocking
structure like braces, parens, or an 'endlambda', all of which I think are
steps in the wrong direction, especially given Python's great feature of
semantic whitespace. I'll take a clean function defined above over braces.

I do know of a way to hack a multi-line Lambda with existing syntax, though,
but I have no guarantees about it. I don't think I've used it in serious code,
since it's just real ugly.

    
    
      k = lambda x: eval(compile("""
      print x
      y = 4
      print x + y
      x = 8
      print x
      """, '', 'exec'))
    
      x = 5
      k(2)
      # 2
      # 6
      # 8
      print x
      # 5

------
jpr
By the way, what is the idiomatic way to do this in Python?

    
    
        (let ((x 1))
           (defun less () (incf x))
           (defun more () (decf x)))

