
I Challenge You to Debug These 7 Lines of Code Under 9 Minutes - Oxitendwe
http://www.theodo.fr/blog/2017/02/i-challenge-you-to-debug-those-7-lines-of-code-under-9-minutes/
======
wdroz
Experienced in Python? This will take you less than one minute.

~~~
zaptheimpaler
The first line tipped me off :p

    
    
         def foo(a, methods=[]): # methods defaults to [] if not specified
         ohno.

~~~
tomlock
I got that one, but the rest confused me - I need to work with lambdas more!

------
Oxitendwe
It's mystifying to me that the author comes to the conclusion that the
solution to this is to educate people on debugging tools and edge-case
semantics. The reason the code is difficult to debug is because the "bug" is
that functions are apparently completely broken and violate every reasonable
assumption anybody is likely to make about them without a priori knowledge
about this specific edge case. Why isn't the conclusion then, that Python is
broken, and has a serious and fundamental problem? Why doesn't anybody see
this as a "broken window", that leads us as a culture to believe that this
sort of thing is acceptable and to be expected and not an ugly, horrible, and
utterly unnecessary problem? Is this really the language we're supposed to
teach to beginners? How many combined hours have been wasted on solving
problems generated by this behavior, and for what purpose? I don't understand
why people accept this.

------
dahauns
It took me a minute to see the problems, but - not having touched Python for
over a decade - I can't for the life of me bring the lambda to capture the
value of the variable.

I know this doesn't work (because python assignment rules)

    
    
        def foo(a, methods=[]): 
        m=[];
        for bla in methods:
            m.append(bla)
    
        for i in range(3): 
            k = i
            m.append(lambda x: x + k)
    
    
    

But this should work, according to quick'n'dirty google (because default
assignment), and yet it doesn't.

    
    
        for i in range(3): 
            def bar(c, d=i): return c+d
            m.append(lambda x: bar(x))
    
    

EDIT: Ok, I'm something missing here.

This doesn't capture the value of i

    
    
        for i in range(3): 
            m.append(lambda x,i=i: x+i)
    

But this does. WTH?!

    
    
        for i in range(3): 
            m.append(lambda i=i: i)

~~~
flavianh
That's an interesting try! I think the default assignment is done by reference
as well, so when i changes in the loop the default value changes as well?
Similarly to methods=[]. Can't say for sure though. The solution I give in the
article is the only one I could come up with, I'm interested if you find
another one :)

~~~
dahauns
Ah - the answer was so simple: I'm an idiot.

    
    
        for i in range(3): 
            m.append(lambda x,i=i: x+i)
    

actually works, I just had a brain fart. :)

------
sixhobbits
Great intro to PDB, but I have to be critical of: "The second group took 9
minutes on average, and the first one 14 minutes! Those are not completely
statistically significant, but that’s still a 35% speedup on an 8-line-long
program."

If you want to talk about statistical significance, don't say "not
completely". It either is or it isn't. If you think the p-value is important
(and I'm not saying it is, but don't bother mentioning it if you don't think
it is), then you can't claim a 35% speedup. If you're reporting results in the
context of significance, and you decided that your results were not
significant, then the 35% speedup is meaningless.

~~~
flavianh
Article author here. I agree with your point, I shouldn't have been so
cautious. I consistently observed people being faster when using debuggers and
I wanted to give it a test on this problem. Most people don't see that they
can get a significant speedup in using debuggers, so that sentence wanted to
give a sense on how much quicker you can be. I'd be happy to hear of a better
way to convince a non-debugger user that he could have debugged this code
faster with a debugger!

~~~
sixhobbits
Well I was convinced :) If it were me, I'd probably just remove the mention of
significance -- or if you have more people and some time you can probably set
up a proper experiment, control for some confounds and you'll probably get
significant results.

------
bad_login
"If I lied, you are allowed to insult me abundantly through your favorite
channel."

Haha racket can here is how.

    
    
        #lang racket
        
        (define (foo a [methods empty])
          (let ([methods
                 (append methods
                         (for/list ([i (range 3)])
                           (lambda (x) (+ x i))))])
            (for/sum ([m methods])
              (m a))))
        
        (foo 0)  ;; Should be 3
        (foo 1)  ;; Should be 6
        (foo 2)  ;; Should be 9
        (foo 2 (list (lambda (x) x)))  ;; Should be 11
        (foo 0)
        (foo 1)
        (foo 2)
    
    
        ;; Now the wrong way
    
        ;; The lambda capturing wrong
        (define (foo a [methods empty])
          (define i 0)
          (let ([methods
                 (append methods
                         (for/list ([z (range 3)])
                           ;; modifying the variable i
                           (set! i z) 
                           (lambda (x) (+ x i))))])
            (for/sum ([m methods])
              (m a))))
    
        ;; this one doesn't do the default parameter wrong, because
        ;; methods and methods-default share an imutable list and we
        ;; modify only methods so beside "empty" they won't share the same
        ;; reference anymore, to get it wrong we should use a mutable list
        ;; (doesn't exist in racket).
        (define foo
          (let ([methods-default empty])
            (lambda (a [methods methods-default])
              ;; (printf "~a ~a\n" (length methods) (length methods-default))
              (define i 0)
              (for ([z (range 3)])
                (set! i z)
                (set! methods (cons (lambda (x) (+ x i))
                                            methods)))
              (for/sum ([m (append methods methods-default)])
              (m a)))))
    
        ;; So i use a box to make it mutable.
        (define foo
          (let ([methods-default (box empty)])
            (lambda (a [methods methods-default])
              ;; (printf "~a ~a\n" (length (unbox methods)) (length (unbox methods-default)))
              (define i 0)
              (for ([z (range 3)])
                (set! i z)
                ;; methods and methods-default share the same reference of box.
                (set-box! methods (cons (lambda (x) (+ x i))
                                        (unbox methods))))
              (for/sum ([m (unbox methods)])
                (m a)))))
    
    

The moral of the story, language design matter, immutable by default is good,
lexical scoping is good, binding over variable is good, ruby, python, php and
javascript (the languages the author mention) are bad in that regards.

One thing php, ruby, python (but not javascript) has good over racket their
identatation tend to be flat and racket (let form, no return statement) tend
to go all the way to the right.

~~~
flavianh
I'm glad I haven't been insulted yet

------
Johnny_Brahms
Is this a designed feature or something that just happened? I really think
that this behaviour is erroneous and unexpected.

With python (and many other languages) I find myself getting bitten by
behaviour like this every now and then. I use scheme for almost all my
personal projects, and I can only remember getting bitten twice. Once was
unexpected, and the other was laziness on my part.

~~~
lalaithion
Basically it just boils down to reference vs value. Both of the errors in this
code occur because python uses a reference instead of a value – I agree that
they can be annoying and somewhat unexpected, but they make sense once you
realize what's behind them.

In the method=[] part, the error occurs because python creates a single list,
and passes that list by reference (as all lists are passed) into the function.

In the lambda part, the error occurs because python captures the variable from
the enclosing scope by reference, not by value (as normal functions do with
globals).

~~~
Matthias247
I understand the "capturing i in a closure by reference" part, since that
error is also common in other languages which capture variables by reference,
and don't create a new reference per loop iteration.

However the default parameter issue looks like a python specific issue which
is really strangely designed. I would normally say [] is a literal for an
empty list which creates a new instance of an empty list every time it's used.
So when it's used as a default parameter a fresh empty list should be created
all times.

Is [] is global reference to a single mutable list which is only initially
empty? That would make it pretty useless, since everyone could mutate it. If
it's a single reference then that should at least be an immutable list which
throws some exception when someone messes around with it.

~~~
zaptheimpaler
>Is [] is global reference to a single mutable list which is only initially
empty?

No, the problem is default arguments to methods are evaluated once, when the
method is created and that value is used for all calls. I don't know why that
is, its definitely strange - any other language would evaluate the default
args each time a function is called just like the regular args.

~~~
jVint
Then you would have the problem that something like mydefault=3, def
f(default=mydefault):... would change it's evaluation if anyone ever defined
mydefault=4. That would leave users unable to define any variable thats ever
mentioned in default values of any function in any library they are using
without causing very unexpected behaviors.

What languages are you thinking of where default values set to a references
are set to a copy of that reference as it was at definition time at every
evaluation?

~~~
zaptheimpaler
Not sure if i completely understand, but the example you mentioned would work
in Scala. If a mutable variable is referenced in default args, its value at
define-time is used.

Example:

    
    
        scala> var x = 5
        x: Int = 5
    
        scala> def func(arg1: Int = (x+1)) = { println(arg1)}
        func: (arg1: Int)Unit
    
        scala> func()
        6
    
        scala> var x = 7
        x: Int = 7
    
        scala> func()
        6 //unchanged
    

For reference vars the closure captures a local copy:

    
    
        scala> var list = List(1,2,3)
        list: List[Int] = List(1, 2, 3)
    
        scala> def printList(arg1: List[Int] = lis) = { println(arg1) }
        printList: (arg1: List[Int])Unit
    
        scala> printList()
        List(1, 2, 3)
    
        scala> var list = List(4,5,6)
        list: List[Int] = List(4, 5, 6)
    
        scala> printList()
        List(1, 2, 3)

------
pmarreck
The fact that "methods" is somehow kept defined in the scope of the function
_and preserved when you call it again_ and isn't always reinitialized to [] is
just dumb. No wonder this was a bug, who the hell would see that and how is
this "feature" ever useful except to create bugs?

~~~
Oxitendwe
It's absolutely insane that this is apparently the expected behavior. It makes
no sense whatsoever except to give people who know this an ego boost when they
can successfully work around Python's unintuitive semantics while
inexperienced people cannot. It completely breaks functions, the most
fundamental unit of encapsulation and abstraction in every modern programming
language.

~~~
pmarreck
It's surprising how few languages actually support anonymous functions _with
closures_ properly.

Not to plug my 2 favorites that do (Ruby and Elixir/Erlang) or anything...

------
stephenbez
It's a common JavaScript interview question to ask about capturing variables
in a closure.

What really frustrates me with go is that it doesn't have a mature debugger so
I have to resort to inserting print statements everywhere.

~~~
zamalek
So long as you are not using cgo, the VSCode debugger (which uses Delve as the
backend) is fantastic. cgo breaks everything[1].

[1]:
[https://github.com/golang/go/issues/10776](https://github.com/golang/go/issues/10776)

------
forcemajeure
Worth bearing in mind that if you develop and ship software to third parties,
you're going to want a lot of debug statements in the logs, so don't omit them
completely!

------
rasta_iprun
Took me less than 7 minutes with a single print statement. I'm a FE developer
though, and python feels awfully similar to JavaScript.

~~~
flavianh
Just one? I guess you found out about the methods default value this way, did
you find the other one by eye?

------
mindfulplay
This is a common mistake in pretty much any language that captures a closure
inside a lambda: C++, JS, etc. Unfortunately, debugging this the first time
takes 20 minutes; the next time, it takes 0 seconds.

~~~
kbwt
> common mistake in C++

Not really. You have to explicitly specify you want capture-by-reference with
C++ lambdas.

The code would also look very out-of-place to a C++ programmer, because you
would never capture a loop counter by reference if you intend to use the
lambda outside of the loop. Lifetimes don't just get magically extended into
completely unrelated scopes.

