
Python's Mutable Default Problem - tswicegood
http://blog.objectmentor.com/articles/2008/05/22/pythons-mutable-default-problem
======
sophacles
I'm not sure this constitutes any problem other than a lack of understanding
of the python runtime. What the author describes as:

 _"the mutable default parameter quirk is an ugly corner worth avoiding"_

could also be described as:

 _"a natural outcropping of python's late binding, "names are references"
variable model, and closure mechanisms, which provide a consistency to the
language that is often crufted up in others"_

I do somewhat agree with the author that this particular functionality should
be a "use only when needed" feature. I don't think it should be avoided at all
costs tho, because there are times where the mutable default allows for a lot
of saved code. In fact in a few cases the code to work around using mutable
defaults can get into some serious voodoo because frequently the writer is
actually trying to work around the bigger mutable/immutable objects and names
are references "issues" in python.

This also reminds me of something I was reading on the front page today about
the old 'use the whole language' vs 'simplicity is king' holy war.

~~~
sjs
Precisely because Python has late binding you would expect the parameters to
be evaluated on each call to the function.

One thing Python lacks is the ability to use preceding arguments in defaults,
e.g. you cannot do this:

    
    
        def f(a=3, b=a+1):
            return (a + b) / 2
        
        NameError: name 'a' is not defined
    

Oops.

~~~
crystalis
That hides default logic in the signature. Why would you favor that over

    
    
      def f(a=3, b=None):
        b = b or a+1 # or use a more explicit version
        return (a + b) / 2

~~~
sjs
Because that can be said more succinctly. It is even easier to read as there
is less to read, which I realize is mostly subjective.

~~~
crystalis
I'm pretty sure this is the same argument used to make Perl a bad guy.

~~~
sjs
I disagree, but we've gotten a bit off topic.

I think Python's behaviour is confusing and basically never what anyone
actually wants. Regardless of whether or not you can use other params in
defaults, the defaults should be evaluated on each call.

------
aston
As noted in the comments, DON'T use

    
    
      stuff = stuff or []
    

because if you pass an empty list, you'll get a new one rather than mutating
the one you passed.

    
    
      stuff = stuff if stuff is not None else []
    

is wordy, but at least it's correct.

~~~
dilap
It's sad that such a common and concise idiom as "x or y" is so perniciously,
subtly broken in Python, and that there is no satisfyingly concise equivalent.

If I were being cavalier and had an extra wish to burn, I'd request

    
    
      x else y
    

to mean x unless x is None.

~~~
eru
Make yourself a function.

~~~
dilap
Not so helpful in a strict language (since you want y to be evaluated only
when x is None)

~~~
eru
Yes, that's true. You would need to wrap it in a lambda, but then that's looks
horrible and you might as well use an if.

------
perlgeek
FWIW, Perl 6 implicitly treats default values as closures, and calls them when
no argument is passed that could bind to the optional argument.

That way you get a fresh array each time, and you can even use defaults that
depend on previous arguments:

    
    
        sub integrate(&integrand, $from, $to, $step = ($to - $from) / 100) { ... }

------
emehrkay
Pylint (or maybe it was pep8) has told me not to make dicts default arguments
when running it against my code, but didn't explain why. Thanks for the post.

------
riobard
You just need to fully understand when Python does evaluation.

    
    
        import types
        def function(item, stuff = lambda: []):
            if type(stuff) == types.FunctionType:
                stuff = stuff()
            stuff.append(item)
            print stuff
    
        function(1)
        # prints '[1]'
    
        function(2)
        # prints '[2]'
    
    

In Scala it has a better syntax because of typed function object:

    
    
        trait Map[A, B] {
            …
            def getOrElse (key: A, default: ⇒ B): B
            …
        }
    
    

the `default` parameter is a function, so when you do

    
    
        getOrElse(someKey, defaultValue)
    

The `defaultValue` becomes a function that generates the value you put there
when called.

------
neutronicus
"stuff = stuff or []"

This idiom is baked into the perl community. It's kind of funny to see a
Pythonista deciding it's a good idea. (You can tell it hurts him too).

~~~
jacobolus
It’s not a good idea in Python. There are legitimate reasons to pass falsy
objects as function arguments.

~~~
robinhouston
Funnily enough, Perl (since version 5.10) actually has a solution to this. The
expression

    
    
      $foo // "default"
    

evaluates to "default" if $foo is undefined, and to $foo otherwise (even if it
is defined but false).

------
spenrose
Yes, it's a wart. You learn the patterns he mentions pretty quickly.

~~~
code_duck
I knew the pattern well from other languages, but would have assumed that it
wasn't necessary in Python. So, this is good to note.

------
mark-r
I keep seeing this "problem" come up, but I don't understand how it's
realistic. If you have a function that modifies a parameter as a side effect,
why would you have a default value for the parameter?

And since the site's comments seem to be taken over by link spam, is this
mention on Hacker News just a clever way to juice the Google rank of said
spam?

~~~
ylem
I saw it bite some people where I work. It's now used for discussion with
potential hires.

------
clay
"stuff = stuff or []"

This would fail to have expected behaviour here:

fill_list = []

stuff = function(info, full_list)

print fill_list

use the if list is None: paradigm

------
dustingram
This 'problem' can actually come in handy when used with a regex callback
function.

See if you can determine what this does:

    
    
        def cbk(match, nb = [0] ):
            if len(match.group())==len(nb):
                nb[-1] += 1
            elif  len(match.group())>len(nb):
                nb.append(1)
            else:
                nb[:] = nb[0:len(match.group())]
                nb[-1] += 1
            return match.group()+' '+('.'.join(map(str,nb)))
        
        str = re.compile('^(#+)',re.MULTILINE).sub(cbk,str)

~~~
d0mine
The code converts:

    
    
      ##
      #
    

To:

    
    
      ## 0.1
      # 1
    

_str_ is builtin, don't use it as a variable name especially if you use it in
its original role.

~~~
dustingram
Correct, & thanks for the tip! But don't worry, I changed the variable name
from my copy & paste and just didn't give it a second thought.

------
billmcneale
The problem is not so much about mutable, it's that default parameters escape
the scope of their method.

Which is very, very messed up (but not the first thing Python messed up).

------
wyuenho
It's on SO's Python FAQ too. This is the only thing in Python that's really
bitten me. I remember it took me like a week to figure this out when I was
tearing down my algorithm bit by bit to find out whether my prove was wrong or
the code.

[http://stackoverflow.com/questions/1132941/least-
astonishmen...](http://stackoverflow.com/questions/1132941/least-astonishment-
in-python-the-mutable-default-argument)

------
pedro3005
You can use this trick for memoization. Example:
<http://paste.pocoo.org/show/341849/>

~~~
code_duck
The article is right, though, in stating that later viewers of your code may
not be expecting this behavior, and it culd lead to problems.

------
viraptor
Not sure if that's a "problem" if the only other way to implement function-
static variables would be to add a variable visible for the whole module, or
tricks with decorators...

    
    
        @statics(blah=[])
        def foo(normal_args, **kwargs):
        # or
        def foo(normal_args, blah):
    

It messes up the idea of looking at the definition to find the function
signature.

------
baq
it's a gotcha but it makes perfect sense once you understand why it works that
way - i.e. the difference between evaluating a function definition and calling
it.

------
jfm3
The Pythonist doth protest too much, methinks.

------
drstrangevibes
i think its clearer and more pythonic in this case to do

def function(item, stuff): .... blah blah def function(item): function(item,
[])

