Hacker News new | past | comments | ask | show | jobs | submit login
Stop writing lambda expressions in Python (treyhunner.com)
123 points by Quyzyx 9 months ago | hide | past | web | favorite | 194 comments



> I’d say that using lambda expressions is acceptable only if your situation meets all four of these criteria:

> 1. The operation you’re doing is trivial: the function doesn’t deserve a name

Sure.

> 2. Having a lambda expression makes your code more understandable than the function names you can think of

This is just vague. Locally defining a function right where you need it makes code much more understandable to me more often than the wacky names people often come up with. Naming things is hard!

> 3. You’re pretty sure there’s not already a function that does what you’re looking for

So the reader of the code has to know what that built-in function does. How is that much different than expecting the reader to know what a lambda is?

> 4. Everyone on your team understands lambda expressions and you’ve all agreed to use them

Or just use them and have the members of your team better themselves by learning a useful language feature.

---

In my experience, names lie--functions don't. If a developer isn't going to spend the time to write a nice comment, they aren't going to use a comprehensible naming convention. In trivial cases where naming is easy, the lambda is also usually very easy to parse.


I've noticed a pattern in among python developers where the faux pas of assigning a lambda is used to signify "I'm going to use this function in a vaguely functional programming manner". For example, by passing it into map, or filter, or sorted.

For this reason, I don't really mind it. It's a sort of cue to the reader about the style of programming that will follow.


I many of those cases people use lambda when a partial would be a better fit.


>4. Everyone on your team understands lambda expressions and you've all agreed to use them

This only might be valid if people on the team weren't programmers, but rather researchers that had to use python. In that case, the code would have to be very easy to understand. But in that case, they should probably learn what a lambda function is. It's really not that hard


Also, wouldn't the existence of a library function (#3) imply that what you're doing is either non-trivial or common enough that someone has already given the function a name? Seems like #1 is sufficient.


To give an example based on the article, I find it cumbersome to use itemgetter. I could use the library function:

  import operator
  sort(values, key=operator.itemgetter(2))
but it seems decidedly trivial and common to use a lambda:

  sort(values, key=lambda x: x[2])


The risk of identifier collisions with itemgetter seems really low. I've never had a problem with

  from operator import itemgetter
  sort(values, key=itemgetter(2))
The main problem is the effort of typing the import, but last time I checked itemgetter performed a bit better.

To me it depends on what I'm writing. If it's a library where performance will matter then it's the itemgetter approach. If it's a script it will be the lambda.


The first returns a callable, the second a value.


The first returns a callable that's the value assigned to the key argument. The second is a callable that's the value assigned to the key argument. They're the same thing.


Python really needs much better anonymous functions - Lambda expressions just don't cut it.

To be clear, what Python does need is multiline, inline anonymous functions. In JavaScript/ES2015, the fat arrow function syntax is just remarkably powerful in its ability to simplify code. I would go so far as to say that the ES2015 fat arrow syntax completely changed the way my programs are written, increasing power and reducing complexity. ES2015 fat arrows I think are possible the best programming language feature I know.

async programming in JavaScript benefits massively from the beautifully terse and powerful anonymous functions in ES2015. Python now has powerful async programming, but no equivalent to the ES2015 terse inline function anonymous function syntax.

I raised this issue elsewhere once and someone said "there will never be multiline, inline anonymous functions in Python because it would require brackets". If so then it is a great pity that Python is simply not capable of including a language feature that is IMO absolutely critical.

Also, as mentioned elsewhere in this thread, using the word "lambda" to describe anonymous functions is a really really bad decision. Lambda sounds very deep, very computer sciencey and very complex. Probably they should just be called anonymous functions in keeping with the rest of the industry, which also is not a great name but is much better than "Lambda" functions.

Beginners might think along these lines: "Lambda .... lambda calculus? I don't know calculus. I'm outta here."

Names matter.


Python is fundamentaly flawed, since it's based on lines (and significant indentation), and foremost, since it's based on statements, and not only on expressions.

Lambda expressions look in python like a sore spot, because Python stresses so much statements, and procedural programming.

IIRC, recently we've seen passing a paper in hacker news, that let beotians write programs, and only ~10% IIRC (or less) of their uterances were procedural statements!


I've seen people make the distinction between statements and expressions in the past, but I don't get the significance. I mean I understand what they are but I don't understand why people view the concept as such a big deal.

Are there any resources explaining this which you would recommend?


Expressions can be nested arbitrarily and infinitely — like in math. You can take “x * x” and “3 * x” and add them into “xx + 3x”. You can pass that to a function like “log(xx + 3x)”. You could then divide that, square root it, on and on. You can combine the expressions however you want.

Statements can also be combined, but in a separate way. You have control flow blocks, and inside of those you have one serial list of statements. Control flow blocks and statements can only go in other control flow blocks, and not in expressions.

If you’re coming from languages like C or Python, that might not seem like a huge deal. Why would you want “x = 5” or “while y < 10:” in an expression?

Languages that focus more on expressions have surprisingly simple ways of expressing some things, though.

In Rust, “if” statements are fully expressions, so you can assign them:

    let x = if y > 10 {
        let a = readLine().parse();
        let b = readLine().parse();
        a + b
    } else {
        y * 2
    }
Another neat feature is that macros can return blocks, and you can still call those macros wherever you like. For example, in the standard library, there is a “try!” macro that checks if a value is successful. It expands basically into:

    if result is successful {
        result.value
    } else {
        return result.error
    }
The “return” there basically causes the error to short-circuit up the stack. In practice, that means that you can use “try!” to simply say “give me the successful value, and if there was an error, just send it up the stack.”

If you wanna get really fancy, languages like Haskell and Elm don’t have side effects, so things like “x = 5” don’t exist. There are only expressions. You can create values that represent doing something (kind of like Redux actions), and there are lots of tools for combining those action values. Because it’s all expressions, you can combine them arbitrarily. If you have a common case for how you combine your effects, you can just make a plain old function for it, because it’s all just expressions.


beotians?


Yes and the fact that they aren’t full closures is a travesty - I can’t tell you how many times a nice dependency injection has been ruined by having to add a bunch of def’s. It bloats code and reduces expressiveness.


Nested functions in Python are "full closures" in every sense I understand it. Is there something in missing?

They can read the value of local variables (including arguments) in the parent function, and write to them (using the "nonlocal" statement). If one nested function assigns a new value to a captured variable then, of course, this change is seen by all other nested functions and the enclosing function. Most importantly, if the lifetime of the nested function object is longer than the runtime of the enclosing function (most commonly because you return the nested function object from the enclosing function) then the lifetime of the enclosing function's local variables is extended appropriately.

This applies to both nested functions defined using a "def" statement and nested functions defined using a "lambda" expression (except you can't assign to variables in a lambda expression). I say this because you don't say what you mean by "they", and it's a common misconception that only lambda expressions capture enclosing variables.

The only disadvantage of this is that, because name lookup is done at execution time rather than parsing time, all the enclosing variables must be captured and their lifetimes extended even if the nested function doesn't use any of them. In this sense, nested functions are too closurey in Python! This is not really so bad because if you don't need variable capture then you should simply not be using a nested function in the first place; this is best anyway because it means you are sharing one function object between different uses.


Sorry if this wasn't clear - I'm talking about lambdas. I'm basically complaining about a piece you pointed out, that you can't assign to variables in a lambda expression, this is very not ergonomic IMO. Having to create a def every time I want a function to assign to a variable is annoying. I pretty much want anonymous nested functions.


This is the number one thing that makes it difficult for me to want to switch from JavaScript to Python despite Python otherwise lining up almost perfectly with my programming/syntax preferences.

JavaScript has things like TypeScript (and Babel in general) that add features that are "missing" from the language. Is there anything similar for Python and anonymous functions?


FWIW, in many languages, "lambda expressions" and "anonymous functions" are two similar but different things. Specifically, a lambda expression is a shorthand syntax for writing anonymous functions whose body consists of a single expression.

Which is pretty much what you get in Python.


Lambdas aren't single expression functions in Lisp.


Haha, I had assumed someone would mention that.

But most Lisps are pretty rough-and-ready about these sorts of things. IIRC, Scheme requires the body of the lambda to be a single expression, doesn't it? And then you just cheat your way out of it with (begin ...).


If it did it would be annoying to type progn all the time and someone would make a macro...


> IIRC, Scheme requires the body of the lambda to be a single expression, doesn't it?

No, it doesn't require it.


It used to, though. From the original Scheme paper: Note that in SCHEME, unlike MacLISP, the body of a LAMBDA expression is not an implicit PROGN.


Changed in the early/mid 80s I think. AI Memo 848 (Revised Revised Report) from 1985 has it already as an 'essential special form': (lambda (var1 ...) expr1 expr2 ...)

R1RS does not permit it. And: In the the R1RS syntax BNF there is a (DEFINE (<identifier> <identifier list>) <form>) with one form. But later in the document it is a '<form list>'... Then the LAMBDA syntax has a BODY, but it is only a FORM...


Which languages?


C# offers three different ways to denote an anonymous function: Anonymous delegates, lambda expressions, and statement-bodied lambdas. Each of them are true closures and can be used to construct a "method" of a given delegate type (i.e. any one of them can be used to as a value for a given variable — interestingly, in any case, an anonymous method must be assigned to a variable (or parameter) before you can use it, unlike, say, the anonymous functions of Lisp, JavaScript, F#, or even the criminally underrated VB).

But they have some important difference:

1) Anonymous delegates (introduced in C# 2.0) allow you to elide the formal parameters when defining a method of a given type if the body of the method doesn't use those parameters. Even though this construct is basically obsolete, I still use it just for this feature.

2) Lambda expressions can be automatically transformed into expression trees, which means that you can use them to construct (awesomely powerful) fexprs[1]. But, they can't contain statements, which means you can't use them for any kind of imperative code. These are the most equivalent to Python's lambda as far as that limitation is concerned.

3) Statement-bodied Lambdas can contain statements (as the name implies), but the language won't convert them into expression trees for (you can do it manually – tedious but not at all hard). At the time that the lambda expression feature was introduced (as a member of the constellation of features that made up LINQ), the framework didn't even contain expression classes that corresponded to most of the imperative constructs, but that changed with the advent of C# 4.0 which implemented a meta-object protocol[2].

Another implementation detail (last time I checked, which has been a while) is that anonymous delegates get turned into instances of System.Multicast delegate, whereas a lambda expression either gets turned into a class with a method that corresponds to the lambda expression (and fields that correspond to closed over variables), or into an expression tree, depending on the class of the variable to which it's being assigned.

[1] https://en.wikipedia.org/wiki/Fexpr

[2] https://en.wikipedia.org/wiki/Metaobject


Always heard them called lambda expressions in c++.


Agree. Without multi line lambdas im javascript, promises would never have been invented.

The braces also makes it easier to find the start and end of a anonymous function even if it's just a one liner, compared to pythons where its only delimited by : and ,


How exactly does not naming your functions make your code more powerful and less complex?


Why does it need that, though? What's wrong with just naming the function?


It's not so much naming the function, but that doing so moves it outside the flow of the logic it is participating in. Especially when it's going to reference local context that is particularly confusing. There are definitely times when extracting out a function and naming it is the right thing to do. But there are an equal number of times when keeping all the local context together is better. It's probably hard to appreciate this unless you spend some time in languages where closures are seamlessly integrated like Groovy.


I've used Lisp extensively and I've used closures extensively. In Python you can simply define the function on the line above where you plan to use it. I really don't see the need. Lisp needs lambda because you can't just randomly introduce new variables everywhere (you have to use flet, which actually is a lambda).


I find it tremendously helpful to move the function out of the flow of logic so it is modularized. When I read scala or haskell code where as soon as you see map or filter or something you know you’re gonna get some crazy anonymous function to follow, I get sad because it’s a miserable and confusing way to write code instead of pulling the function out into a separate definition with documentation, and then having the map or filter part be extremely concise using only predefined functions.


> I get sad because it’s a miserable and confusing way to write code

you just have to visualize it as a tree


Yes, that is a confusing way to write and read code, rather than a linear flow, like flattening the tree by extracting functions into separate definitions.


> Yes, that is a confusing way to write and read code, rather than a linear flow, like flattening the tree by extracting functions into separate definitions.

I don't understand. It's harder to flatten the tree: you have to unflatten it in your mind afterwards to understand what's happening. It's easier to just visualize, say, this lisp function as a tree directly.

    (defun good-enough-p (guess x)
      (format t "~% Guess =~7,4f     Guess^2 = ~7,4f    Error= ~7,4f" guess 
              (* guess guess) (abs (- (* guess guess) x)))
      (< (abs (- (* guess guess) x)) .001))


There's real readability and clarity to positioning an anonymous function right at the point in the code that it will be used.

When you name and define a function some other place and use it elsewhere, you have increased the overall complexity. What is this named function? Where is it used? Is it accessed by other bits of code? Should it be? All these questions come up when you see a named function. It is in fact less readable and requires more code to define a named function elsewhere and then call it. Inline terse anonymous functions are more straightforward, self explanatory and simple.

It is often the case that you know an anonymous function will only ever be used once, in this particular bit of code so it is much more clean, readable and understandable if the anonymous function lives right there.

Especially valuable in reducing the complexity of async programming.

It's hard to make a strong case just in words, but once you really understand the power of the fat arrow syntax for anonymous inline functions then you use it more and more and it becomes second nature and the programs you write have a completely different style, oriented towards neatly positioned inline anonymous functions everywhere that typically can be read and understood at a glance as you skip through the code.

There is simply no other way to write a function that absolutely cannot be called by some other bit of code. That's the problem with functions that are defined outside their usage context - you may know for sure that the function will only ever be called from this one point in your code, but if you have to declare it as a named function elsewhere then there is always the possibility that in the future someone will come along and make use of that function - it is more complex in the immediate term and leaves open the door for increasingly code complexity in the longer term. Terse inline anonymous functions such as the fat arrow syntax solve this perfectly. They are so clear and easy to understand that it's actually rather beautiful.

This is how terse you can make a JavaScript fat arrow function:

  x => x
It is a function that take a param of x and returns x. A more useful example might be:

  message => console.log('new message: ', message)

My primary languages are Python and JavaScript .... and coming first and primarily from Python, I know that the Python community feels it has a monopoly on readability. But having gained a fair amount of experience in JavaScript too, I can now say that things like inline anonymous functions and the extremely terse fat arrow syntax greatly increase readability even further. If Python had both then it would be even more readable, terse and powerful. Add destructuring on top of that and programming in Python would be almost a new experience.


I actually think that positioning an anonymous function at the site where it’s used is extremely unreadable and unintuitive. I work in Scala and Haskell a lot so I see it all the time and I’m usually forced to write code that way to stick to local conventions, and I despise it.

It makes no sense at all to break up the wonderful conceptual flow of functional components like map, fmap, filters, folds, monadic operations, with sudden harsh definition of a function that takes your brain out of the context of what was flowing and into the context of what is the function.

I want to see something like

    val someStat = myData.groupBy(thingExtractor)
      .foldLeft(baselineValue)(statAccumulator)
I want the mental task of grokking thingExtractor, baselineValue or statAccumulator to be a wholly separate task from seeing what flows into the calculation of someStat. I want to look elsewhere in the code for those things, kind of like expanding or collapsing a block of text: keep it out of mind when it doesn’t matter.


It comes down to a preference thing.

I want to read code like an essay. I don't want to have to jump to a function definition to figure out what `thingExtractor` does. To me, that's like taking a book and rearranging the chapters in alphabetical order. No, put them in chronological order. Don't define a function three "chapters" ago in your code and then expect me to remember it if you're only going to use it once. I wouldn't read Ikea instructions that way, why should I read code that way?

But again, personal preference. And you don't have to choose either/or -- I love that Javascript's anonymous and named functions are so similar because it means you can switch between the two styles with basically zero cognitive overhead.

I will say, having worked in large enterprise before, I have seen bugs come out of people not inlining code and instead attaching it to classes or leaving it otherwise accessible. Specifically, people will use functions that are intended to be method-specific in other places. Even if those functions are well-written and don't have side-effects (which is often not the case), the original author still doesn't know that other people are now depending on them as an API.

So later on the code gets refactored and those methods get changed or removed, and suddenly you have a broken build. That's something you can partially mitigate by making a function private (although not entirely, because I've also seen it happen in code within the same class/scope). Python doesn't have true privates, but you can also mitigate that problem by just using code reviews to force people to respect `_`. In practice, I often found that it was easier to make the function anonymous, so everyone knew 100% that it was only being used in exactly one place.


I also work in an enterprise situation and very many bugs that I deal with from legacy code and others’ code comes directly from in-lining anonymous functions.

Many bugs have to do with utilizing closures to access variables needed in the function body, which then make refactoring harder and make modifying unit tests harder.

This can be even worse in languages like Python where there is a distinction between early binding and late binding and you have to be aware if by closure you’re using a reference name that may be associated with different data during its lifetime, or a name whose value won’t change, because a change to the underlying value could make a difference in what is bound inside the anonymous function at different times when it’s called. The classic example is trying to define functions in a loop where functions use the loop variables by closure. Then being surprised when every one of the functions has a reference to only the final value of the loop variable.

Even in statically typed languages, this makes things much harder to reason about than they should be. On the other hand, making an anonymous function that accepts many arguments for all the data it would try to access by closure is stupid: those arguments need to be documented, and it’s just so much cleaner and maintainable to do that with a regular function definition, which also makes it much clearer what all the conditions are for calling the function.

What’s worse is that these things can be deeply welded into some coding context, like using a flatMap over some TypedPipe in Scala / Scalding, and can result in needing to make in-line functions that are hundreds of lines long with arbitrarily complex function bodies, which then become tied into assumptions about the runtime context you’re embedded in, and then nobody can figure out how to refactor it into a standalone function, so it just grows by attrition to an inline lambda over years and is extremely fragile. Change something seemingly unrelated about the outer context it’s defined in, and suddenly you get unexpected, cryptic compiler errors complaining something’s wrong with the TypedPipe, and you have to dig deeper to understand why it’s related to the anonymous function.

I would say many of the most serious closure-related bugs and bugs related to unrefactorable yet undocumented dependence on an enclosing context that I’ve seen have been largely a direct result of the programming style of in-lining anonymous functions inside functional programming constructs.

I sympathize with your claim of “not reading an essay” especially because people can be prone to try to use functional programming or overloading the Python data model and operator syntax with cutesy bullshit that they try to pass off as expressiveness.

But I think there’s a middle ground where you think about it not like an essay, but just basic modularity and separation of concerns, and write functions separately except when they are very short and really trivial.


Hrm. I wonder if the differing bugs just comes from which one people are doing more.

I often have to fight to get people not to expose huge amounts of state whenever they build a module, so by far the most common bugs that I see are people being too loose with turning things that really shouldn't be modules into very fragile, cumbersome modules that depend on being used only in specific (undocumented) places with specific (undocumented) setup.

If I was in the opposite situation, and everyone I worked with already inlined code all the time, then probably most of the bugs I'd see would be related to people reusing variables, abusing hoisting by making spaghetti references to variables that are defined later in the function, etc... and in that case I could definitely see myself agreeing with you.

I have on occasion wanted the ability to define an anonymous function that didn't inherit variables from the scope that it was defined in. So I'll give you that - I would love for the ability to make an anonymous function that only has access to variables that are explicitly passed in. If I could isolate variables going into a closure as easily as I can isolate variables going out, I suspect a lot of the problems you're talking about would be easy to solve.


That is a good point that whichever approach displays the most bugs in a given team is likely to just be whatever is the most common approach for that team, by simple base rates.

I’m not sure how we could objectively decide if either of these two approaches is definitively better, but in the specific, restricted case of a framework like Scalding, I’d strongly wager that avoiding in-lining ends up better in the long run. Those cases also have little connection to the weak module design issue you brought up, since it’s usually a module with de facto map reduce boiler plate and then just a few isolated places with any actual implementation, and when those parts are expressed as huge in-lined anonymous functions inside Scalding data type wrappers, I know right away it’s a bad code smell.


> I’m not sure how we could objectively decide if either of these two approaches is definitively better

The correct approach might just be the opposite of whatever you and your team's predilection is. In your case, you're saying that the teams you work with are using inline functions instead of following a restricted framework, in part because they're using languages that encourage them to just slap a bunch of nested code in instead of writing out the extra boilerplate.

Well, they probably already know to be careful about module design -- so if you encourage them to inline less code, odds are pretty good you won't suddenly wake up in the morning with a codebase with a hundred classes and a bunch of obscure private/public methods named `setupEntityExtractorForCollisionPart3`.

On the other hand, if your team is coming from a Java background and half of them are starting from the position that lambdas are just witchcraft, then it's probably not a bad idea to get them over that fear.

In a setting where everyone is inlining most of their code, probably the tests that are coming out are all integration tests, so... yeah, bias towards creating units so you can unit test. In the opposite situation, I'm actually just trying to get people to stop testing private methods and leaking implementation details into their tests. So I would love if people were testing with a little less granularity.

I know that my initial reaction to you listing off the problems you've run into with people building anonymous functions that couldn't be refactored was just, "yeah, but why the heck would anyone make that mistake? How hard is it to organize the variables in one function?" So I assume that other people might listen to my complaints about improper code reuse and think, "yeah, but why the heck would anyone ever just reuse a method in a class without checking the documentation first?" So my takeaway from that is, "different people struggle with different things."


Python has a convention for "private" functions, which is to prepend "_" to the name.

> It's hard to make a strong case just in words, but once you really understand the power of the fat arrow syntax for anonymous inline functions then you use it more and more and it becomes second nature and the programs you write have a completely different style, oriented towards neatly positioned inline anonymous functions everywhere that typically can be read and understood at a glance as you skip through the code.

This is exactly why I think Python has such limited anonymous functions: because it does not want to promote that style.

> There is simply no other way to write a function that absolutely cannot be called by some other bit of code.

Python also believes that you do not need to defend against this possibility, as long as you make it clear to other developers that they should not call your "private function". This is also why the language allows monkey patching anything at will. If someone wants to ignore good sense, let them.


Private functions do not restrict the usage of that function to one point in the code. There's a world of difference between a coding convention - essentially a comment to say "please don't access this function from outside the current class" versus it simply being impossible to do so. And the "private" convention does not say "don't use this function at any other point except one", it says "please don't access this function from outside the current class", which is completely different to the point I make. Anonymous inline functions reduce lines of code and complexity.

>>because it does not want to promote that style.

Why would Python not want to promote a more readable, simple, powerful, reliable and maintainable programming style?

Can you reference anything to back this assertion?


The parent comment about private naming convention is a red herring.

99% of the time in places where you might use a multi-line anonymous function you're already inside another function

So when you define your named function in Python, it is only available within the scope of the current method call and there's no danger of it being abused by other developers.

The lack of true private vars in python is a separate issue, and one of the best things about the language (because library authors inevitably get carried away and mark things as private which don't need to be - then you're stuck in copy & paste land... I did loads of ActionScript back in the day and this was super frustrating)


As a user of JavaScript and es6, fat arrows are only necessary if you haven't started using async and await syntax, and I found myself almost never using fat arrows after moving to async/await.

Indeed naming my functions resulted in more testable and clearer code.


Hmmmm. I don't see the relationship between async/await and fat arrows.

Can you explain more about how async/await obviates the need for fat arrows?


Just async is callback based.

You don't need callbacks with async/await. No callbacks, no need for fat arrow.


Interesting idea.


Getting rid of fat arrows is one thing, but I use async/await pretty much all the time, and I still use anonymous functions in my code.

  await (async function () {
    //Do a thing
  }());
I don't think async changes anything about how useful or problematic an anonymous function is.


That feels like a weird pattern (for one because you can have async fat arrows), but also because in general I find

    await request_some_data()
much clearer than

    await (async () => {
        // just make the async request inline
    })()


When arrow functions were first released, they didn't support the `async` keyword. I guess that wasn't fixed until ES2018? Maybe earlier - I think Babel would compile them pretty early on. Regardless, some people who adopted async patterns early got into the habit of avoiding arrow functions entirely, so it wouldn't be crazy to me to hear someone say, "I used arrow functions before I learned async."

Otherwise, this is just restating the same arguments people already made above.

There's no practical reason why

  (() => {
  
  }());

would be useful but

  await (async () => {
  
  }());
wouldn't be. The scenario you're talking about where async code removes the need to have an anonymous function only makes sense if you were only using anonymous functions for callbacks. In practice, that's not the only (or even primary) use case for people who advocate inlining code in Javascript. They're specifically advising against the pattern where single-use functions are given names and removed from program flow.


Correct, and I fundamentally disagree with that pattern. If it's more than a single expression, it's doing something complex enough to deserve a name or to be mocked in tests, etc.


While I can see async/await replacing a lot of the most common use in JS, anonymous functions have a lot more uses than that (passing a single use function to generic algorithm, like an ordering function to a sorting routine, is one of the more common other uses, across languages.)


Come on. You've defined a function inside another function. It's very clear that you've done it so you can pass it to some higher level function. If someone comes along and starts using your inner function willy nilly in the outer function without anyone noticing then you've got much bigger problems that multiline lambdas are not going to help you with.


> Python also believes that you do not need to defend against this possibility, as long as you make it clear to other developers that they should not call your "private function".

Python has apparently never worked in a large, corporate environment with a multinational team.

Yes, in theory you can just not call private methods. And in my own personal code I don't care much about restricting variable access, because I trust myself. However, it is a tremendous amount of work to get a company culture to start respecting privates if they aren't already. I've had so many phone calls trying to explain why "yes, you can technically call this function, but in reality you can't, and I don't care that your deadline is tomorrow, you still can't. And yes, I know that technically you're on a separate team and not under my jurisdiction, but you can't just remove me from the code review and call all of my library's private functions anyway."

It's taking the path of most resistance. True privates are very helpful on large teams.


You do realise you can define a function inside another function in python, right?


So if I understand correctly, in Javascript a fat arrow is just the syntax sugar for:

    var x = function(x){ return x };


It also changes the behavior of this. [0]

[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...


The two biggest differences iirc:

- it can never be named

- it inherits `this` lexically from its parent scope (which imo is the sane default)


all i could think of is "stop writing python". it seems to me that a lot of these problems are fundamental problems with python. i continue to not understand why anybody likes python the language.

the fact that creating a function with lambda versus the normal way is different is bonkers. for example, in f# (and other sane languages), the following are identical:

  let test1 = fun x -> x * x

  let test2 x = x * x
both return:

  x:int -> int
are they really different in python or is it just a weird naming problem? the article isn't explicit on the actual differences other than what is reported by the REPL.

also, you can't pass operators as functions in python? in f#, you simply wrap them in parentheses.

  let test3 f x y = f x y

  > test3 (*) 2 3 ;;

  val it : int = 6
i read an interview with hal abelson where he called python's lambda broken. seems so.

edit: also, this guy's thoughts on map and filter, in my opinion, show the python community's backwoods thinking regarding functional programming. yes, it seems generator expressions are nice (they are called sequence expressions in f#), but map and filter, even for his simple examples are much more clear. they communicate better what is actually happening. the fact that he never uses map and filter at all and then goes on to basically say map and filter aren't even needed in python says a lot.


"i continue to not understand why anybody likes python the language."

As someone who has had a lifetime of on-off relationships with attempting to become a competent programmer, (BASIC, assembler, C, Pascal, Visual Basic, C++ and others that I can no longer remember), Python is the only language that has helped me become anywhere near useful or proficient as a programmer. I've managed to build a couple of -genuinely useful- applications (one of which is being used daily in my girlfriend's school), which hasn't been the case for anything else.

I've found that Python is simple enough to learn, and hasn't really been any kind of a limitation for me - this may be because of me, not the language - OK, I appreciate that I'm not going to be writing 3D games or real-time audio apps in it, but for what I've been trying to do (solve real-world problems), it's been amazing. I even programmed a 'play sound effects when you press a MIDI controller' system which I used at a show with 20,000 people attending this summer, replacing a massively over-complex (and expensive) audio environment with about 30 lines of Python combined with two libraries I spent an afternoon learning.

That, for me, is why I like Python.


> I'm not going to be writing 3D games or real-time audio apps in it, but for what I've been trying to do (solve real-world problems)

TIL 3D games and real-time audio apps are not "real-world problems".


I think the idea is that python is appropriate for solutions to some problems that are "real" problems (as opposed to being valuable purely as a toy) -- not that python is appropriate for solutions to EVERY "real" problem, and any problem that python would be a poor choice for isn't a "real" problem.


Exactly! Looks like the 'deliberately take umbrage over a specific interpretation of what you've written' crew have arrived. Clearly I didn't mean 3D or audio apps aren't real world problems, I meant I can solve (some) real world problems with Python.


I'm not sure what you mean by the first half of your comment. Both def and lambda create functions, and quoting from the article "Lambda expressions are just a special syntax for making functions. They can only have one statement in them and they return the result of that statement automatically.".

def binds it to a name at the same time, which is remembered in the object (which AFAIK isn't really used for anything except maybe debugging tools), lambda returns an anonymous one. The resulting object can be passed around in both cases.


it's because i am not that deeply familiar with python and the article is not clear itself on this. what is he complaining about then if they are identical in what's returned? is it simply because the REPL returns <lambda> instead of the function name? why does the REPL do this? are they different in some subtle way?

the point was in something like f#, lambda creates a function which can then be bound to a name if you want. if you do so, this is identical to the "normal" function definition because that is really just syntactic sugar for creating lambda functions and then binding it to a name.


Just as in f#, Python's lambdas can be bound to a name. There's no real difference between a def and a lambda. You're complaining about some distinction that doesn't exist. You didn't even try to discover whether it does before complaining about it.


Modules and functions in Python have a __name__ attribute. If you've used Python for scripts, you've probably written `if __name__ == "__main__":` at some point, which is the module name for executed scripts, whereas imported modules get their proper name.

Similarly, Python basically keeps track of "proper" names for proper functions, but can't/won't do so for lambdas.


> is it simply because the REPL returns <lambda> instead of the function name? why does the REPL do this?

Because lambdas don't have function names.


i know that. in the example, he assigned the lambda to normalize_case.


Sure, but an object doesn't know what variable it is assigned to (and which variable name should it pick if there's multiple).

the def syntax merely sets the name property on the function object too, since it has a name to set available. As said in other comments, nice for interactive use, irrelevant otherwise.

    >>> normalize_case = lambda s: s.casefold()
    >>> normalize_case
    <function <lambda> at 0x034B04B0>
    >>> normalize_case.__qualname__ = "normalize_case"
    >>> normalize_case
    <function normalize_case at 0x034B04B0>
vs

    >>> def normalize_case(s):
	return s.casefold()

    >>> normalize_case
    <function normalize_case at 0x00594420>
    >>> normalize_case.__qualname__
    'normalize_case'


thank you for clarifying exactly what's going on. other than the __qualname__, in what other ways do the two different bindings differ from each other?

however, this still goes in the bucket of why i personally dislike python. this just seems sloppy and overly complicated.

for example, i tested this out myself.

  > test = lambda x : x*x
  > test
  => <function <lambda> at 0x7ff6221d76e0>
  > test.__qualname__
  Traceback (most recent call last):
    File "python", line 1, in <module>
  AttributeError: 'function' object has no attribute '__qualname__'
  > test.__qualname__ = "test"
  > test
  => <function <lambda> at 0x7ff6221d76e0>
so the repl still reports <lambda> instead of test. actually it turns out that even def doesn't seem to set the __qualname__ property, so i don't get the same thing as you (i am using repl.it which apparently uses python 2.7.10). however, using dir(test) showed that there is the func_name property.

  > def test2(x):
  >     return x*x
  > test2.__qualname__
  Traceback (most recent call last):
    File "python", line 1, in <module>
  AttributeError: 'function' object has no attribute '__qualname__'
  > test2.func_name
  => 'test2'
  > test.func_name
  => '<lambda>'
that seems to be the property that the repl reports, as it gets set by the lambda assignment and the def keyword to the function name. (i verified that by altering the func_name property manually.)

yet, somehow, people call python "simple".


Van Rossum in https://python-history.blogspot.com/2009/04/origins-of-pytho... comments that "lambda" was likely the wrong choice for a name, saying "the choice of terminology had many unintended consequences. For instance, users familiar with functional languages expected the semantics of lambda to match that of other languages. As a result, they found Python’s implementation to be sorely lacking in advanced features."

Perhaps it would help if you didn't think of "lambda" as defining a lambda function, but something more prosaic like "defexpr", and that Python doesn't have lambda functions at all?

As to your experiments, you have found one of the many differences between Python 2 and Python 3.

After researching F# for about 10 minutes, perhaps you might think of Python 2 as being like OCaml and Python 3 being like F# - they are two instances of the same language family, with a mutually compatible language subset, but many incompatibilities.


As you've discovered, the reported property had a different name in python2. The repl doesn't report a property, it asks the object to give a representative string (__repr__() function) and prints that. Other languages wanting to expose something like this would maybe hide it deeper in the interpreter or build it into each tool separately, Python chooses to make everything accessible from the language itself.

You've bitten onto a more or less internal detail (I've TA'ed a bunch of undergrad courses in Python, and it's firmly in the land of things students don't know if they don't discover it on their own) and use that and seemingly quite little grasp of how Python works to argue that it isn't simple, while of course languages you are more familiar with are better.


FYI, repl.it has Python3, you just need to select Python3 and not Python.

As far as I know, the __qualname__ property was introduced in Python3.3, so obviously you wouldn't see it in any Python 2 version.


Don't complain about design warts if you're using an old version. Python 2 is just a couple years from end-of-life, and that's after the support guarantee was extended.


i didn't choose python 2.7. it is what i have installed on my work computer because the three projects i have worked on at work that used python were all using python 2.7, and the projects were led by "python people". in a couple cases, it was me who got them to upgrade from 2.6 to at least 2.7, and i have tried communicating to people to use python3 if they insist on python. i wouldn't choose python for a new project unless i had to again help someone else's project.

python's version bifurcation problem is its problem, not mine. the problem exists.


Sure, just like how it's Microsoft's problem that Windows 98 still gets used in a few places.


Since the objects are identical type, you could set a doc string and a name on a lambda (they're just properties of the object) and see them during program execution/on the REPL, might make sense for generated stuff like classes from an ORM?


> i continue to not understand why anybody likes python the language

Here are some reasons why I like it:

It's cleaner to read because there aren't curly braces and semicolons cluttering up the code.

It's cleaner to write because the expressions are simpler and look like something readable instead of line noise.

It has a REPL, so it's easy to experiment.

It doesn't force me to write boilerplate code, such as types for every variable declaration or everything having to be a class or huge amounts of scaffolding just to write "Hello, world!".

I'm sure I could think of more, but those are what come to mind on the spur of the moment.


These are good reasons for why people might prefer python over other imperative languages, though basically all these points also apply to Haskell/OCaml/F#/etc., though some people would still say that a lot of things there look like line noise.

I think the GP's point was that functional languages give you similar benefits, but with more uniform and disciplined behavior of many language features. Why FP is not more widely used compared to imperative language is another topic though.


I can only speak for myself and I'm more interested in getting a job done. If that means my UI component uses class syntax, and my state and communications channels are functional reducers, so be it.

I love JS because it's flexible and doesn't lock me into a paradigm. I can be as strict as I want to be. In the end, I think it depends on how someone thinks about problems. I tend to bend my thinking depending on the type of problem and how I might optimally solve said problem. When I need something else (often for performance), I'll circle around at that point for that part.


f# (and also ocaml and sml) have all those and much more. they are much more consistent and principled in their design, which makes things simpler to understand.

for example, in all three languages above, expressions are much easier to understand because they always return a value. this makes code easier to reason about. this is not true in python.

the languages i mentioned do have static typing but they have type inference. so you get the best of both worlds. you do need to type annotate at some points to clarify things, but this isn't a problem as it documents to both the user and compiler.

given your above reasons, you owe it to yourself to learn an ml. there is the programming languages course on coursera by dan grossman, which uses sml in the first part, and the ocaml mooc just started where you still have time to register and complete it.

https://www.coursera.org/learn/programming-languages

https://www.fun-mooc.fr/courses/course-v1:parisdiderot+56002...


> f# (and also ocaml and sml) have all those and much more

F# has curly braces. Also, as you say, it requires some type annotations.

I see that F# has a REPL, but it seems bolted on as an afterthought instead of being an integral part of the language. (Also, it's not clear whether the REPL is available in Mono on Linux. I am allergic to Windows and anything built by Microsoft.)

> expressions are much easier to understand because they always return a value. this makes code easier to reason about. this is not true in python

Huh? Python expressions always return a value.

Perhaps what you mean is that Python also has statements, which do not return a value, whereas these languages do not--everything is an expression, as in Lisp?

> you owe it to yourself to learn an ml

I don't owe it to myself to learn anything unless I decide to. I wasn't trying to convince anyone else to use Python; I was just giving reasons why I like Python. Please extend me the same courtesy when you talk about things you like about your favorite languages.


okay, f# has curly braces. but not in the sense of other C-like languages like c++, java, and c#. they are primarily used for sequence expressions and computation expressions like async.

> I see that F# has a REPL, but it seems bolted on as an afterthought instead of being an integral part of the language. (Also, it's not clear whether the REPL is available in Mono on Linux. I am allergic to Windows and anything built by Microsoft.)

well that first sentence is simply not true, and i don't know what gave you that impression. f#'s repl, f# interactive, is available on linux through mono and will soon be available through .net core. i can't help you being needlessly allergic to a major operating system or company. f# came out of microsoft research.

> Huh? Python expressions always return a value.

they do? take:

  def test(x):
    "something"
so what does

  test(2)
return?

> I don't owe it to myself to learn anything unless I decide to. I wasn't trying to convince anyone else to use Python; I was just giving reasons why I like Python. Please extend me the same courtesy when you talk about things you like about your favorite languages.

i don't know why you're upset. i am not for sure where i was discourteous. i didn't mean anything by my statement other than to check out the languages and courses (the coursera one is very good). if those are your reasons for python, i just thought you would honestly like an ML-based language. i just used "owe it to yourself" as an expression, which apparently came out wrong through text. of course you don't owe anyone anything. i was just making a suggestion.


> of course you don't owe anyone anything

Yes, and thanks for acknowledging that. But then the phrase "owe it to yourself" didn't really communicate your intent correctly, which is why I objected to it.


Replying to your reply of the child comment:

> i guess that is true in python3 but it is not in python 2.7.

No, it's always been there.

    >>> def f():
    ...     pass
    ...
    >>> f() is None
    True


>>> test(2) is None

True


i guess that is true in python3 but it is not in python 2.7.


False. You're just not familiar with how the REPL works.

    >>> None
    >>> print(None)
    None

In the future, you might want to refrain from strong criticism of a thing you're not familiar with. That habit leads to all sorts of -isms.


i do know how the repl works. i explicitly tried it out. i defined the function just as i said and ran

  test(2) is None
and got false. i just now went to try it out again at home and got true. i don't know what to tell you other than something must have got shadowed somewhere.


Here's what I get from the Python 3 REPL:

    peter@localhost:~$ python3
    Python 3.5.2 (default, Nov 23 2017, 16:37:01)
    [GCC 5.4.0 20160609] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> def test(x):
    ...     "something"
    ...
    >>> test(2) is None
    True
And Python 2:

    peter@localhost:~$ python
    Python 2.7.12 (default, Dec  4 2017, 14:50:18)
    [GCC 5.4.0 20160609] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> def test(x):
    ...     "something"
    ...
    >>> test(2) is None
    True
If you post such a transcript from your REPL session where you get False, it should be easier to tell what's different between your setup and mine (which is a stock Python install on Ubuntu 16.04).

However, there is another even more direct experiment you could have run at the Python REPL to see what Python expressions return: just type the expression directly at the REPL and see what it prints back at you. In Python 3:

    peter@ToshibaSatellite:~$ python3
    Python 3.5.2 (default, Nov 23 2017, 16:37:01)
    [GCC 5.4.0 20160609] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> "something"
    'something'
And in Python 2:

    peter@ToshibaSatellite:~$ python
    Python 2.7.12 (default, Dec  4 2017, 14:50:18)
    [GCC 5.4.0 20160609] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> "something"
    'something'
So, in your test function above, the expression "something" does return itself; it just does it inside the function, and that return value then gets thrown away when the function exits, because you didn't return it from the function itself.


i know how to use the repl. i moved between computers during comments and wasn't able to inspect my repl session where it originally returned false. as i mentioned, i must have redefined something or shadowed something which caused confusion, and i just didn't realize it at the time.


Guido van Rossum was explicitly against functional programming (well, in the Python language anyway). It's an imperative, object-oriented language that eventually threw functional programmers a bone with map, reduce, and filter, and that's about it.


Could you elaborate further? I wasn't there then, but what you wrote doesn't agree with my understanding.

Lambda, map, reduce, and filter were part of the Python 1.0 release. The actual commit date was Tue Oct 26 17:58:25 1993 +0000:

  commit 12d12c5faf4d770160b7975b54e8f9b12694e012
  Author: Guido van Rossum <guido@python.org>
  Date:   Tue Oct 26 17:58:25 1993 +0000

    * compile.[ch]: support for lambda()
    * PROTO.h, mymalloc.h: added #ifdefs for TURBOC and GNUC.
    * allobjects.h: added #include "rangeobject.h"
    * Grammar: added lambda_input; relaxed syntax for exec.
    * bltinmodule.c: added bagof, map, reduce, lambda, xrange.
    * tupleobject.[ch]: added resizetuple().
    * rangeobject.[ch]: new object type to speed up range operations
       (not convinced this is needed!!!)
You can see the commit had its doubts about xrange, but said nothing negative of the functional programming aspects.

Van Rossum described that history at https://python-history.blogspot.com/2009/04/origins-of-pytho... . I don't see anything which sounds like "throwing a bone", much less being "explicitly against functional programming" when that discussion occurred.

My understanding is that he wasn't opposed to their inclusion initially but became negative towards them only years later.


Thanks, I'm probably misremembering some argument that I last heard about around 2000-2004, probably involving list comprehensions and why lambdas weren't more powerful.


Guido seemingly has a strong distaste for functional ideas, so anything that comes from that paradigm ends up getting an almost half-assed implementation.

See also: map and filter being the only 2 other functions implemented and both of them being less performant than their imperative equivalents; outright denial of any attempt at TCO because "Guido doesn't like recursion".


> outright denial of any attempt at TCO because "Guido doesn't like recursion".

No, don’t ignore problems of TCO by saying “guys don’t like functional programimg”. Guido cleary states why he doesn’t like it: [1]. Even though TCO is one of the ES6 standard, TCO also is largely denied by JS implementors (Firefox, Chrome, and Edge devs) because of the similar reasons. As a result, Safari is the only browser that supports TCO. [2][3] And Rust team is not eager to introduce TCO [4].

[1]: http://neopythonic.blogspot.com/2009/04/tail-recursion-elimi...

[2]: https://www.chromestatus.com/feature/5516876633341952

[3]: https://github.com/kangax/compat-table/issues/819

[4]: https://github.com/rust-lang/rfcs/pull/1888


It’s not that we’re not eager, it’s that it’s harder in a systems language and historically there’s been limitations in LLVM.

We will sometimes optimize for TCO, that issue is that we can’t guarantee it just yet.


One of the python motto's is "explicit is better than implicit". Not having to pattern-parse the structures makes programs much easier to read for some people (lke me).

But the point is, even with f# syntax, most of the blog post would still apply (needless function calls, non-trivial functions, multiline-unpacking and so on)

I do not agree with some of the premises of this post, but saying that changing lambda syntax will somehow help is just not true.


> I do not agree with some of the premises of this post, but saying that changing lambda syntax will somehow help is just not true.

i wasn't really suggesting changing the syntax would help (although the confusion here is more semantic than syntactic). i was pointing out that in other languages, like in f#, things are much more clear than in python. the section mentioning the name thing is still not clear to me (in python) in terms of what exactly the differences are, and that was my point. that exists a lot in python.


In F#, can you ask a function object what its original name was? If so, what is the original name of "test1" and "test2" in your example?

That's what python is doing and why the two are different.

(I don't think a language should track that sort of thing, but that's a different story).


i am not for sure what you mean by "original name". the value of each of those function definitions are:

  val test1 : x:int -> int

  val test2 : x:int -> int
you call them the same way, and they have the same type. a caller has no knowledge of how they were defined because it isn't important. they're identical.


I mean in python, one can ask a function what it was originally defined as. Sounds like in F# one can not, so that's why there's a difference in python between "var x = lambda..." and "def y(): ..."; the latter knows it is named "y" and the former has no idea what its name is (so it is "lambda").


> they're identical.

Does that mean that test1 == test2 will return true?

Personally, I have found it very useful for debugging that function objects in Python carry their name. That makes it much easier to find out where something is coming from.


Your examples don't seem that great, to be frank.

The lambda syntax isn't that different from the syntax for defining functions, unless you're referring to the fact that it's different than anonymous functions in other languages.

I don't find the inability to pass around operators to be a hindrance, but the fact that getting into python requires understanding all of these dunder or magic methods is another story.

However, on map, filter, & reduce, I agree with you completely.

One of the worst aspects of python is the extensive use of bad practices, and ignoring functional programming principles is one of the top offenses.


> Your examples don't seem that great, to be frank.

they weren't intended to be great or mind blowing. they were just examples to make the point that in a good language, it's clear what is going on.

the operator thing is an extension of that. in f#, the "infixness" is just a syntax thing. the semantics is that they are functions and can be treated as such. it seems operators are something weird in python.


Re: use of map / filter type stuff.

Yep. Had a coworker like that recently. Loved “simple” code, chock full of mutable data everywhere, where everything was written out a good 3 or 4 times longer than I would have liked to see it.

I call that sort of “simple, explicit” (repetitive) code “My Summer Vacation” code, after a bit in an old Cheech and Chong skit where the highschool kids have to read their essays to the class.


Simple and explicit code? sounds easy to read and debug!


I agree with you, I felt the same when reading the article. It feels that Python's design is deeply at fault here, it is actively harming people's perception of anonymous functions, which is the opposite of what a supposedly educational multi-paradigm-capable language should be doing.


It also doesn't have tail-recursion or macros. There are many who also regard that lack as a deep design flaw.

There never was a goal in Python development to support all programming paradigms.

How do you distinguish between "design flaw" and "different design goal than what you want"?


I think, it was an example of intentional hostile design (say, see this video https://www.youtube.com/watch?v=NWZLB8CyPbM for what I mean). All, seemingly for the sake of that "only-one-way-to-do-things" goal.

EDIT: (Van Rossum's own explanation of why he thinks lambda's design is okay: https://www.artima.com/weblogs/viewpost.jsp?thread=147358 )


Do you have any evidence for your hypothesis about hostile design?

The phrase, by the way, is "one obvious way".


Guido van Rossum said that many times that he does not like functional primitives. It is pretty well documented in his own writing that he had no intention of making functional programming very convenient in the language.


Amen, brother. I'm not proficient in Python, but from my (admittedly, limited) experience with it, it feels like programming with a straightjacket on.


It is, and that's the point. The Python community coined TOOWTDI (there's only one way to do it) in response to Perl's "There's more than one way to do it". What that way is seems to be "what even the most clueless programmer could reasonably understand". This to the point where even the adoption of a concept as common as assignment expressions was met with a community shitstorm.

It seems like a language designed for people who really have no interest in craft of programming, and just use it as a means to an end (note I'm not saying there's anything wrong with this, but if your career is in programming, and you keep that attitude, I'd suggest you find another one or move to management, you're not gonna be happy).


To be honest, I feel like Python's philosophy makes it easier to focus on the "craft" of programming; instead of worrying about small details like how to iterate through a collection, I'm able to just do it the boring-but-effective Pythonic way and move on to higher-level details of the program. So I guess I'm not strictly disagreeing with you, but I think the craft of programming is about far more than just how your code looks, and I find Python to be excellent for pursuing those other areas of craftsmanship.


I suppose so, but the higher level details of a program will often follow from the nature of the language you are coding it in. I know a lot of my code would be "architected" differently if the language I was working in had a general disdain for passing anonymous functions around, and I don't think it would end up as effective.

And finding new more effective ways to do common things, new tricks so to speak, is important in keeping the mind engaged.

The tools might not be as important as the end product, but for the person who is always using those tools, being able to improve and customize them to their liking is extremely important.

I guess the difference is a programming language is less a tool than an end product in itself, that needs to be able to be understood and modified by other people. As opposed to say, a plumbers preferred toolset, of which no trace will be left when the next person comes to do the job.


> It seems like a language designed for people who really have no interest in craft of programming, and just use it as a means to an end

It's kind of funny how much that echos the past criticisms of Java - a language lacking in all kinds of features that frustrate advanced programmers. Yet Python's popularity was in many ways a reaction against Java - perhaps for other reasons.


> i read an interview with hal abelson where he called python's lambda broken

Can you give a link? I'd be interested to read it.


sure. here you go.

http://www.gigamonkeys.com/code-quarterly/2011/hal-abelson/

the paragraph he says it in is:

> It’s just a different course. We could have done it in Scheme and I sort of wish we had. And for random reasons we didn’t. But the thing that ties it together—if you had fifteen seconds to describe the whole course—it’s still about abstraction and modularity. The beginning of that course is not very different from the beginning of 6.001 other than you do it in Python. But at that point, where you’re not building interpreters yet, you can pretty much do this stuff in Python. You have to contend with Python’s broken notion of lambda but other than that it’s not really that different.

he mentions it as an aside really, but the overall interview itself is very good. abelson is a treat to listen to.


Thanks for the link. The paragraph you quote makes me think he is comparing Python's lambda to Scheme's equivalent, which of course will make Python's lambda seem broken. IIRC the course was taught in Scheme at one time, but it switched to Python because it seemed easier for students to grasp the basics that way.


> IIRC the course was taught in Scheme at one time, but it switched to Python because it seemed easier for students to grasp the basics that way.

as explained in the interview, the course changed to python because the course itself changed. they basically killed off the old course and created new courses built around their new degree program structure. the new course hits a lot of different aspects of electrical, computer, and software engineering, many of which heavily use existing libraries, so they seemed to pick python for this.

in my opinion, they could have easily written libraries in another language, so i suspect there were some politics at MIT that lead to the decision to use python.


It's not just Scheme; in Common Lisp and Haskell and every other language that supports lambda, lambda is a first-class construct that is used all the time. Python's lambda feels like a war veteran that had its legs blown off. Best policy is probably just to pretend it's not in the language.


> It's not just Scheme

Agreed. I only mentioned Scheme specifically because Abelson did in what was quoted.


Nobody really likes Python the language especially much, but lots of people love Python the ecosystem, because it lets them get stuff done. I can think of a dozen languages I like better than Python, but none that's more practical in as many different contexts.

--------------------

Edit: Apparently, there are some people who do like Python the language, and some people who dislike Python the ecosystem. If you want to find a thing, just say it doesn't exist, and it will find you.


I disagree... I came for the syntax and readability; I've stayed for the community.


I suppose I like the readability too, but there are a number of things about the language I would rather had been done differently; I still insist that its approach to scope is worse than any choice except making everything global.


I tend to recommend people refavtor their functions into smaller ones if scope is not clear, but I accept it's different from many of its predecessors.


If the local/ global scooping rules are hurting you that much, you are probably putting more state in module-level variables than is healthy.


It's not that: it's the hobbled function scope that doesn't handle nested functions the way I would expect and hope.


in my experience, the python ecosystem is a mess. yes, there are lots of libraries, but that is just one part of the ecosystem. for one, everyone uses a different distribution, whether it's anaconda, python(x,y), or just the default python install. this creates a lot of fragmentation, and it's hard to come into a project and fix the mess that's been created. then there's the 2.x vs 3.x issue that is still a problem. i have been introduced to three python projects at various times in my work, and all still used python 2.7 and even python 2.6. (these were projects that i helped out on and didn't create or own.) in each case, python made it quite difficult to even understand if you had setup the environment correctly.

but yes, there are lots of libraries. i think people need to take a stance and simply pull those libraries into other languages (which many are already) or simply realize that most are just wrappers around some other language library, which can be easily done in other languages as well.


You're right, it is a mess.

Don't get me wrong, there's a huge number of packages available and a lot of them are really quality and I have used the language a lot, but good gosh is the packaging a deployment a fucking disaster.

Environments are a 3rd party bolt on, that relies on hacking around with environment variables. There's no distinction between dev dependencies and actual, application dependencies. The packaging/deployment tool has gone through a bunch of different formats/styles and when it comes to deploying your code, you have nothing better than writing a bash script that automates your environment creation, activation and running of the packaging tool (you then just hope that Pip doesn't fail on some bizarre edge).


Python packaging has a lot of warts, but you can specify dev dependencies in extras_require. Also, you can just run a virtual environment's interpreter without "activating" the environment.


> Nobody really likes Python the language especially much

I beg to differ. I like Python the language. I also like Python the ecosystem.


Personally, I use them often and disagree with a lot of this sentiment. I also disagree with a significant portion of PEP8, however.

Anonymous functions are vital to any modern high level language.

I think most people get hung up on the word. If the keyword was changed from “lambda” to “fun” for example I think it wouldn’t be as obtuse to more intermediate developers.

To be honest I think the lambda does Python a service vs a disservice. It helps to illustrate that everything in Python is an object. Once you realize you can treat functions similarly to how you treat things like integers and strings your mind opens up to functional style. Assigning a function to a variable the same way you do with a string, for instance, helps to reinforce this concept.


Van Rossum started to work on Python in 1989 and released the first version in 1991. Python is modern in the sense of being still very well alive but it's deeply rooted in the 80s, and it shows with all those double underscore __magic functions__, the self argument in methods (the way we were doing object oriented programming in C, without the ++) and others.

Compared to JavaScript it didn't evolve it's syntax much. Of course JS was much worse off to start with.


`self` is about being explicit with parameters. Rust does the same thing and that's a more "modern" language.

The `__magic__` is about avoiding naming conflicts while still being explicit. It's a choice between magic or reserving a lot of names.


Or use normal method names chosen by the class author like for example Ruby and many other languages. In the case of the standard library it's easy to be consistent. I concede that in any other case it can be hard or impossible.

About self, it's so strange to define a method with 2 arguments and call it with only one. Not only it's strange, it feels the opposite of being explicit. Most other languages manage to do without that self (Java, Ruby).

I run into another oddity today. Yesterday if forgot about

    a = 0
    if not a:
        print("like in C")


Rust also has the issue of having three kinds of self, and those differences matter.


> Anonymous functions are vital to any modern high level language.

I don’t think you’re wrong, but I am also not experienced enough to know why this would be the case. What is the virtue of having an anonymous function? I get that it makes code somewhat easier to read and write if you’re only using the function once and it’s short, but that seems like an edge case. What am I missing?


First class functions are vital. Not so sure that anonymity is vital.


I don't know an easy answer to this, but I recommend reading SICP: https://mitpress.mit.edu/sites/default/files/sicp/index.html


The expression inside a {list,dictionary,etc} comprehension is semantically the same as an anonymous function.


This is a great question with many great answers. The process of discovering those answers will illuminate huge swaths of the computer science landscape beyond the tiny, relatively insignificant patch that people call "programming."

You are about to embark on a great adventure. I wish you luck on your quest!


I'm sorry but this article is mostly not Scottish. Lambdas in Python are an old non-issue. They are what they are.

I like 'em because you can do "stupid python tricks" like Church numerals: https://en.wikipedia.org/wiki/Church_encoding

    plus = lambda m: lambda n : lambda f: lambda x: m(f)(n(f)(x))
    succ = lambda n: lambda f: lambda x: f(n(f)(x))
    mult = lambda m: lambda n: lambda f: m(n(f))
    exp = lambda m: lambda n: n(m)

    c0 = lambda f: lambda x: x
    c1 = lambda f: lambda x: f(x)
    c2 = lambda f: lambda x: f(f(x))
    c3 = lambda f: lambda x: f(f(f(x)))
    c4 = plus(c1)(c3)
    c5 = plus(c2)(c3)
    c6 = plus(c1)(c5)
    c7 = succ(c6)
    c8 = succ(c7)
    c9 = c3(succ)(c6)
    c10 = mult(c2)(c5)
    c11 = succ(c10)
    c12 = mult(c3)(c4)
    c64 = c3(c4)
    c256 = exp(c2)(c8)


I tend to abuse lambdas plus itertools/functools to write "obfuscated" Python for fun. For example...

* FizzBuzz with no control-flow keywords: https://github.com/ubernostrum/interviewer-hell/blob/master/...

* Detecting if a number is a perfect square as a one-line lambda: https://github.com/ubernostrum/interviewer-hell/blob/master/...


Ah, those are great.

I can't find it now but some daredevil wrote a Lisp interpreter in a single Python expression. A work of art.


What do you mean by "not Scottish"?


It's a Mike Myers bit from SNL. He plays a Scotsman who insists, "If it's not Scottish it's crap!"

I mean it as a self-deprecating way to decry the quality of the post without being too harsh. The author of the blog post had good intentions, but in my opinion he should have put it off for another decade or so, to gain perspective.


https://en.wikipedia.org/wiki/No_true_Scotsman

>No true Scotsman or appeal to purity is an informal fallacy in which one attempts to protect a universal generalization from counterexamples by changing the definition in an ad hoc fashion to exclude the counterexample.

edit: ok he didn't mean it that way!


The crippled nature of Python's lambdas are a big weakness of Python. They sit in a no-mans land of providing half a solution but being too crippled to actually to clarity / improve programming style.

I much prefer languages that go all the way and give you full integration like Ruby, ES6 and Groovy. For example, in Groovy closures just blend completely into the language:

    [1,2,3,4,5].grep { it > 3 }.collect { it * 5 }
Compare to python is almost incomprehensible:

    map(lambda y: y * 5, filter(lambda x: x > 3, [1,2,3,4,5]))
But breaking this into functions reduces to an almost silly level of verbosity. Of course, idiomatic python would write this as a list comprehension, but that is just proof that the lambda construct is broken, and the list comprehension is a band-aid and doesn't scale up to more complex scenarios.


Your Apache Groovy example doesn't work in Jenkins Pipelines, which cripples Groovy so the functional methods such as `collect` don't work.

You wrote elsewhere that

> I won't miss Grails because I think that, a bit like Gradle, it is an antipattern use of Groovy - needlessly applying its dynamic features where they are not even required

Groovy was designed as a dynamic language to complement Java. Its static compilation was tacked on much later, whereas Kotlin and Scala, like Java, were designed to be statically typed from the ground up, and they, unlike Groovy, also run on other platforms besides the JVM. If you're not going to use Groovy's dynamic features, you might as well use Java, Kotlin, or Scala.


    [x*5 for x in [1,2,3,4,5] if x > 3]
Though yes, I agree that lambdas in python are crippled. Maybe because list comprehension exists which replaces most uses of lambda, its just that they forgot about all other uses.


Great post, love all the examples categorizing types of misuse (at least what he deems as misuse, and I won't argue because I have a lot to learn by reading through this).

Reminds me of a story.

A few years ago Peter Norvig (Google) posted a "Show HN" posts with one of his amazing notebook excursions where he went through nice little problem, building up a library of functions to make a solution. I think this has happened multiple times.

One such time, one bit of code used a lambda expression, and I was new to Python, and wondered in a possibly-slightly-whiney comment (as I recall) whether it was the most readable way to do things.

Within an hour or so, his post was silently updated and the lambda keyword was gone. I don't remember the nature of the rest of the change, whether it was just the one block of code or more, but the code was more readable to me, at least as a newbie, at the time. Not sure my comment had anything to do with it, but I was impressed that he did make the change in any case. I have never worked with him, but my respect for him grew with that tiny little incident.

BTW in case anyone feels the urge to explain lambda expressions to me now, that's OK, no need, but thanks anyway.


The argument used by the author for 'abs' and 'min' can also be useful for all of the functions in the operator module:

https://docs.python.org/3/library/operator.html

In particular 'attrgettr' and 'itemgettr' are very useful functions to map over objects and sequences.


Is attrgetter recognized by static code checkers like mypy and pycharm? One of the benefits of lambdas is that arguments autocomplete even without manual type annotations.


oh nice I've been using itemgetter for sorted and things for a long time. didn't realize (but should have) there was attrgetter to match. will definitely come in handy


    def length_and_alphabetical(string):
        """Return sort key: length first, then case-normalized string."""
        return (len(string), string.casefold())
You just said the same thing three different ways. Not having to do that is exactly what lambda expressions are for.


The first two situations where he recommends avoiding lambda I would agree with. But when he assigns `colors_by_length`, that's a perfectly valid usage of lambda. It's actually an ideal example of why it exists. His `length_and_alphabetical` is not an improvement. The tuple unpacking is something that destructuring in lambda syntax would improve, as it did with JavaScript. In ES6 it would be:

    const points = [[[1, 2], 'red'], [[3, 4], 'green']]
    const points_by_color = points
      .sort(([point, color]) => color)
I know it's kind of futile to argue against a point using what-ifs, but personally I wish Python embraced this style of functional programming a little bit more and adapted its syntax to it, rather than coming across articles where people recommend against this style just because the language isn't (currently) as suitable to it.


Python 2.7 had tuple parameter unpacking, but it was removed in Python 3.

https://www.python.org/dev/peps/pep-3113/


Headline should actually be: “Stop abusing lambdas in Python: how you are doing it wrong and how you can do it better.”

This tutorial is not an argument against lambdas, but an argument against abusing Python lambdas.


The author uses this as an argument against lambdas:

> lambda expressions are an odd and unfamiliar syntax to many Python programmers

Well these python programmers now have an opportunity to learn this unfamiliar syntax. You don't avoid using parts of a programming language just because it's unfamiliar! To new Python programmers, a lot of the syntax is unfamiliar at first, especially if coming from a curly-brace language or new to programming in general.

While I like many of the ideas in the article (especially the use of standard library functions that already do what your lambda was trying to do), it seems to make some cases against using lambda where the justification isn't entirely valid. :/


I can't agree with this article, but I fear it may be a case of over simplified examples. For example, I generally prefer a simple lambda as a `key` to a named function I have to go find the definition of when I want to understand it and I find lambdas more readable than itemgetter at al. Ofc, pass around normal functions and use generator expressions when you can.


I don't see how anyone could have a hard time using lambdas... they've been around forever, and most programming languages nowadays have them.

A good rule of thumb is given by the google python coding guidelines [1]:

"Okay to use them for one-liners. If the code inside the lambda function is any longer than 60-80 chars, it's probably better to define it as a regular (nested) function."

[1] https://github.com/google/styleguide/blob/gh-pages/pyguide.m...


Another post about discussing a functional programming tool in [insert language with a different primary paradigm here], another few threads from the functional programming fans stating how their preference is absolutely correct and how that paradigm is veritably better than any other.

It isn't, it's just a different way to approach programming.

As for the article, it isn't really a case against lambdas themselves, just against misusing them. It certainly didn't make a strong enough case for me to stop using them.


So in Python, imperative style is often more idiomatic than code that uses lambda expressions.

While in some functional languages (Haskell) and array programming languages (J), tacit or point-free style is often more idiomatic than lambda expressions (or equivalent).

There's something funny about that.


I somewhat agree but I think there are two things wrong with tacit style in Haskell:

1. It is usually taken too far (indeed I think flip is probably too far and (.).(.) is certainly too far. I’m also not a big fan of eg fmap.fmap)

2. (.) is the wrong way round. Code reads much better if it has type (a -> b) -> (b -> c) -> (a -> c)

In APL or J was I don’t have enough experience to say other than that I find it difficult to read but the puzzle is amusing


That comes down to what code is easiest to read, and that is different for different people. If playing with the APL family has taught me something, it's that what someone sees as a completely incomprehensible mess might be very readable and intuitive to another.

I personally have a hard time understanding almost any point-free Haskell code and deciphering a line of J might take me an hour. But as long as it's readable to the intended audience it's alright. Of course it isn't sometimes.


Many of the suggested alternatives to lambdas in the article are non-imperative though... they just don't use 'lambda'.


Certainly, and I do agree with most of the article. But the instances where lambdas are better replaced with an ordinary function don't use a different programming paradigm. My observation was about how different communities consider different paradigms idiomatic in situations where you get to choose.


> When I encounter a lambda expression in the wild, I often find that removing it improves code readability.

I have never seen a code that wasn't made clearer when moving from a three-liner non-lambda somewhere upper in the code, to a one-liner lambda used at the right place


I wouldn't say never, but I'd say the vast majority of the time this has been the case for me as well.


The whole article, and his reasoning, reminded me of PG's Blub programmer. Seems like a perfect example. (that being said, lambdas in Python are rather lame, though not for the reasons he wrote about)


> I get a lot of questions about lambda, I’m hesitant to recommend my students embrace Python’s lambda expressions.

Lambdas are in so many languages than it seems worth it to spend 10 minutes learning what they are.


I generally like coding with lambdas, but in Python one thing I hate is that they will capture a numeric loop index by ref and not by its current value. If you try generating some polynomials x -> x, x^2, x^3 ... in the "obvious" way you'll see what I mean, all polynomials will be the same as the last one after the loop or list compr is done.

Theres a workaround using default arg value to capture by val but its ugly and easy to forget and hard to "see" in review that its missing.

This is as much an issue of loops not generating "fresh" instances of their index vars as it is a problem with lambdas, to be fair, but it still sucks, esp in mathematical domains. Julia used to suffer from this but fixed it, I believe the same is true for C# too.


Yes! This caused one of the trickier bugs I've had (the functions were supposed to be almost the same anyway, but ended up identical). I thought of it immediately when reading this article.


Do you mind fleshing out your example with some actual code? I'm not quite following, and I'm completely lost by the time you talk about the workaround.


Sure, what I meant by the "obvious" way of making some polynomials is:

    polys = []
    for i in range(0,4):
        polys.append(lambda x: x**i)
But that actually just yields the same polynomial x -> x^3 in each element of polys array:

    polys[0](2)
    > 8
    polys[1](2)
    > 8
    polys[2](2)
    > 8
The same thing happens if we use a list comprehension:

    polys = [lambda x: x**i for i in range(0,4)]
The trick is to add a default argument value to the lambda, which catches "i" by value instead of by reference:

    for i in range(0,4):
        polys.append(lambda x, i=i: x**i)
    
And now we get the different polynomials we expected:

    polys[0](2)
    > 1
    polys[1](2)
    > 2
    polys[2](2)
    > 4
    polys[3](2)
    > 8


The first example could be written as:

  colors = ["Goldenrod", "Purple", "Salmon", "Turquoise", "Cyan"]
  normalized_colors = map(str.casefold, colors)


I don't write Python. But I do program in a few different languages (Ruby, Elixir, ES6, CoffeeScript, Rust...)

Is this a thing? Are Lambdas "Unpythonic?" or is the writer of the article totally off base?


The author is entitled to his perspective, but I would equate this article to JS articles titled "A framework author's advice to avoid frameworks". Its a trendy article because it takes something that a lot of people use and describes its weaknesses. Fun to read and argue with, but not a mainstream idea.


Guido van Rossum actually considered removing lambdas in Python 3. See https://www.artima.com/weblogs/viewpost.jsp?thread=98196

Though it didn't happen in the end: https://softwareengineering.stackexchange.com/a/252162/29892...


Law of headlines applies to the headline giving me a command too, I've noticed.


The benefit of purely functional programming is that side effects can be managed explicitly and checked by the compiler, speeding up code refactors and making them safer. Using high order functions to save a few lines of code is not the big benefit of functional programming and is more a superficial feature more akin to syntax sugar. Same with list comprehensions. This seems to be the case in many hybrid languages that borrow superficial purely functional language features.


It seems like if you want the expressiveness of the simply typed lambda calculus, higher order functions are more than just syntactic sugar.


I've struggled with this balance for a while.

I've come down to the view that _defining a function using `def` syntax adds structure to a program_, even when it's not desired. It's multi-line, includes an indent and a keyword. It look more like creating a `Class` than a `list` for example.

As a result, in order to define a function _without_ adding structure people fall back on lambdas.

I think the best resolution would be to embolden lambdas with some of the benefits of functions - e.g. multiple lines


    sorted_numbers = sorted(numbers, key=abs)
Need to remember that, I'm sure I've committed that sin!


You can pry them from my cold dead hands. If functions are first class objects, which they are, then they should not necessarily have a name, just like not every object created during runtime is at some point assigned to a variable.


for the record, the 'useless function call' paragraph is known as eta-conversion in the FP world. see: https://wiki.haskell.org/Eta_conversion

First line says:

    An eta conversion (also written η-conversion) is adding or dropping of abstraction over a function. 
    For example, the following two values are equivalent under η-conversion:
      \x -> abs x
    and
      abs


Python lambda expressions have the same problem that Fortran statement functions have, that's why Fortran statement functions were abandoned a few decades back. Not much new here.




Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: