
Origins of Python's “Functional” Features (2009) - tosh
http://python-history.blogspot.com/2009/04/origins-of-pythons-functional-features.html
======
tommikaikkonen
While it's far from a functional language, I find that it's possible to build
functional style abstractions without too much headache. The iterator protocol
is so well supported by the language and the standard libraries that using
map, filter, itertools, generator expressions & comprehensions gets you a long
way. You get the rest of the way by writing any specialized combinators you
need and then imperative code to tie things together. You can build up a lazy
pipeline of data transformations that works pretty well.

There are some gotchas where you need to explicitly copy (itertools.tee)
iterators when you have multiple consumers, and be careful with mutable
values, but it's manageable.

It would be nice to use curried functions as functools.partial gets verbose,
but at that point you're far from Pythonic code.

~~~
maweki
The state of a generator is always a problem that trips me up the most when I
try to do FP-Python. In general a function shouldn't care whether the input
was a list, set or generator. While iterating over it (or yielding from it),
we always mutate some internal state of the iterator.

One solution would be, I think, to always pass tee-d copies of iterators and
never(!) any iterator like it is (because you don't know whether next() has
side effects or not). The other solution would be to not ever do anything
lazy. One could also always consume all iterators fully. That way any
generator re-use would be apparent immediately.

Some function that moves a generator forward just one or two items (something
with the semantics of findFirst, for example), can subtly introduce bugs.

~~~
xapata
Make your functions consumers. Don't pass tee'd iterators in, but tee after
receiving if necessary.

I don't see how find_first would cause any more problems than filter or sum.

~~~
Tulip68
That is a very bad idea in general and will lead to enormous memory leaks when
large tee'd iterators get out of sync with one another.

See here:

 _This itertool may require significant auxiliary storage (depending on how
much temporary data needs to be stored). In general, if one iterator uses most
or all of the data before another iterator starts, it is faster to use list()
instead of tee()._

[https://docs.python.org/2/library/itertools.html#itertools.t...](https://docs.python.org/2/library/itertools.html#itertools.tee)

~~~
xapata
Eh? I discouraged tee'ing iterators, or at least I thought I did.

------
fermigier
I have started a few weeks ago an index of useful resources (docs and code)
related to functional programming in Python:

[https://github.com/sfermigier/awesome-functional-
python](https://github.com/sfermigier/awesome-functional-python)

I'm going to include this blog post as well.

Pull requests are welcome, BTW.

------
wheaties
I never understood his desire to remove 'reduce' from Python. I would love to
hear a cogent reason as to why we should have to write a for-loop for
everything 'reduce' like. In other languages, 'reduce' (akin to 'fold' for
non-Python people) is one of the things you reach for daily.

~~~
d0mine
It is still available as functools.reduce

An explicit loop is easier to read as a rule than the corresponding reduce()
code. See [http://stackoverflow.com/questions/15995/useful-code-
which-u...](http://stackoverflow.com/questions/15995/useful-code-which-uses-
reduce-in-python)

~~~
jghn
"Easier to read" for some people, I think you mean

------
StClaire
I love the functional aspects of Python. It's nice to break the monotony of
loops every once and while.

~~~
Alex3917
I dunno. I love the functional parts of Groovy. But with Python I generally
only break out the functional stuff when it's a choice between one unreadable
line of code with a comment or else six unreadable lines of code. (E.g. when
parsing an API response with weird and arbitrary nesting.)

~~~
foota
Which is supposed to be functional in that example?

~~~
Alex3917
The one line would be functional. Here is a good example:

list(filter(lambda x : ('widgets' in x), mixed_widgets))[0]['widgets']

I almost always go for more lines of readable code rather than less lines of
unreadable code. But in cases like this, you can either write that as one line
of garbage unreadable code or six lines of garbage unreadable code. So I'd
rather just leave the overall codebase more dense to make it easier to
understand the program flow, and then leave a comment explaining what that
line is doing.

~~~
aftbit
You could also use the list comprehension form of that:

    
    
        [x for x in mixed_widgets if 'widgets' in x][0]['widgets']
    

More readable? I dunno. More "Pythonic"? Definitely.

Related: I wish the list type in Python included an analogue to dict's
".get(key, default)" operation.

~~~
m_mueller
definitely more readable - it describes in basic english words what it's doing
and it's basically the same as mathematical set notation. The main question to
me is the performance implication though. Are generator expressions doing the
iteration at C level like map and does that give performance parity then? What
about branching - am I correctly assuming that filter is always faster than
comprehensions or generator expressions with if-parts?

~~~
clusmore
Must watch:
[https://www.youtube.com/watch?v=OSGv2VnC0go](https://www.youtube.com/watch?v=OSGv2VnC0go)

I believe that the list comprehension will be faster than filter, but as
always, any time you replace readable code with unreadable code for
performance reasons, you damn well better time it.

~~~
viraptor
~4.5 times faster with comprehension.

    
    
        In [11]: timeit.timeit('''list(filter(lambda x : ('widgets' in x), mixed_widgets))[0]['widgets']''', '''nw={'abc': 'def'};w={'widgets':'s'};mixed_widgets=[nw]*100+[w]+[nw]*100''', number=100000)
        Out[11]: 2.6956532129988773
    
        In [12]: timeit.timeit('''[x for x in mixed_widgets if 'widgets' in x][0]['widgets']''', '''nw={'abc': 'def'};w={'widgets':'s'};mixed_widgets=[nw]*100+[w]+[nw]*100''', number=100000)
        Out[12]: 0.5911771030077944
    

But not generating the list at all is still going to be faster (with bigger
gains for bigger data)

    
    
        In [13]: timeit.timeit('''next(x for x in mixed_widgets if 'widgets' in x)['widgets']''', '''nw={'abc': 'def'};w={'widgets':'s'};mixed_widgets=[nw]*100+[w]+[nw]*100''', number=100000)
        Out[13]: 0.3324074839911191

~~~
m_mueller
I guess my next question is then - why is anyone using filter/map instead of
comprehensions or even generator expressions? Familiarity when coming from FP?

~~~
clusmore
Well, for one I believe they pre-date comprehensions. Having them as functions
is also occasionally useful for partial application, e.g.

    
    
      from functools import partial
      to_strings = partial(map, str)
      # vs
      def to_strings(seq):
          return (str(elem) for elem in seq)

~~~
m_mueller
I can see it together with partial, yes, that's when it can become a bit
cleaner. Another reason why I use map is when I want to use multiprocessing or
multithreading (with IO heavy functions). But on a fine grained level of code
I find it really hurts readability compared to comprehensions.

------
mangecoeur
I have sometimes wondered why python used 'lambda' for anonymous functions
rather than just... well literally anonymous function - normal function syntax
without the name. Could have been something like

`def (x,y): x+y`

This makes it sound like that was just how it was implemented. Nothing
fundamentally wrong with lambda i guess

~~~
nerdponx
I suspect this is because

    
    
        def f(x, y): x+y
    

is valid one-line syntax for a _regular_ function definition (that in this
case computes `x+y` and returns `None` becaus Python does not have implicit
returns). Maybe the parser could distinguish between them by saying that "if
the function definition is all on one line _and_ the function is missing its
name, it's a _lambda_ ":

    
    
        def f(x,y): return x+y
        g = def x,y: x+y
        assert f(3,4) == g(3,4)
        assert f != g
    
     That said I have no 
    

Nothing gives me the willies in mathematical code like writing out Greek
letters. I don't know why, it just does. The fact that the otherwise excellent
Stan ([https://mc-stan.org](https://mc-stan.org)) only accepts ASCII variable
names is depressing. I would love to be able to actually write `map(λ x: x+7,
range(5))`. I think a big reason why people don't use `map`/`reduce`/etc. more
is that writing anonymous functions is still verbose (`lambda` is just too
long a word) and relatively hard to read compared to list comprehension.
Better still a syntax like `x -> x+7` would be ideal, since I'm pretty sure
`->` would be a unique syntax element.

