
Reducers, transducers and core.async in Clojure - ingve
http://eli.thegreenplace.net/2017/reducers-transducers-and-coreasync-in-clojure/
======
whalesalad
Disclaimer: as a Clojure developer and functional programming evangelist, I do
not want to be too critical of this article. It's a really great post about
some incredibly valuable and advanced Clojure features.

That being said, it really bothers me when functional programming evangelists
write out such horrible Python code examples. It makes Python look like it's
_not_ a functional language. It perpetuates the idea that Python won't let you
write elegant and stateless programs that behave in a functional manner.

If the author had not mentioned Python and had instead used it as pseudocode
to represent the entirely non-functional way to write something, I would have
been okay with it. But calling out Python specifically is just incorrect!

I have rewritten the code in a very pythonic manner that illustrates the
functional capabilities of the language:

    
    
        even = lambda x: x % 2 == 0
    
        # alternatively, depending on your religious beliefs:
        def even(x): return x % 2 == 0
    
        def process(seq):
          return sum(x + 1 for x in seq if even(x))
    

Note the (parens) for the comprehension instead of [brackets], which creates a
generator. The above code is lazy (in Python 3)!

And if we fire up a python repl and play around:

    
    
        In [2]: process(range(10))
        Out[2]: 25
    

I really love Clojure and use it daily, but I find the Python version to be
far more legible. It reads more like english.

Anyway, the moral of the story here is that you can do immutable/functional
programming with regular ol' Python.

~~~
Veedrac
Please use def over lambda when feasible (which is most of the time); using
lambda needlessly hurts debugability and saves a total of three characters.

    
    
        def even(x): return x % 2 == 0

~~~
eriknstr
Furthermore I would prefer

    
    
        def even(x): return not(x % 2)

~~~
scott_s
I would not. That's incidentally depending on non-zero numbers to become True,
and zero to become False. I would prefer to think of the operation as "is x
modded by 2 equal to zero?" rather than "does x modded by 2 become true when
negated?"

~~~
Veedrac
Though you might find core developers disagreeing with that sentiment.

[https://stackoverflow.com/a/3175293/1763356](https://stackoverflow.com/a/3175293/1763356)

~~~
scott_s
I do find it interesting, but it doesn't change what's easier for me to reason
about.

I suspect that it will be easier for core Python developers rather that
general Python programmers because they will be more intimately familiar with
the conversion rules. I would be more comfortable with such a construct in C
or C++, because I am more confident in those conversion rules. But, even
though Python's are quite similar, I had to check myself before making my
comment, because I knew they were _similar_ , but I was not sure what they
were. The situation in the stackoverflow comment is also not quite the same:
asking a question about the relationship of numbers (is x modded by 2 equal to
zero) is a special case of using integers in a boolean context. Saying it's
Pythonic to use an integer in a boolean context does not necessarily mean it's
_best_ to always do so.

------
raspasov
I have probably read most transducer explanatory blogs that ever existed and
watched many, many talks on transducers, etc. I have read books about them (as
in book chapters about them). I use the built-in transducers almost every day
and I consider them an essential tool. I have written the occasional
transducer myself for certain purposes.

This is the best transducer explanation and breakdown I’ve seen!

Very well structured and IMO easy to follow for anyone who understands Clojure
code already. Great explanation and progression that builds towards the full
transducers picture! (edit: typos)

~~~
smnplk
You need to checkout ClojureTutorials channel on youtube, where Timothy
Baldridge goes very deep into transducers. But the videos are not free, I
think there is a small subscription.

~~~
raspasov
(I have subscription for Timothy’s tutorial videos) I have watched most of
them as well some time ago, they are great also!

I really like this one because of the “first principles” approach of starting
with reduce and building from there.

------
piercebot
Something I found interesting when playing with the example code is that,
while folding in parallel certainly speeds up the execution for realized
sequences, it may not make sense to realize a sequence that you're only going
to use for one operation:

    
    
      user=> (require '[clojure.core.reducers :as r])
      user=> (def s (range 9999999))
    
      user=> (time (r/fold + (r/map inc (r/filter even? s))))
      "Elapsed time: 450.318641 msecs"
      25000000000000
    
      user=> (time (r/fold + (r/map inc (r/filter even? (vec s)))))
      "Elapsed time: 463.632907 msecs"
      25000000000000
    

Note the `(vec s)` instead of just `s` in the second timed run above

    
    
      user=> (def sv (vec s))
      user=> (time (r/fold + (r/map inc (r/filter even? sv))))
      "Elapsed time: 156.731683 msecs"
      25000000000000
    
    

Far from a ~3x speedup (in these contrived examples), realizing the sequence
in-line yields approximately the same performance as if it was operated on
lazily.

Something to keep in mind if you're trying to optimize your Clojure. This is
still the best resource I've read on reducers/transducers :)

[edit: code formatting]

~~~
wellpast
The performance disparity is even more dramatic on my machine. What's the
reason for that - is it b/c of concurrent contention during vec/realization?
Or ???

~~~
wellpast
For posterity, I just answered my own Q. Performance overhead was just the
cost of the linear cost of vec realization as you pointed out. At first it
seemed like I was seeing additional overhead.

------
WaxProlix
Possibly stupid question:

At the top of the article, we see `(reduce + (map inc (filter even? s)))` --
here, `reduce` is a two-place predicate, taking a function and a collection.

Later, `reduce` is taking a function, a 'base case' [], and a collection --
it's a three-place predicate, more in line with a signature for `fold` than
what I'm used to seeing for `reduce`.

Is this a clojure specific overload thing? I'm not at all an expert in FP or
anything so it might be fairly standard.

~~~
dragonwriter
It's fairly common in functional programming languages for functions to be
differentiated both by name and arity (reduce/2 being different from
reduce/3.)

~~~
WaxProlix
I feel like this is fairly opposed to things like partial application and
currying in general, no?

~~~
dragonwriter
It has implications for the viable syntaxes for partial application and
currying, but is not opposed to it.

For instance, you could use a distinct sigil for an omitted argument for
partial application.

~~~
WaxProlix
Good point, thanks.

