
Clojure transducers from the ground up: using them in practice - ghosthamlet
https://labs.uswitch.com/transducers-from-the-ground-up-the-practice/
======
ulber
Transducers are also familiar from automata theory. There’s a recent line of
research into automata and transducers with symbolic representations for
transitions. I’ve studied using them to apply fusion to pipelines of stream
transformations [1]. Here’s talk I have for a related paper [2] and this talk
by Loris d’Antoni gives a general overview of symbolic automata and
transducers [3]. In a few weeks I’ll have a talk up that summarizes the last
three papers I have on the subject.

[1]
[http://dx.doi.org/10.1145/3062341.3062362](http://dx.doi.org/10.1145/3062341.3062362)

[2] [https://youtu.be/0IL60qJEKLU](https://youtu.be/0IL60qJEKLU)

[3] [https://youtu.be/ca9IF-7nSOA](https://youtu.be/ca9IF-7nSOA)

------
BoiledCabbage
From reading this it seems like clojure transducers are the same thing as linq
expressions over IEnumerable. Is there any additional functionality they
provide?

~~~
arijun
One thing I _believe_ is not possible with linq is you can pass a transducer
to a function (since a transducer is just another function), which can then
insert it into a pre-existing chain of transductions.

~~~
Veedrac
The analogue of a transducer is an iterator transformation, aka. something
like `IEnumerator → IEnumerator`, which is also just a function.

------
twic
So are transducers basically this Java, but in Clojure:

[https://gist.github.com/tomwhoiscontrary/e41d52e9c98a706552b...](https://gist.github.com/tomwhoiscontrary/e41d52e9c98a706552be66f8a5f2fc17)

?

I've done that in terms of Consumer, but it might also be possible, and
perhaps more elegant, in terms of flat-mapping functions (Function<T,
Stream<T>>).

If i've understood right, then from a Java point of view, transducers are an
attempt to reify the various operations on streams - mapping, filtering, etc.
To do that, you have to figure out a type which is general enough to model all
those operations.

The need for a general type is where you get the perilously-close-to-general-
abstract-nonsense "a transducer is a transformation from one reducing function
to another". Here, "a reducing function" is what stands in, in a very oblique
way, for the stream - it's ultimately a conduit for values, and transducers
take some existing conduit, wrap another layer of processing round it, and
give you the resulting conduit. So they're a bit like backward streams, i
suppose. If anything, it's like how you go around wrapping up layers of
OutputStreams when doing IO:

    
    
      new PrintStream(new BufferedOutputStream(new GZIPOutputStream(new CipherOutputStream(new FileOutputStream("file.dat"), cipher))));
    

Except instead of explicitly constructing wrappers, transducers know how to do
the wrapping.

~~~
phaedrus
I thought that transducers also abstracted away the _difference_ between an
input stream and an output stream? (Or the difference between pulling values
on demand versus pushing values.)

------
yomly
This is a small point, but to the author: in your first part of the
article[0], binding the function argument to f in map, and then binding the
head of the collection in filter was a bit confusing when scanning through...

~~~
reborg
Happy to review the code, but I'm having trouble understand which snippet
you're referring to.

------
exikyut
This is not related to electrical transducers, which (without any additional
scoping context) was what I thought this article was about.

Relabeling it "Clojure transducers" would help the title in every way. But
it's been up for 7 hours now, so we'll see if it changes.

~~~
ulber
Transducer is the standard term inside CS for this model of computation [1].
Often there are some disambiguating terms, such as finite-state transducer,
but this article does not limit itself to any particular kind. Of course
Hacker News has content from both fields so the overloading of the term is
unfortunate.

[1] [https://en.m.wikipedia.org/wiki/Finite-
state_transducer](https://en.m.wikipedia.org/wiki/Finite-state_transducer)

~~~
moomin
Transducers don’t really directly map to any existing CS construct. In fact,
Clojure transducers occupy a strange twilight world between “fold
endomorphism” and “stream transformer”. Most transducers are both. Some are
just one or the other.

------
todd8
The domain (uswitch.com) along with the word "Transducers" really threw me
off. It's about clojure not electricity!

~~~
reborg
Ops, didn't think about that! Added Clojure to the title. Thanks for raising.
(author here).

------
ssttoo
Reducing the font size (CMD -, CMD -) helps immensely :)

