
Magical, Mystical JavaScript Transducers - ecbu
https://jrsinclair.com/articles/2019/magical-mystical-js-transducers/
======
donut
Impressive amount of code and words. I think I'd think twice before approving
this in a code review, though.

Why is this not the obvious solution to the stated problem?

    
    
      function avgPopularity(slang) {
        let sum = 0;
        let count = 0;
        for (item of slang) {
          if (item.popularity > 0) {
            sum += item.popularity;
            count += 1;
          }
        }
        return sum / count;
      }

~~~
southerndrift
Yours is the obvious solution but the problem is a simple example to help
people understand the more complex idea of transducers. It's for situations
where the code inside the for loop is so complex that it would be nice to
organize it with functional programming principals.

The problem with functional programming is that the obvious approach is not
efficient since it passes the array several times. So the goal is to make the
functional approach as efficient as the imperative for loop.

Now if you do that by hand, you end up with some messy code. So the article
shows how you can combine the operations with a helper library in a structured
way.

That said, it kind of reminds me of The Evolution of a Haskell Programmer [0]

[0]
[https://www.cs.utexas.edu/~cannata/cs345/Class%20Notes/10%20...](https://www.cs.utexas.edu/~cannata/cs345/Class%20Notes/10%20Haskell%20Programmer%20Evolution.html)

~~~
cousin_it
For me it's the other way around. A for loop is a general purpose solution
that changes continuously with the problem: whenever I need to gather some
extra data, just add some hairs to the for loop. The time complexity is
obvious, so is the space complexity (how much data is in memory at any given
time), and it's easy to pause and debug. But if you have a bunch of FP
combinators and recursion schemas, when the problem changes slightly, you have
to unfold the whole origami crane and re-fold it carefully again. It's what
James Hague called a puzzle language.

The puzzle nature of FP can sometimes prevent the simplest solution to a
problem. For example, try writing a function that accepts a list of N numbers
between 1 and N and returns its histogram (list of counts of each number).
There's an imperative solution in O(N) time with one for loop. But in FP, I'm
not sure O(N) can be achieved, and even O(N log N) requires tree-like data
structures that aren't needed in imperative.

~~~
aurelwu
That seems to be such a basic task. Please take it not as me not believing it
but could you provide some source about there not being a known O(N) solution
for that problem, I'd like to know what the hurdles are and why there might be
no solution at all or why finding one is so difficult. (I tried googling it
but failed to find something useful)

~~~
cousin_it
The hurdle is that when you're going through the input, you need to update the
output out of order. An imperative array can be updated out of order in O(1)
time, but FP data structures can't do that. Strict vs lazy doesn't seem to
help. I have no source, but I've given this problem to some strong Haskell
programmers and they couldn't solve it (without using escape hatches like ST).

~~~
mrgriffin
As a professional Haskeller who solves numerical computation problems I
wouldn't call ST an escape hatch. You're right that I can't think of a "pure"
way to compute a histogram, but I doubt anyone uses Haskell specifically to
prevent all mutations but rather to carefully control where mutation can
occur[1]. I'd wager referential transparency is "good enough purity" for
almost all tastes.

[1] Of course we have plenty of other reasons to use it, but I digress.

------
chunkstuntman
Every time I see this blog I'm extremely interested in the content, but the
stylized presentation is so jarring and tough to read. I doubt this is an
original qualm and I'm fully able to switch to reading mode in Firefox to
mitigate this problem, but frankly it's off-putting.

~~~
noir_lord
On the other hand I have zero issues with the design, like it aesthetically
and find it a refreshing change to see a blog clearly designed to be
functional and refreshingly different.

~~~
Waterluvian
I wouldn't describe it as "functional" given the unnecessary font and
background.

------
SigmundA
I am somewhat confused here, I primarily do C# so I am used to IEnumerable and
Linq for doing this kind of stuff and you would just do
.Where().Select().Average() without worrying about intermediate memory usage.
In C# you do this stuff with millions of results from a database (although you
would do in the db if you can, but might be a flat file too).

Linq with IEnumerable doesn't necessarily create intermediate list (js arrays)
in memory unless it has to like say .OrderBy()

So I basically assumed js was the same since it has generators but its seems
map and filter etc don't support them yet?:

[https://dev.to/nestedsoftware/lazy-evaluation-in-
javascript-...](https://dev.to/nestedsoftware/lazy-evaluation-in-javascript-
with-generators-map-filter-and-reduce--36h5)

So then you get a solution like this that looks to me very confusing vs a
fluent style.

Please tell me I am missing something

------
bjoli
I recently wrote a srfi (scheme request for implementation) for transducers,
if anyone is interested. The code is guile-specific, but is easily portable to
other schemes. It is still a draft, so it is bound to change:
[https://srfi.schemers.org/srfi-171/srfi-171.html](https://srfi.schemers.org/srfi-171/srfi-171.html)

------
Waterluvian
In almost all cases you don't need to worry this much about performance. Just
traverse the array three times and move on. Go measure and improve if it
becomes an issue.

------
blunte
It's so depressing that JavaScript is growing in use, especially when I see
articles like TFA. I hate that momentum has allowed a clusterf*ck of a
language to become the standard for front end and now one of the top 3 for
back-end.

Here's Clojure's explanation of transducers:
[https://clojure.org/reference/transducers](https://clojure.org/reference/transducers)

The whole concept is explained more clearly and with far fewer words and code;
and the final result is pretty, or at least clean and tidy.

~~~
Waterluvian
I really disliked how talkative this author is. So many unnecessary sentences.
But I don't think that has anything to do with the language.

------
talkingtab
The concept of transducers - I think of it is a conveyor belt of functionality
- is great and I periodically run into problems where having an easy to use
JavaScript transducer system would be great. Right now I'm processing sets of
markdown files, reading shortcodes from them, generating html, applying more
shortcodes, adding variables and then using mustache ... It is just a conveyor
belt, but with lots of little odds and ends - perfect for a transducer,
especially with async.

Ramada looks interesting ...

------
sametmax
Isn't it exactly the use case of generators ? I though js had the yield
keyword by now.

------
tomxor

      function isFound(item) {
          return item.found;
      };
    

Every time I see this pattern in code (one line accessors) I have a sinking
feeling in my stomach in expectation of whats to come.

This is the best example of a forced pattern.

------
Veedrac
I recommend ignoring transducers unless you know you really need push-based
iteration, which probably isn't the case. Generators are generally a superior
approach otherwise.

------
mbostock
I know this is cheating and missing the point, but using d3-array:

d3.mean(victorianSlang, d => d.popularity)

This implicitly ignores invalid values (null, NaN or undefined) after coercing
to a number.

------
fulafel
Related: Transducers for Python and related HN discussion
[https://news.ycombinator.com/item?id=10591118](https://news.ycombinator.com/item?id=10591118)

I don't know how JS generators work but Python users seemed to think Python's
generators cover most transducer use cases.

------
mtarnovan
Correct me if i'm wrong but if you use something like Elixir's Stream you get
composability for free and don't need transducers.

