
Why Y? Deriving the Y Combinator in JavaScript - braythwayt
http://raganwald.com/2018/09/10/why-y.html
======
abetusk
For those of us who don't know what a Y-Combinator is or why it's useful, I
found a SO answer [1] that makes sense to me:

    
    
        A Y-combinator is a "functional" (a function that
        operates on other functions) that enables recursion,
        when you can't refer to the function from within
        itself. In computer-science theory, it generalizes
        recursion, abstracting its implementation, and
        thereby separating it from the actual work of the
        function in question. The benefit of not needing a
        compile-time name for the recursive function is
        sort of a bonus. =)
    
    

[1] [https://stackoverflow.com/questions/93526/what-is-a-y-
combin...](https://stackoverflow.com/questions/93526/what-is-a-y-
combinator/94056#94056)

~~~
amelius
Wondering if anybody here has ever used the Y-combinator in practice (and not
as an academic exercise) ...

(though I get that it can be a useful intermediate construct inside a
compiler)

~~~
billsix
It's not useful in practice. But defining recursion via a function is a good
mental exercise.

I'm unaware of compilers using the Y combinator. Perhaps you were thinking of
continuation passing style

~~~
taneq
Functional programming: Making you feel clever by allowing you to solve
problems that nobody else even knew existed, in order to let you do what
everyone else could do from the start.

~~~
billsix
Discussion of the Y Combinator on news.ycombinator.com should not be derided.

------
wcarss
I slightly wish the exponentiation function from the original article weren't
the more complex "faster" version -- it's not really relevant to the article,
and it makes playing through the calls in your head just a little bit harder
with a topic that already stretches your mind the first several times around.
Regular exponentiation is a fine example without the added fun of working out
why and how exactly this alternative form works.

That said, as always raganwald's stuff is a pleasant and informative read.

~~~
paulddraper
I disagree. It nice to have a non-FizzBuzz problem. I.e. code that you could
actually ship.

~~~
anonytrary
I disagree. If you are writing shippable code in an article, it's because the
article is about how to write shippable code. This article is not about that,
so writing shippable code risks detracting from the main purpose of the
article.

~~~
braythwayt
Exponentiation is almost never shippable code, most languages have an operator
or standard library for that.

~~~
paulddraper
For certain ranges of numbers.

But you could imagine this being part of, say, a "big integer" library. (Which
C, C++, PHP, JS all lack.)

~~~
ZeikJT
You will hopefully have to edit out JS someday:

[https://github.com/tc39/proposal-bigint](https://github.com/tc39/proposal-
bigint)

[https://developers.google.com/web/updates/2018/05/bigint](https://developers.google.com/web/updates/2018/05/bigint)

------
cakeface
It's nice to see an article like this every once and a while to remind myself
that there is so much about programming still to learn. I particularly liked
the javascript examples so I could play with them in my scratch pad. I am
still not sure I could describe what a Y Combinator is and where to use it,
but I'm much closer than I was 15 minutes ago.

~~~
arethuza
It's over 30 years since I was taught about the Y combinator and I still find
it amazing - it allows you to define recursion in a language which has no
concept of recursion or named functions - such as λ-calculus.

The fact that you then define this in terms of amazingly simple functions like
the S and K combinators just, in my opinion, adds to how wonderful it is.

~~~
guelo
In those 30 years did you ever encounter the need to use these combinators?

~~~
arethuza
Of course not - I haven't practically applied Turing Machines either. You'd
have to be mad to implement a functional programming language purely in terms
of S and K - which of course is what I did for my final year project and got a
1st and a prize!

What my encounter with the fundamentals of Computer Science did give me (apart
from the necessity of implementing things like garbage collectors) - was an
abiding interest and love of the mathematical foundations of computation and
with maths as something interesting rather than something to be endured.

That interest did lead to me doing postgrad research in a control engineering
group, which did lead to me co-founding a startup.

So did I ever apply S & K in a practical circumstance - no. Am I glad I took a
turn down the more mathematical side of CS - absolutely.

------
mFixman
There's a nice Haskell library for dealing with combinators, where the types
of most functions are intuitive enough to understand what they do:
[http://hackage.haskell.org/package/data-
aviary-0.4.0/docs/Da...](http://hackage.haskell.org/package/data-
aviary-0.4.0/docs/Data-Aviary-Birds.html)

The Y combinator isn't there and can't be defined trivially since Haskell only
allows non-recursive data types. However, there's a nice solution using non-
monotonic data types[1]:

    
    
        newtype Mu a = Mu (Mu a -> a)
        y f = (\h -> h $ Mu h) (\x -> f . (\(Mu g) -> g) x $ x)
    

[1]
[https://stackoverflow.com/a/5885270/305597](https://stackoverflow.com/a/5885270/305597)

------
markc
I find Matt Might's post on this topic (Y in JavaScript) the best I seen (even
though it's at least 8 years old by now):
[http://matt.might.net/articles/implementation-of-
recursive-f...](http://matt.might.net/articles/implementation-of-recursive-
fixed-point-y-combinator-in-javascript-for-memoization/)

It's clear, minimalist, insightful, and even has an in-page live code demo.
Bonus points for including fixed-points as a key concept, and memoization as
an extremely useful technique.

~~~
braythwayt
I've always liked that post, and I think it inspired me to do a similar thing.

That being said, I'm a big fan of decorators, so I expressed the memoization
using a decorator, rather than baking it into Ymem.

On the other hand, trampolining is the perfect application for a special-
purpose Y combinator, and thus the decoupled Trampoline.

------
golergka
While I enjoyed the subject matter, I can't help but feel that Y Combinator is
to functional programming as AbstractFactoryFactory is to object-oriented
programming. My inner geek revels at these abstractions, but my inner engineer
and team manager wants something simpler and more robust.

~~~
jacquesm
Even simpler?

~~~
golergka
Simpler in understanding and mental overhead. I want code in my projects being
written in a way so that it's maintainable by developers this business can
reasonably hire. Which means that they didn't read SICP or even Knuth, don't
have experience with functional language and overall are not the x10
developers everybody pretends to have.

~~~
braythwayt
I'm old enough to remember when we said the exact same things about OOP, then
about Ruby, and now we say them about Elixir, ClojureScript, and so forth.

I very much doubt anybody needs to have SICP at their fingertips, but it seems
like many techniques that we deride as being "too far out there" turn out to
be quite manageable by all sorts of programmers, it's just a matter of there
being enough resources to learn and a little motivation.

This article definitely does not prescribe hurling combinators at every
problem, but I will go on record and say that if there's somebody on your team
you think cannot understand this code, perhaps you should be a little more
optimistic and give them a chance.

Anybody who can figure WebPack, Gulp, Babel, closures, promises, async/await,
generators, &c. out can figure this out.

~~~
golergka
Oh, they can figure it out if they wanted to, definetly. And in about 10
years, they will. That's not my point.

My point is that today, when developers encounter this code while debugging
something that needs to be done yesterday, will not spend the time educating
themselves and will just ignore it or (even worse) __assume __they understand
it, which is even worse. I don 't think that I'm better than them - I just
acknowledge my own stupidity. And in my stupidity, in the heat of working on a
real project I want to encounter familiar technologies and abstractions that
by themselves are almost boring, rather than research new things.

At least when it's out of scope of our technology focus, the core of the
project that actually helps it be unique and earn a profit.

------
zelon88
I would argue this is Google's fault.

I tested the theory myself. I went to Boston recently and followed my GPS. It
took me over the Tobin bridge through solid bumper-to-bumper gridlock. When I
arrived at the gridlock my GPS said 30 minutes left. 45 minutes later I had
travelled about .5 miles and it still said 25 minutes left. Google was
completely wrong, but I swear it had me right where it wanted me the entire
time.

After getting fed up with traffic, I eventually turned my GPS off and got into
the right lane to head towards Cambridge. My logic was that GPS could navigate
from anywhere, to anywhere. I would simply go somewhere with less traffic and
try the GPS again.

Sure enough I made it to Edward Land blvd and turned my GPS back on. I had a
clear 15 minute drive with average traffic for the rest of my trip.

Google saw the traffic I was dealing with, and queued me right in behind it
all. It could have recalculated around it, but it legitimately "thought" I
only had 30 minutes to go. I guarantee it told all 500 other cars in front of
me the same thing.

So we were all following our GPS's, and our GPS's were taking all of us the
fastest route. It created a traffic jam and then optimistically pretended
there was no traffic jam.

If street lights were truly "smart", with cameras and machine learning to
maximize throughput, and GPS's programmed to load balance their own
congestion, this problem wouldn't be a problem anymore.

~~~
FridgeSeal
I think you commented on the wrong article lol.

~~~
arkades
Yet the topmost response.

------
skybrian
It's interesting to see how the Y combinator is derived, but it seems like
it's being oversold here?

I like the mockingbird better since it's less magical. Passing "self" as the
first argument seems idiomatic from an object-oriented point of view; this is
sort like how Python has an explicit "self" parameter. We can take the next
step and pass "self" explicitly as well.

The ability to change how a self-call is handled seems quite like overriding a
method.

~~~
yiyus
AFAIU the problem, from a theoretical point of view, is not that you need to
pass the self function, but how to define it. In pure lambda calculus, there
are no named functions, just lambdas, so you actually need the Y combinator
for recursion.

------
sjroot
See also: [https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Refe...](https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Reference/Global_Objects/Math/pow)

Joking aside, I thought this was a very approachable article. Thank you for
sharing.

------
maze-le
Nice. Very thorough article, that touches more concepts of functional
programming than just Y like tail recursion.

For someone who never had contact with functional programming, the concept of
the Y combinator can seem a bit... strange. But trying to figure out what it
is and does can be a real eye opener.

~~~
9dev
I always thought windows was the only exe opener

Edit: ah, you fixed the typo ;)

------
janci
This feels like Alice in Wonderland

~~~
braythwayt
Author here. I take that as a compliment.

Not everything is meant to be deployed to production.

~~~
DonaldPShimoda
Are you suggesting that I shouldn't be writing production code in the lambda
calculus?

...hm.

~~~
fooker
How about SK rewriting systems? ^_^

------
curryhoward
This is actually the Z combinator, which is the eta expansion of the Y
combinator. The Y combinator only works in languages with non-strict
semantics, but JavaScript is call-by-value.

~~~
braythwayt
This is included as a footnote in TFA.

------
seanmcdirmid
Is there something like the Y combinator for self types in F bounded OO
generic type systems? I just realized it’s the same kind of problem at the
type level. E.g.

abstract class FooImpl<Self extends FooImpl<Self>> { Self doIt() { .... } }

As a class with a self type, analogous to an open recursive function, and
then:

class FooFinal extends FooImpl<FooFinal> {...}

As it’s call.

------
gfaure
Excellent article, Reginald.

I followed it entirely with one exception: I can see that `maker(maker)` has
the right type, but why is it the right value? How do we know, as we fill in
the blanks, that this is the implementation that yields the right result?

~~~
braythwayt
This is the best comment!

I wondered this myself. I mean, I can test it and see that it appears to give
the correct result, but how do I _prove_ that it gives the correct result?

I didn't include it, but you can do reductions, successively replacing the
names with their expansions, and you see that it works out was we want.

And in Combinatory Logic, there really is nothing else. If an expression has
the correct "type," then it works. There's nothing except rearranging terms,
duplicating terms, and erasing terms.

But that being said, I don't know if that's a "proof," and I especially don't
claim that a proof about Combinatory Logic would say anything at all about
JavaScript.

------
oneweekwonder
So anybody up for a short snippet or explanation on how to use a
@decorator_function to implement a Y Combinator in Python?

------
danjoc
So we start out with an attempt to build an anonymous recursive function.
That's cool I suppose. But doing it in a language like javascript (or java)?
It turns out that tail recursion == stack overflow. Not daunted, the plucky
author decides that we can solve this particular issue by creating a closure
over stack variables and put them on the heap instead!

You know, if you want to use Haskell, use Haskell. Trying to shoehorn this
stuff into languages designed for loops instead of recursion just means
impressionable JS developers are going to see this and rush out to implement
it in some project to prove how smart they are. Then everyone suffers. The
implementation is slower because it goes to heap. The memory usage explodes
because it goes to heap.

Purely for illustration of a Y combinator, this is great. But there should be
a big red warning somewhere on the page as a reminder of the downsides of this
approach.

~~~
braythwayt
We actually don’t start out with an attempt to build anonymous recursion.

That is the goal of the Y Combinator in the Lambda Calculus and Combinatory
Logic, but here the initial goal was to decouple an anonymous function from
itself so that we could decorate it.

~~~
danjoc
Going out to main memory is orders of magnitude slower than hitting cache.
Even a stack overflow would be preferable to debug vs an out of memory error.
At least with the stack overflow, the stack trace pinpoints the problem.

Like I said, I have no problem with this article as an illustration of a
ycombinator for JS devs. The problem is it does not warn them sufficiently of
the very major drawbacks in using it. This is why we have Gate's law;
Developers using inefficient constructs because they're neato.

~~~
braythwayt
Well, given that the post links to two different discussion forums, I attest
that it’s not my responsibility to outline every single consideration of every
single technique, nor is it my responsibility to include the entire syllabus
of computer science to explain the background.

Instead, I outsource that to people like you. You voice your concerns here,
and people can read them and make up their own minds.

Speaking of making up their own minds, please explain in more detail how the
trampoline’s memory consumption grows unboundedly, to the point where it could
cause an out-of-memory error. That would be most interesting for the community
to ponder.

~~~
danjoc
Why don't we just illustrate the problem using the code from the article?

[https://jsfiddle.net/q59Ljeu7/](https://jsfiddle.net/q59Ljeu7/)

A loop is not just fewer LoC and easier, it is orders of magnitude faster than
the ycombinator decoupled trampoline. The first alert is nearly instant. The
second one takes almost a minute. My chrome dev tools are hard locked waiting
for the ycombinator to finish.

This is not something I would encourage JS devs to use in practice.

~~~
braythwayt
Well, refactoring isEven or exponentiation to a loop is faster. But wait, you
know what's even faster? Using built-in math primitives.

I wrote an entire article about refactoring tail-recursion to iteration and
linked to it in TFA. I've also written a Scheme interpreter that can perform
this optimization for certain functions on-the-fly. But arguing that the code
is slow is missing the point by a country mile, and then some. The article is
not arguing that the code is fast, or that you should always do this. I find
it amazing that every time a programming technique is discussed, people want
to worry about whether it belongs in production.

Is there no recreational programming any more? Or is it all about shipping the
next app to deliver food by electric scooter? My world would be depressingly
boring if the only things I thought about were the things I use at PagerDuty
every day.

Now as to what I asked you about:

What you said was that this implementation leaks memory in such a way that
using the trampoline it trades a stack overflow for an out-of-memory error.

I wish to understand that problem. Specifically, how does the decoupled
trampoline leak memory?

------
epx
Y cromossome :P

