
Generators Are Like Arrays - jkrems
https://gist.github.com/jkrems/04a2b34fb9893e4c2b5c
======
Animats
The Blind Men and the Elephant John Godfrey Saxe (1816-1887)

It was six men of Indostan To learning much inclined, Who went to see the
Elephant (Though all of them were blind), That each by observation Might
satisfy his mind.

The First approached the Elephant, And happening to fall Against his broad and
sturdy side, At once began to bawl: "God bless me! but the Elephant Is very
like a WALL!"

The Second, feeling of the tusk, Cried, "Ho, what have we here, So very round
and smooth and sharp? To me 'tis mighty clear This wonder of an Elephant Is
very like a SPEAR!"

The Third approached the animal, And happening to take The squirming trunk
within his hands, Thus boldly up and spake: "I see," quoth he, "the Elephant
Is very like a SNAKE!"

The Fourth reached out an eager hand, And felt about the knee "What most this
wondrous beast is like Is mighty plain," quoth he: "'Tis clear enough the
Elephant Is very like a TREE!"

The Fifth, who chanced to touch the ear, Said: "E'en the blindest man Can tell
what this resembles most; Deny the fact who can, This marvel of an Elephant Is
very like a FAN!"

The Sixth no sooner had begun About the beast to grope, Than seizing on the
swinging tail That fell within his scope, "I see," quoth he, "the Elephant Is
very like a ROPE!"

And so these men of Indostan Disputed loud and long, Each in his own opinion
Exceeding stiff and strong, Though each was partly in the right, And all were
in the wrong!

~~~
glibgil
"An Elephant is soft and mushy", said the man a few feet behind the animal.

------
ObviousScience
Generators don't have either memory locality or fast indexing, which are
basically the two defining features of an array in virtually every programming
language.

They're nothing like arrays, except that you can iterate over them. That is,
they're no more like arrays than trees, lists, and other iterable data
structures are like arrays, which is to say, not particularly.

~~~
jkrems
I'm not claiming they are like arrays as in "they have the same properties as
arrays". I'm not talking about complexity of operations or even availability
of operations. In the very beginning I name 3 categories. One of which is
"data structures". And sets, lists, arrays, maps, etc. all are classical data
structures. You can even - to some degree - interchange one for the other. It
might not be efficient but they can simulate each other. E.g. you can
implement a map in terms of a list, you can implement an "array-like" in terms
of a list, etc.. I'm talking interface here, not complexity.

But you can't implement a promise using an array. The array will never
"magically" come to life a couple of seconds after you last touched it and do
something. And that's the whole point of the gist.

~~~
pdubroy
You also can't implement generators using an array. There's a reason why
generators were added to ES6 -- they aren't just syntactic sugar.

~~~
jkrems
If they aren't just syntactic sugar - how to you think regenerator works? Show
me a generator and I show you perfectly fine, synchronous (though verbose) ES5
code doing the exact same thing.

~~~
pdubroy
Well, "syntactic sugar" usually refers to things that can be translated in a
syntactically local, structure-preserving way. I'm aware that generators can
be compiled down to an ES5 representation, but it requires non-trivial
analysis. Take a look at the example on their home page:
[https://facebook.github.io/regenerator/](https://facebook.github.io/regenerator/).

Do you also consider functions to be syntactic sugar? Because you can also
implement them in terms of a switch statement.

------
chrismorgan
This is the whole point of an iteration protocol—it’s not _supposed_ to be
that generators are like arrays, but rather that generators and arrays share
the property of being iterable. I’m imagining that the fuss over this now is
because JavaScript simply hasn’t been like this in the past, where languages
like Python have been. I do know that I’ve missed it when working in
JavaScript as every other language I use tolerably frequently has sound
iteration semantics. I’m certainly looking forward to the time when for..of
can be considered a baseline feature.

------
nostrademons
This gist is thinking at too high a level of abstraction. Sure, generators are
like arrays. Functions are also like arrays, because at the compiler level,
the function body is basically an array of statements. Well, technically an
array of basic blocks, each of which is an array of statements. Don't believe
me? Write some Lisp, and study how progn works.

The reason generators & coroutines are popular is because they allow
programmers to use the same user-interface they're familiar with - namely,
statements separated by semicolons, all sharing a common lexical environment -
to reason about async code that may yield back to an event loop. Promises and
callbacks do not have this property: they force the programmer to make the
continuation explicit. While under the hood a generator is just an array of
thunks and a common environment, the whole point of a high-level language is
to hide what's under the hood.

~~~
jkrems
> Promises and callbacks do not have this property: they force the programmer
> to make the continuation explicit.

This is exactly what I'm complaining about in the gist... :(

Generators != coroutines. You can use generators to simulate coroutines - but
only if you combine them with either Promises or callbacks (or another async
abstraction). Pretending like generators "replace" promises or callbacks
doesn't help anybody imo. ES7 async functions are a great example of what
happens when you embrace the fact that generators _need_ promises of some sort
to act as coroutines. You gain return values of functions - something that
kinda got lost when JS started using callbacks:

    
    
        async function getUsername(id) {
          let user = await getUser(id);
          return user.name; // returns a Promise<String>
        });
    

By pretending that generator-based coroutines "got rid of" callbacks and
promises, we are taking a huge step back. Even co is by now embracing the fact
that coroutines require more than "just" generators.*

(*) Unfortunate only that koa gained traction before co got that memo.
Otherwise it might ended up with a saner API than "mutate random properties on
this". Maybe even something crazy/revolutionary like "return a response".

~~~
nostrademons
You certainly don't _need_ promises for generators to act like coroutines.
Take a look at how Python 2.5 handles generators; yield is an expression, and
then the external interface to a generator is a stream with a send() method
you can use to resume the generator. (As another poster pointed out, a stream
is a much more precise description of the abstraction than an array - arrays
support constant-time random access, while streams are just ordered
collections of elements.)

I do think that combining futures/promises with generators gives a much
cleaner programming model. Take a look at how Python 3.4 handles generators &
async programming; here a generator is implicitly a Future, and 'yield from'
runs a future to completion, letting you compose coroutines regardless of how
many yield points they have.

But my point is that these are all programming _models_ , different ways to
present the underlying abstraction to programmers. It's silly to talk about
"the fact that coroutines require..." or "generators _need_ promises" or
"pretending that...", as if there were only one way to do async programming -
these are not universal abstractions, they are models to present the
underlying scheduler to programmers, and we choose between them based on how
convenient they are to programmers.

------
tree_of_item
The `yield` operator is entirely about control flow. I'm not sure how you can
see generators as being "like arrays" and not about control flow. Generators
are related to continuations, not arrays.

It's this extra flexibility in control flow that lets you write nicer
asynchronous code.

~~~
jkrems
Generators can definitely be used as a tool for control flow - just like
arrays. But the thing that actually provides the control flow part is not the
generator - just like `async.series` is providing control flow and not the
array you pass in. I certainly am not objecting to using generators as one way
of modeling flow. The only thing I said was "don't pretend that they somehow
replace Promises". Because they don't. The solution I favor is ES7 async
functions, which actually is defined in terms of Promises (async abstraction)
+ spawn (control flow) + generators (model/"data structure").

There's no way to use just generators and nothing else and get async control
flow going. That's not how generators work. And `yield` is not "about control
flow". It's about lazy sequences. And one of the many possible ways to use
lazy sequences is to model control flow, when combined with a control flow
library.

~~~
pdubroy
I believe it's pretty common in the programming language literature to refer
to `yield` a delimited control operator. E.g., see
[http://parametricity.net/dropbox/yield.subc.pdf](http://parametricity.net/dropbox/yield.subc.pdf),
which says "yield...turns out to be a delimited control operator of comparable
expressive power to shift/reset". A good hint that you're dealing with a
control statement/operator is if it's responsible for transferring execution
to another place in the code: e.g., `if`, `for`, `while`, `throw`, etc.

Saying that "yield is not about control flow, it's about lazy sequences" seems
like saying "functions are not about control flow, they are about producing
values".

------
LewisJEllis
"People keep putting generators, callbacks, co, thunks, control flow
libraries, and promises into one bucket."

Yes, and that bucket is called a Stream, as used effectively by FRP libs like
RxJS and Bacon.js, and by Highland
([http://highlandjs.org/](http://highlandjs.org/), made by the guy who made
async).

------
spion
Generators are factories of programmable lazy lists. You can give up
"programmable" if you e.g. use for-of on them. But if you don't, they're no
longer just in the "data structure" category.

I agree with the gist of the article though. Programmable lazy lists don't do
anything special to solve the async problem unless they're programmed to.

------
ajuc
You can use generators to make cooperative multitasking inside one thread. I
can'd do that with arrays.

Generators aren't a data structure - or if they are, then functions are a data
structure too. This isn't array, this is function:

    
    
        function doStuff() {
          return [
            fs.readFile.bind(null, 'hello.txt'),
            fs.readFile.bind(null, 'world.txt'),
            fs.readFile.bind(null, 'and-such.txt')
          ];
        }

------
raverbashing
It's a good idea to tell Python people then, since, from what's being said,
range and xrange are the same...

However I give it to them it's not the whole story, and things like
promises/callbacks are needed for async behavior.

Now, you could have an async yield.

~~~
ubernostrum
ES6 generators appear to lack some features of Python generators, which may
account for the JS-specific leaning here.

~~~
raverbashing
Interesting (some of this is explained here
[http://davidwalsh.name/es6-generators](http://davidwalsh.name/es6-generators))

