
Escape from Callback Hell - ianbishop
http://ianbishop.github.com/blog/2013/01/13/escape-from-callback-hell/
======
Swizec
I honestly find "callback hell" a lot easier to follow and understand than the
vast majority of fixes everyone is coming up with.

They're just continuations, seriously, what's everyone's problem? You define a
function, it gets access to the current scope, it defines the rest of the
program flow.

If you feel like your code is nesting too deep, you define the function
elsewhere and just reference it by name. Then you don't get access to the
current scope.

Why is this so difficult to people?

~~~
pkulak
It's not difficult, it's just gross. And these promises don't solve the main
problem, which is that synchronous functions return the result, until at some
point you add some IO, so now it takes a callback. And everything that calls
it now has to take a callback. And hours later all you've done is add a
network call in some basic function but your diff looks like a total re-write.

~~~
MatthewPhillips
A function that doesn't do IO and a function that does IO sound like totally
different functions to me. Why aren't you writing a new function?

~~~
daleharvey
That isnt the point, its that you have some nice code that does |var x =
someBigFunction()| and at some point in the programs life, something inside
someBigFunction turns async, you have a whole restructuring to do.

~~~
MatthewPhillips
This is just not something that happens in real life. You know what functions
are going to be async up-front, and likely make them async even if you don't
know, just in case. Since all IO in js is async by default, having a code that
does a simple calculation and introducing IO into it is actually a very big
change that warrants the refactoring you'll be required to do.

~~~
rdtsc
> This is just not something that happens in real life.

You never change code in real life? You don't always know what functions are
going to be async up-front. They might become async later, 4 levels down, when
someone decides to do some IO. What happens then? Exactly. You have to ripple
that code all the way up to the top of the API.

------
etrinh
Good overview of jQuery Deferred and how to use promises (at least the jQuery
flavor). Promises (or futures) are a simple concept: an object-level
abstraction of a non-blocking call, but they're very powerful when you see
them in action. For example, the $.when method:

Let's say you have 3 ajax calls going in parallel. With $.when, you can attach
callbacks to arbitrary groupings of those ajax calls (callback1 runs when
ajax1 and ajax2 are done, but callback2 runs when ajax1 and ajax3 are done).

I first learned about promises in Trevor Burnham's excellent book Async
Javascript (<http://pragprog.com/book/tbajs/async-javascript>) and it is still
the best explanation of promises I've ever read. If you like this article and
are interested in reading further about promises or the asynchronous nature of
Javascript in general (both for browser and node.js), I highly recommend you
check out this book.

~~~
ianbishop
Awesome, I didn't know that such a book existed. Thanks!

------
daleharvey
I dont find that promises really help callback hell that much, they are useful
but in the case of doing a series of sequential async functions the result is
pretty similiar (the promises version is usually longer)

I got to write some firefox only code recently and added a delay to a function
by just adding

    
    
       ... some code 
       setTimeout(continueFun, 5000)
       yield;
       ... more code
    

it felt like magic, I dont like generators and would much prefer to see
message passing and blocking calls like erlang, but failing that it will be
nice to be able to use generators more regularly

~~~
thejsjunky
Promises are not an alternative to callbacks - they are a wrapper around them
that make working with them much nicer (though still a little painful as you
imply). They add composability which I think has wider implications than many
realize at first.

Consider that with things like jQuery deferreds I can start a request in one
function and pass it around adding handlers in others...something very hard to
do with the old CB model. Maybe later on you have some optional action that
you only want to occur if a previous action had completed. This sort of thing
is easy with deferreds. It makes bundling together disparate actions easier as
well as handling series of things as you mention.

Having an abstraction layer in there also has a lot of potential that is
(somewhat) untapped. There is a lot of possibility for library authors to add
excellent error handling or debugging capabilities in there - things that are
currently a bit painful with POC (plain 'ol callbacks).

I agree generators have strong possibilities in this area- I would note
thought that they are not an alternative to promises - the two are orthogonal,
and in fact they work quite well together. Just take a look at what some other
Mozilla folks are doing with <http://taskjs.org/> for example.

------
digisth
A great library for structuring your callbacks is "async":

<https://github.com/caolan/async>

I've only used it with node.js, but it's supposed to work in web browsers as
well.

It allows you to think a little more procedurally ("waterfall" is especially
handy here) while writing CPS code. Very good.

~~~
Jare
I have used it in browser games to load assets (images, sounds, data files)
before starting a new state (menus, gameplay, etc). It's a fantastic library.

------
pkulak
It's a real shame to have a language this high level, yet still have to go
through this much crap just to get things done. Manual memory management is
easier than this. But while including GC in the runtime has it's drawbacks,
there is no reason that a language can't just handle task switching for you
(like Go does, for example).

~~~
doktrin
IMHO the event model / callback spaghetti / what-have-you is tricky precisely
because it operates at a high level of abstraction.

Memory management, by comparison, is conceptually simpler because... well...
the concept _is_ simple. Allocating and de-allocating resources, while tricky
at scale, is something for which everyone (including non-programmers) probably
have existing mental models for.

Event driven programming, on the other hand, is a slippery high level concept.
There are relatively few analogues for it in the "real world", and therefore
requires additional mental gymnastics to internalize and understand.

Essentially, we need to 1) follow the execution pattern of event driven code
(annoying), while at the same time 2) "visualizing" or conceptualizing a
fairly un-natural manner of abstraction.

~~~
Swizec
You do event driven programming every time an alarm wakes you up or you set an
egg timer.

~~~
coldtea
And you do neural networks every time you think and aolve differential
equations every time you catch a baseball.

That doesn't make it any less difficult.

------
harshaw
Deferreds are cool although they have their own set of issues. Mainly, that
when you start chaining them there are situations where it can be a bit
counterintuitive what is going on. My background is the Deferred from Twisted
and Reimplemented in MochiKit.

You really need to read the Deferred implementation if you are going to use
it. Otherwise you are asking for trouble long term. Of course, the other issue
is that you may run into challenges explaining deferred's to your co-workers.
:)

Twisted explored some cool ideas where you basically would write asynchronous
code in an interative style using a blend of iterators and generators. Sadly
until Javascript has those capabilities in every browser (and not just
Firefox) I don't think it is possible.

~~~
jlgreco
Particularly Twisted's inlineCallbacks
([http://twistedmatrix.com/documents/current/api/twisted.inter...](http://twistedmatrix.com/documents/current/api/twisted.internet.defer.html#inlineCallbacks))
are _really_ nice. They are so nice, they actually completely offset all the
other unpleasantness I experiance from Twisted.

~~~
harshaw
Yup thats what I was thinking of.

------
spion
If you want this problem solved, vote on the generators issue in v8:

<http://code.google.com/p/v8/issues/detail?id=2355>

If v8 implements this, both Firefox and Chrome as well as node.js will have
yield, enabling libraries like taskjs [1] to be used.

From taskjs.org:

    
    
      spawn(function*() {
          var data = yield $.ajax(url);
          $('#result').html(data);
          var status = $('#status').html('Download complete.');
          yield status.fadeIn().promise();
          yield sleep(2000);
          status.fadeOut();
      });
    

[1]: <http://taskjs.org/>

~~~
ufo
Does anyone here have experience with using one of those X-to-JS compilers
that do a similar thing but without requiring those Javascript extensions? The
only one I hear people talking about a lot is icedcofeescript but I'm not a
big cofeescript fan...

------
estavaro
The main issue I have with "escaping from callback hell" is that it's a half-
truth. Although I don't know much about how the Reactive Framework created by
Microsoft works, I know they went well beyond the basics to try to make it
all-encompassing coming closer to making it a full-truth.

Just transmitting data back and forth may play well to the strengths of your
abstraction. But we have other uses with Timers that should also need such
abstractions.

With Timers I have other needs like delaying the execution, resetting the
delay countdown, stopping it before it executes it at all (like cancelling
it), and finally with an Animation class I needed a way to finish executing a
string of events in an instant in order to start a new animation. Also the
Animation had other Animation versions at play that could need to be sped up
before a new Animation started.

In .NET they seem to have a handy feature that waits the code to run before
proceeding that comes into play with their .NET version of the Reactive
Framework.

As far as I can tell, it's tough to really solve it. JavaScript doesn't have
extra features like .NET does. We are more limited in what we can do. In Dart
they have a version of this called Future that has been streamlined recently.
As simple as it may seem to be, it comes with other related abstractions
called Streams that altogether make it a bit daunting to escape from that hell
only to land on the fire outright.

------
tomlu
It seems like this problem would be elegantly solved by starting a thread,
green thread or coroutine (depending on language) for each task and calling
the API functions synchronously from within that. I'm not sure what support JS
has for these things.

~~~
dons
Precisely. Threads allow you to separate concerns better - one thread per
task, rather than trying to process all tasks.

Simon Marlow captured this well: <http://stackoverflow.com/a/3858684/83805>

"the abstractions that threads provide are essential for making server code
easier to get right, and more robust"

------
jart
My favorite solution to this problem is a thing the OKCupid developers made
called IcedCoffeeScript <http://maxtaco.github.com/coffee-script/>

~~~
ufo
Do you know if there are any good alternatives that are pure-ish JS? I know
that same guy worked on tamejs but they don't seem to be working on that
anymore...

~~~
nimrody
There's tame.js from the same guys (<https://github.com/maxtaco/tamejs>)

~~~
ufo
I have heard about tamejs. What I find weird is that 1) the original tamejs
website is offline and 2) I never seem to find nice comparisons between the
different CPS compilers (I have this problem where I have a hard time using
something unless I can know beforehand what sort of "pros and cons" to expect)

------
lucian1900
It seems odd that so many people defend callback nested chains. You end up
writing in continuation passing style, when many compilers can do CPS
transform for you (Scheme, C#, things like IcedCoffeeScript.

------
jj2
If you have to deal with lots of nested callbacks, it's probably worth to use
a CPS transformation like Streamline.js. It makes your code a lot cleaner. As
if it was written synchronously.

<https://github.com/Sage/streamlinejs>
<http://en.wikipedia.org/wiki/Continuation-passing_style>

------
iamwil
I recently also tried my hand at promises using the node libs Q and when.

There's a gotcha with the progress handler. If you try to call the progress
handler before the progress handler actually gets a chance to attach itself
outside the function, it'll never actually fire. Some of the bugs with using
promises are rather subtle.

------
cwiz
I find LiveScript's back-calls (<-) very elegant. In fact it makes concurrent
code very easy to write and comprehend. Combined with async
(<https://github.com/caolan/async>) it is a pure joy.

As for pure JavaScript, dealing with callbacks is definitely not fun.

------
Offler
I just don't understand the issue people are having here. Nested callbacks are
a design choice, if you do not want nested callbacks stop writing code with
them in it.

Here is a simple example in some Node.js code I've just written (I've never
written any before this little project).

[https://github.com/Offler/node-
bundler/blob/master/lib/bundl...](https://github.com/Offler/node-
bundler/blob/master/lib/bundler/filesystem/JSFileReader.js)

As you can see you just need to write in an OO manner and use
function.bind(this) and Bob's your uncle. Really don't get the difficulty.

------
Too

        function getItem(id) {
          return $.get("/api/item/" + id);
        }
        $.when(getItem(3), getItem(4)).then(handleSuccess, handleFailure);
    

Maybe I'm taking the point of the example out of context here, but i worry
that promises makes it too easy to write non thought through code like this.
Why make two separate http-requests for something that should have been
possible to do with one?

Future prediction: Someone will create code similar to the one above but
inside a for-loop calling getItem hundreds of times instead of just 2.

~~~
ajanuary
You frequently don't control the API's your hitting. I've seen plenty of API's
that don't support bulk operations.

I'm not sure I'm convinced if someone doesn't know how to do a bulk operation
they're more likely to look because they've got an extra callback or two.

------
moe
If only someone could come up with an abstraction over all this... We could
perhaps call it threads, or co-routines, or such. It could make things so much
easier!

------
rorrr
> "This is better, it definitely makes it clearer what exactly is going on"
    
    
        function createItem(item, successCallback, errorCallback) {
          var req = $.post("/api/item", item);
          req.success(successCallback);
          req.error(errorCallback);
        }
    

How is this more clear than the default jQuery $.post?

~~~
ufo
Not really better. The advantages of promises show up when you pass promises
around, letting other people add their callbacks to it (this way the "callback
structure" is implicit from the program flow, just like what happens when you
do normal sync programming. Another advantage of promises is that they have
some better support for error handling (you don't need to thread the error
handler around and you "throw" statements are caught correctly)

Honestly, I didn't like the examples in the article very much. Not only do
they fail to show the major advantages promises have, but he also uses named
functions everywhere and that is quite silly.

~~~
ianbishop
The named functions were just an attempt at better highlighting what is going
on for some readers. Not sure if it worked, but sorry it annoyed you :).

~~~
ufo
I personally find that creating too many functions makes the code look too
different from the "natural", synchronous alternative. But I also hate when
people write tons of 1-line methods (Uncle Bob style) so YMMV.

------
bestest
Well, frankly, it looks like you're jumping from one sheep to another. Using
deferreds does NOT make more sense, and is actually even more difficult to
perceive as it's quite counter-intuitive. MVVM is the way to go for web apps.
Actually, anything else would be better than this.

The methods and approaches jQuery provides should not be used as a mainframe
to achieve your goal, they should only be used as helpers here and there if
ever.

~~~
lucian1900
The deferred monad is pretty much the standard way to deal with this problem
and it composes nicely. Have a look at Twisted's.

