
Escape from Callback Hell: Callbacks are the modern goto - wheatBread
http://elm-lang.org/learn/Escape-from-Callback-Hell.elm
======
tikhonj
If you don't want to use another language and compile down to JavaScript--
which is what Elm offers--there are some interesting options that are just
JavaScript libraries.

The one I've personally played with is called Arrowlets[1], which introduces a
control structure called an arrow that lets you abstract over callbacks and
event handling (among other things). Using that style of programming can
significantly simplify some fairly common tasks in JavaScript; the drag-and-
drop demo on their site is a good motivating example. However, unless you are
already familiar with functional programming and arrows, you should probably
read some background before diving into the examples.

[1]: <http://www.cs.umd.edu/projects/PL/arrowlets/>

Another interesting option I've toyed with is RX.js[2]. This is a JavaScript
version of C#'s Reactive Extentions (RX). If you are familiar with Linq, then
this library's style should seem natural immediately. The core idea here is to
abstract over events as streams that can be composed and manipulated
conveniently.

[2]: <http://rxjs.wikidot.com/>

If you don't mind using a different language, but want something that mostly
looks like JavaScript, another option is FlapJax[3]. I haven't tried it
myself, but I've certainly heard good things about it.

[3]: <http://www.flapjax-lang.org/>

There are probably more options in the same vein that I forgot or don't know
about. However, I think these three are a good starting point and could help
clean up much of your event-driven JavaScript code in the short term.

Of course, if you are willing to use a language radically different from
JavaScript, then Elm is a great option. Once you get used to functional
languages with good type systems, there is really no going back ;). The syntax
is also simpler and more minimalistic than JavaScript's, which leads to more
readable code.

~~~
tlrobinson
Promises are a good option as well: <https://gist.github.com/3889970>

Node.js had promises early on, then were removed, but they're slowly gaining
traction again.

~~~
ikawe
My understanding is not that promises weren't valued, but that there were
conflicting opinions on how to implement them, so promises were left in user
land, in hopes that the community would vet a better solution than could be
prescribed by the node team.
[https://groups.google.com/forum/#!msg/nodejs/jaufClrXU9U/ov5...](https://groups.google.com/forum/#!msg/nodejs/jaufClrXU9U/ov5WHIk7SAwJ)

q seems to be a popular choice. <https://github.com/kriskowal/q>

------
magicalist
Callback hell is certainly a real thing, but that Javascript snippet is a poor
example for a goto comparison, since it's pretty much as linear as you can
get.

The problems with Javascript and callbacks are usually (in reverse
importance): noisy verbosity (all those "function()"s), the deeper and deeper
indentations, and then ensuring execution order on interdependent async steps
while keeping it readable. In the blog post's example, you pretty much of a
serial chain of dependent steps, and the only thing really wrong with it is
that it's just ugly and approaching unreadable (syntax highlighting will help
quite a bit, though).

I think most people heavily involved with Javascript recognize those problems,
though. Promises/deferreds have entered mainstream js usage. They can be
somewhat confusing for newcomers, but several libraries can help, as others
have pointed out. Language support is evolving: "let" as an option for more
control over scoping, the arrow syntax for simpler function expressions, yield
for shallow continuations, etc. These will in turn feed back into making
libraries smaller and easier to use (I'm really looking forward to when I can
use <http://taskjs.org/> for all my async needs. Combined with the arrow
syntax, I feel like I can pretty much always avoid a callback mess and retain
clarity of (high-level) flow at a glance).

This isn't a knock on elm (this article is the extent of my knowledge of it),
and it isn't a dismissal of the problem, but it isn't clear to me from this
article what is broken in JS that is fixed in elm. In other words, this could
be another tutorial on promises in Javascript and make the same points about
excessive callbacks being poor coding style and bad news for readability and
maintainability.

Syntax that makes clear code the lowest energy state _is_ a feature, but (if
we limit our discussion to callbacks) in JS it's partly solved, partly being
worked on, and it's not clear to me yet what the energy differential is in
typical elm usage between this code and the nasty spaghetti code you can
always write if you try.

------
ender7
Personally, any solution to callback hell will also need to be a language that
supports returning multiple values. The following convention is simply too
useful to me:

    
    
      foo.doSomethingAsync(function(err, result) {
        if (err) {
          ...
        }
        ...
      });
    

You can obviously accomplish this with exceptions, but then you have a million
try/catch blocks floating around all over the place and the code becomes even
_harder_ to read (and more verbose to boot).

~~~
pufuwozu
There's a way without exceptions. Use an ErrorT[Promise[A]] monad. I've done
this in Scala before and it is _so_ much simpler than the JavaScript
convention you pointed out.

It allows you to write code like this:

    
    
        val query = "Brian"
        val result = for {
          db <- connectToDatabase
          user <- userFromDatabase(db, query)
          friends <- friendsFromDatabase(db, user.friends)
        } yield friends
    

Whenever one of the functions above returns an error, the whole expression is
that error. The outcome is either an error (if there was any) or the end value
of Set[User]. No need to manually handle an error until you get to the end
result.

~~~
pfraze
That's basically what the promises spec for Javascript gives you.

~~~
pufuwozu
Exactly. The promises spec defines the Promise monad. The problem is that
JavaScript doesn't have monadic syntax, which would make the code a lot more
readable.

------
webjprgm
The "callback hell" example is a rather tame one, since the code is readable
in a single location just with some nesting. So when I see the FRP solution
which is the same amount of code I'm not certain that in a complex example
this actually solves the problem. You can still have lift statements scattered
around a program just like you can have callbacks to various functions
scattered around.

The solution to GOTO was to remove it and replace it with a few control
structure forms that enforced locality. I remember converting someone's GOTO-
laced code once and basically everything could be re-written with some clever
use of do-while with break statements and an occasional flag variable. do-
while, while, for, etc. replace GOTO in 99% of cases and enforce the desired
code locality for readability.

So what syntactical structure could enforce locality of time-connected
variables?

E.g. some idea like this:

    
    
        data, err <- $.ajax(requestUrl1)
        if( err ) {
          console.log(err)
          return
        }
        data2, err <- $.ajax(makeRequestUrl2(data))
    

Where the <\- syntax could be like an = statement but say that the assignment
and all following statements are placed in a callback.

~~~
romaniv
C# 4.0 has async/await to solve this.

[http://msdn.microsoft.com/en-
us/library/vstudio/hh191443.asp...](http://msdn.microsoft.com/en-
us/library/vstudio/hh191443.aspx)

~~~
Someone
Async looks nice, but this ([http://www.jayway.com/2012/10/07/asyncawait-in-c-
a-disaster-...](http://www.jayway.com/2012/10/07/asyncawait-in-c-a-disaster-
waiting-to-happen/)) makes me wonder whether it is the right solution for this
problem. Breaking what most programmers think are the 'laws' of refactoring is
not a good thing.

~~~
int_19h
The problem in the linked article is not async/await, but rather specifically
async void, which you don't get by following the straightforward sync->async
refactoring rules. A normal void method would start returning Task when turned
async (and a non-void method returning T would start returning Task<T>). Async
void is a special, distinct beast which is fire-and-forget by definition, and
it only exists in async land. Perhaps they should have used a special new
keyword for that instead, but in any case, it's a completely orthogonal
problem.

------
debacle
Callbacks are different than gotos in that they are aren't even remotely close
to gotos.

With a callback, you can get into 'callback hell,' however the root cause of
that is that you probably don't understand the nuances of properly
architecting a solution that involves the power of first-class functions.

JavaScript is nice because the scoping of the callback is easily controllable
through your invocation method, and if you've created a good object model then
it's relatively easy to maintain an understandable state.

When you explicitly define callbacks like in the examples, you're tightly
coupling the response handlers to their requests, which is a relatively poor
implementation and will bite you in the ass later on.

~~~
pufuwozu
_Callbacks are different than gotos in that they are aren't even remotely
close to gotos._

The analogy is that callbacks create non-linear control flow. Using a monadic
syntax like in Roy, we can easily have callbacks without having them look non-
linear:

    
    
        let deferred = {
          return: \x ->
            let d = $.Deferred ()
            d.resolve x
            d.promise ()
          bind: \x f -> x.pipe f
        }
    
        let v = do deferred
          hello <- $.ajax 'examples/helloworld.roy'
          alias <- $.ajax 'examples/alias.roy'
          return (hello ++ alias)
    
        v.done console.log
    

Which compiles into continuation passing:

    
    
        var deferred = {
            "return": function(x) {
                var d = $.Deferred();
                d.resolve(x);
                return d.promise();
            },
            "bind": function(x, f) {
                return x.pipe(f);
            }
        };
        var v = (function(){
            var __monad__ = deferred;
            return __monad__.bind($.ajax('examples/helloworld.roy'), function(hello) {
                return __monad__.bind($.ajax('examples/alias.roy'), function(alias) {
                    return __monad__.return((hello + alias));
                });
            });
        })();
        v.done(console.log);
    

Anyway, continuations (with call/cc) are definitely a controlled form of goto.
Take a look at an example from Paul Graham:

<http://lib.store.yahoo.net/lib/paulgraham/cint.lisp>

    
    
        ((call/cc
          (lambda (goto)
            (letrec ((start
                      (lambda ()
                        (print "start")
                        (goto next)))
                     (froz
                      (lambda ()
                        (print "froz")
                        (goto last)))
                     (next
                      (lambda ()
                        (print "next")
                        (goto froz)))
                     (last
                      (lambda ()
                        (print "last")
                        (+ 3 4))))
              start))))

~~~
tptacek
All of structured programming consists of various controlled forms of goto.

~~~
m0nastic
Yesterday, I found an email from Jon Callas where he was giving advice on
where SAML falls in the Chomsky hierarchy of languages:

 _"If it has backwards gotos in any form, it's Turing-complete. Loops,
recursion, etc. are backwards gotos. If-then-else is a forward goto."_

I hadn't ever thought of things that way, but I thought it was interesting.

~~~
hvs
Once you've worked in assembly language, you realize everything is just
another form of _jmp_.

------
Sephr
One way to escape callback hell is to use a async.js
(<https://github.com/eligrey/async.js>), which uses yield to abstract away
callbacks. It's Firefox-only (JS 1.7+) though, but that can probably be
resolved by using a JS parser and replacing every yield with JS 1.5 callbacks.

Full disclosure: I wrote async.js.

~~~
nickporter
There's also this library: <https://github.com/caolan/async>

~~~
Sephr
That library uses callbacks and isn't really related other than the name. I
don't think that the author googled async.js to see if the library name was
already taken.

~~~
tlrobinson
<https://github.com/caolan/async>: 3293 stars on Github

<https://github.com/eligrey/async.js>: 15 stars on Github

Sorry, you lost this one.

~~~
Sephr
Sorry, but what do stars have to do with it? That's like saying DuckDuckGo
should be allowed to rename itself as Google if they become more popular than
Google. <https://github.com/caolan/async> had zero stars (i.e. didn't exist)
back when I released async.js.

~~~
tlrobinson
I'm just saying you're going to have a hard time convincing everyone that your
project is _the_ "async" JavaScript library. I presume you don't plan on suing
them for trademark infringement.

It's annoying (it's happened to me too) but I don't think there's much you can
do about it. Especially when the name is incredibly generic.

------
wyuenho
The problem is not callback, the problem is that callbacks exists in
Javascript.

Callbacks themselves, when used wisely, can often enhance code readability,
hell LISP has had function references since forever, but I think the most
complain about callbacks are actually complains about callbacks in noisy
languages, mostly likely languages with noisy syntaxes like Javascript and
Java. When read that way, the disgust towards callbacks do seem to have
merits. As the author has pointed out, the 2 getPhoto() functions at the end
express and do exactly the same things, but obviously the CoffeeScript version
reads better.

Callbacks have been around a long time and I've never heard of people complain
as much about them as people have for Javascript and I conjecture the reasons
are as follows:

1) There's no named parameters (keyword arguments) in Javascript, so people
pass around objects literals into functions to emulate them. 2) Making lambdas
in JS is too easy, but the syntax is too noisy. 3) Oh so many aliasing of
_this_ ; 4) Self-chainable JS libraries like jQuery makes the style of calling
multiple functions too easy. But lines can only go to long before becoming
unwieldy, so people tend to indent chained method calls multiple times. 5) No
modules and global namespace pollution is frown upon, so people are hesitant
to flatten deeply nested callback chains. 6) There are a dozen ways to make a
JS class and/or object depending on frameworks, and they are not at all
compatible.

All of these "features" coagulate in JS into giant blobs of snot like this:

    
    
      <script>
        $(document).ready(function() {
          var $main = $("#main");
          $main.
            hide().
            click(function(e) {
              $.ajax({
                type: "POST",
                dataType: "json",
                contentType: "application/json",
                success: function(data, textStatus, jqXHR) {
                   data['rows'].forEach(function(line) {
                       $main.append($("<p>", {
                          className: "row"
                       }).append(line));
                   });
                }
              });
            }).
            show();
        });
      </script>
    

Words for the wise, when you see a shiny new jQuery plugin, stop, think for 3
minutes, and then put it inside a Backbone View or whatever your favorite
framework is other than jQuery*. If you don't know anything other than jQuery,
now is probably the best time to learn a few.

------
herge
This reminds me of the pain of dealing with python's twisted library, albeit
before inline callbacks were implemented.

Inline callbacks as implemented in python can make asynchronous code a lot
easier to read: [http://hackedbellini.org/development/writing-asynchronous-
py...](http://hackedbellini.org/development/writing-asynchronous-python-code-
with-twisted-using-inlinecallbacks/)

~~~
pjscott
And using something like Eventlet or Gevent can make asynchronous code
downright pleasant. Of the two, Eventlet has the better docs, and both are
easy to get started with, and stable enough for production use:

<http://eventlet.net/doc/>

Give it a try! You almost definitely won't regret it!

------
peterbe
Isn't `yield` the solution to all the problems? It makes things responsive and
avoids the callbacks entirely.

For example: <http://www.tornadoweb.org/documentation/gen.html>

~~~
masklinn
It is _a_ solution, kind of, but annoying on a few accounts:

0\. it plays hell with threadlocals-as-dynamic-scoping, which is the only way
most "modern" languages permit dynamically scoped variables

1\. it needs to be correctly passed along to callers, and given it's usually
used in dynamically typed languages there's a high risk of forgetting and
dropping a yield

2\. yield being also used for iteration, it can be confusing to keep them
straight.

It's definitely a better solution than callback hell though. An other approach
is runtime support as in gevent, where the "yielding" is done by the
library/IO code and invisible to the caller. The final two I know of are
baking lightweight concurrency and communications into the language itself
(Erlang, and to a lower extent Go and Rust) or monadic systems (Haskell)

------
ccleve
You've got to ask, why is async programming used at all? The reason is
twofold: first, the C10K problem, where too many threads kill performance, and
second, sometimes you want to have multiple tasks run in parallel.

There are fairly simple syntactical solutions to both problems.

    
    
      result = doSomeAsyncTask()
      result.yield() // drops the thread, picks it up on response
      // do stuff with result here
    

This magic yield() doesn't exist (to my knowledge), but if it did, it would
preserve linear code and also solve the C10K problem.

You could have similar code to solve the multiple task problem:

    
    
      result0 = doSomeAsyncTask0();
      result1 = doSomeAsyncTask1();
    
      while (result = getNextCompletedTask(result0, result1)) {
        // do something to result
      }
    
    

A Future in Java does something like this, but it doesn't drop threads.

------
jblow
Callback Hell is certainly a real thing. I decided 12 years ago that I would
never use callbacks if I could avoid it (the only wai you can't avoid it is if
an API forces you to use them); I have never looked back. Regular, simple,
straightforward imperative flow control is a very powerful thing, and any time
you give it up or make it more squishy and indirect, you had better be getting
something _big_ in return. Usually you aren't.

That said, what the article proposes as a solution is bananas. You don't need
to do crazy functional acronym things; just don't use callbacks. Good C/C++
programmers in the field where I work (video games) do this all the time. It's
not hard except that it requires a little bit of discipline toward simplicity
(which is not something exhibited by this article!)

~~~
tactics
Actually, you're wrong. This is bananas (based on very closely related FPR
concept)

<http://www.haskell.org/haskellwiki/Reactive-banana>

Secondly, you come off sounding defensive and ignorant. This is a new
programming paradigm. Hopefully it will give people new ways to approach the
same difficult problems. (And I really hope you believe GUIs are inherently
difficult...)

No one is twisting your arm to learn FPR. If callbacks work for you in your
job, then stick with what works.

~~~
jblow
When I was in college, and shortly afterward, I was very much into "new
programming paradigms" and would get excited about lazy evaluation or
continuations or whatever was the new cool idea going around. I have designed
and implemented several programming languages built around new / wacky
features; the most recent of these was ten years ago.

What you are hearing now is not ignorance, it is experience. I am a
tremendously better programmer than I was in those days, and the way I got
better was not by getting excited about wacky ideas; it was by really noticing
what really works, and what doesn't; by noticing what are the real problems
that I encounter in complicated programming projects, rather than what
inexperienced / pundit / academic programmers tell me the problems are.

Clearly you didn't really read my comment, though, since you are saying "If
callbacks work for you in your job..." and my entire point is that callbacks
are terrible.

------
Jacob4u2
The author offers an alternative that would require a change to the language.
Callbacks and their use in "callback hell" are a little different than use of
"goto"; "goto" appears to have an obvious alternative that was more logical to
use already implemented in the language. For javascript, there is none of the
nice syntactic sugar (reminds me a lot of C# recent async changes) that the
author suggests and is not even being proposed for ECMA 6.

I agree it would be nice to have that stuff, and that callbacks can get a
little hairy, but they are the best solution available at present. Shall we
stop developing applications in the mean time while the language catches up,
or even worse, browsers actually consistently implement the changes?

~~~
dllthomas
No, we should stop writing javascript and start writing elm - a language that
compiles to javascript, works like described in that page, and is the subject
generally of the site hosting the article.

At least, that's what the article is saying. There are a few interesting
things on the horizon there, but I've been watching elm with some interest.

------
wglb
This kind of confuses two important ideas, both discussed by Dijkstra.

The most popular was his article about gotos.

Another idea in his writings was that time-dependent programming was
dangerous. He was talking about interrupt based programming specifically, and
also addressed the common practice of some hardware to have asynchronous IO.
You would start an IO operation, and go on and do other things, come back
later and see the values there.

So these two things are not alike. They both cause confusion about what the
program is doing, but they are not "like" each other.

To be a better programmer, it is good to read Dijkstra. It is really all about
avoiding errors in programming.

------
darwinGod
As someone who writes C code for a distributed system that uses event-driven
callbacks ( Zscaler) (yes,the binding is at compile time), I was aghast when I
saw goto's in the codebase. I mean,I believed programmers were indoctrinated
with " using goto = goto hell". I have realized that if used smartly,goto's
cause no problem-say in error handling. I can confidently say I have not seen
a single bug because of improper usage of goto in the last 1.7 years. And we
do a lot of interesting things in C,including talking to a Postgres
database,having a messaging protocol layer,doing shared memory manipulation
etc.

------
grimtrigger
One thing I'd like to see from languages that compile to js: Some kind of
evidence that output readability is a concern. You can make some beautiful
abstractions, but if I can't debug it when things go wrong, then there's no
way I would use it.

Not making any comments about Elm's output, but the author clearly doesn't
consider it a priority in the post.

~~~
Evbn
But how? Does your C code compile to readable assembly? Does Haskell compile
to readable C? The high level language is created because the lowlevel
language is unreadable when solving problems the high level language solves.

~~~
spartango
While I agree with your premise, the thing that is driving the parent comment
is the fact that when debugging a language such as Elm, there is no tooling to
make sense of the output code. This forces the user to use standard JS
debugging tools on a pile of JS that was not written to be debugged. This is a
contrast to Java or C, where debugging tools and hooks have been built to
indicate where the low-level code is mapped to high-level code.

At the end of the day this is more a call for language authors and the people
around them to develop tools to debug the language.

------
whatgoodisaroad
This seems like a bad analogy. Dijkstra's paper was in favor of "structured
programming",and the problem was that goto was too-unstructured. If anything,
callbacks are excessively structured.

Also, why is nonlinear code a bad thing? If the program behavior should be
nonlinear, then neither should the code.

~~~
Evbn
Because people have trouble reasoning mon-linearly.

------
chubbard
Two observations. First, great how do we debug it? How can we see our signals
between each step? How about beyond simple print/logging?

And two, I like his contrast between async vs synchronous flows, and
recognizing synchronous style programming has many benefits that CPS doesn't.
However, I think even this style still hasn't solved the bigger problem with
asynchronous style programming. The ability to reuse it easily. In synchronous
style programming I can reuse code and add to that block by calling the
method, then after that method is done add my code.

    
    
       ... my code before ...
       var result = someMethod()
       ... my code after ...
    

It's just that simple with synchronous style. With async style the author has
to provide you a hook to hook onto the end or beginning of this flow (adding a
callback param, returning a promise, etc). I think even with using signals you
have the same issue. Without explicit hooks you can't hook more code onto it
like you can with good old fashion synchronous programming. Not to mention
error control flow is turned upside down too.

I'm intrigued by the ideas of signals over callbacks, but I don't know if they
fix enough problems with callbacks yet.

~~~
graue
The following is a dead comment by seanmcdirmid... reposting since it seems
perfectly legitimate and I have no idea why this would be voted down.
(Glitch?)

Debugging is one of the problems with FRP/signals, or functional code in
general. No one has come up with a good dataflow debugger yet, and it might
not even be viable. The best you can do is interpose "probes" on your code
like you would take measurements with an oscilloscope. Disclosure: I did my
dissertation on signals (object-oriented ones to be precise), and am a bit
disillusioned with it.

On the other hand, the argument from the declarative community is that you
don't need to debug your code.

A better alternative to FRP/signals might be immediate mode user interfaces.
Since they are conceptually called on every frame, you get the benefits of FRP
while still being able to debug in the old way. On the other hand, they are
quite inefficient, though I think we could play some tricks with technology to
make them better (memoize, refresh blocks of computations only as needed via
dependency tracing).

------
jawns
So, this Functional Reactive Programming stuff compiles to Javascript, right?

Is the resultant Javascript just a bunch of nested callbacks, as in the
example the blog post uses to illustrate spaghetti code?

~~~
eblume
The 'big problem' with callbacks isn't the callbacks themselves - it's the
resulting difficulty to the developer to keep track of the flow of control.

This is analogous to goto's - _programming_ with goto is a nightmare, but all
code 'compiles to' some sort of immediate form using goto's. The goto is not
the problem, programming with goto is.

------
aurelianito
Actually, callbacks are the Intercal's COME_FROM instruction
(<http://en.wikipedia.org/wiki/COME_FROM>).

So, it is even worst!

------
absconditus
Here are the slides from Evan's talk at Strange Loop:

[https://github.com/strangeloop/strangeloop2012/blob/master/s...](https://github.com/strangeloop/strangeloop2012/blob/master/slides/elc/Czaplicki-
ElmMakingTheWebFunctional.pdf?raw=true)

------
jcampbell1
@mpolun - It appears your account has been hell-banned. You need to create a
new account so I can upvote your comments:

mpolun> I agree that raw callbacks can get out of hand, but the typical
solution in js is to use an event emitter
(<http://nodejs.org/api/events.html>) or promises (like
<https://github.com/kriskowal/q>), the latter of which seems to be pretty
close to what this article is talking about. Is there a fundamental
difference, or are promises an example of functional reactive programming in a
language without direct support for it?

------
cbsmith
Voting up just for not using a "...Considered Harmful" headline, particularly
since the author obviously is familiar with the idea.

------
grogs
Another approach to async IO is CPS (continuations passing style), in which
you write imperative style code. This imperative style code is then compiled
such that the blocking IO operations are called with callbacks, which are the
remainder of that block of code (the continuation) - allowing the calling
thread to be re-used while blocking for IO. Relies on the continuation having
access to the outer/parent/previous-part function via closures.

It'll be interesting to see if people start doing this. Requires people to
understand continuations and closures (which more people have exposure to now
via JavaScript), and library support.

~~~
ambrop7
Hey, that was my idea too, see my own comment [1]. I've actually developed a
(very simplistic) programming language utilizing this approach. I even
compiled the interpreter to Javascript so you can try it our from the browser.

[1] <http://news.ycombinator.com/item?id=4736630>

------
halayli
Agreed. It's one of the reasons why I wrote lthread. Implementing any non-
trivial protocol over a socket for example will lead to callback hell. There
are plenty of states to transition from/to and it's very easy to get it wrong
no matter how good the code is structured.

Forget Javascript for a second, and take a look at a typical http proxy
written in C using callbacks to see what a callback hell looks like. If 5
developers are working on such a project, it will require a lot of mental
effort from each developer to keep the callback flow up-to-date in their head.

------
apeace
Having programmed in many languages, but most recently Node.js for the last
two years, I don't think "callback hell" is as big a problem as the OP makes
it out to be.

Debugging huge, complicated, and even poorly written Node applications doesn't
feel much different to me than debugging huge, complicated, or poorly written
Java applications. Sometimes it's a pain, that's unavoidable. You can prevent
it to an extent by writing clean, tested code.

I don't see a strong resemblance between goto and callbacks. The resemblance
is just as strong between goto and any function, or class

------
vvpan
I've been a JS (which is the land of callbacks) programmer for only a few
months now, and I would disagree. Yes you can write deeply nested callback
chains, but you don't have to most of the time. There are a couple of ways to
avoid it.

* The async library mentioned by other posters helps a lot.

* Libraries like backbone make writing event-driven software easier.

But to sum it up: it's like anywhere else, bad programmers write "callback
hell" code, and good programmers don't.

~~~
tibbe
This excuse is as old as programming languages.

* Programmers who need a higher level language (e.g. C instead of assembly) are just bad programmers.

* Programmers who can't manage manual memory allocation are just bad programmers.

* etc

------
zimbatm
I'm not convinced.

I'm not familiar with the last approach but it seems to me that with a couple
of higher-order functions in JavaScript, the code will quickly become more
manageable.

    
    
      function getPhoto(tag, handlerCallback) {
         asyncChain(requestTag, requestOneFrom)(tag, function(photoSizes) {
           handlerCallback(sizesToPhoto(photoSizes));
         });
      }
         
      getPhoto('tokyo', drawOnScreen);

------
sses
I used a functional-reactive-language-that-compiles-to-javascript for a web
app, in a project that lasted about 3 years. It solved callback hell, and
solved some UI problems, but created some hard UI problems as well. I'm not
sure how this would translate to a server, but some examples anyway.

It seemed impossible to completely escape imperative programming. Mouse click
handlers for example were much more natural to write imperatively; changes
made in the imperative code would propagate as signals in a reactive way.

Reasoning about what happened around the boundaries of imperative and reactive
code was hard, especially as the application grew in complexity. If I have a
UI element that depends on two other signals - think spreadsheet cell with a
formula that depends on two other calculations - do I want to update it as
soon as one signal changes? do I wait for both to change? Do I want different
behaviors in different circumstances? It often led excessive layout changes as
values passed through intermediate states, or code being executed multiple
times unnecessarily.

~~~
Evbn
Is non-reactive code any better at solving those problems? Those are hard
problems.

------
bunderbunder
The number of languages that compile to JavaScript is starting to become
disconcerting.

How long until we get tired of adding epicycles and just specify a VM and
bytecode standard that all the browsers can implement and all the client-side
languages can compile to?

~~~
CJefferson
Who would implement in their browser a VM no website used, and who would use a
VM no browser implemented?

I'm not sure who you think "we" are, who should specify a VM and bytecode
standard. If we wanted a bytecode standard, there doesn't seem anything deeply
wrong with Java, and we seem to be in the process of phasing that out of every
browser.

~~~
bunderbunder
It'd have to be bootstrapped the same way other Web standards such as CSS had
to be, which admittedly might take a while.

Java and Flash are no-gos because they're not open enough - they're owned by
companies that want to maintain tight control over the platforms. They're also
too heavily built around their own private APIs; what would really be needed
is something that sticks to the same APIs and DOM that JavaScript interacts
with, in order to ensure a reasonable migration path for existing
technologies.

My guess is that the easiest option would be to base such a standard on a VM
and bytecode format that a major browser already does implement: the one from
the Spidermonkey Javascript engine.

~~~
masklinn
So alternative VMs and VMs progress can go die in a fire, one runtime to rule
them all and in the darkness bind them? No Carakan, no
JavaScriptCore/Squirrelfish, no Chakram, no V8?

Here's an idea: javascript is your bytecode, and you have every single
javascript VM as your runtime.

~~~
bunderbunder
Perhaps not all of those specific ones, at least not without modification. But
you're being hyperbolic; there's absolutely no reason you couldn't have
multiple implementations of the VM standard. In fact, there should be - in the
grandparent I pointed out that single-owner standards like Java are a no-go in
my opinion.

My only thought here is that having a bytecode standard to work from might
give a little more flexibility and power to folks who want to experiment with
alternative client-side languages.

~~~
CJefferson
A byte-code based on Javascript VMs would be less useful than you might think.

For many years people who have tried to build dynamic languages on Java have
had to go through horrible pain to build their languages, and not gained the
performance they would want -- only the recent addition of invokeDynamic have
finally allowed fast implementations.

Personally, I would hope any browser bytecode would have big integers as a
primitive, as implementing them in dynamic language is very slow. However, no
Javascript byte code would have big integers, as they aren't in Javascript!

~~~
bunderbunder
What Javascript's capable of would really only need to be a starting point.

I certainly wouldn't want to limit what's possible to what Java or Javascript
are capable of. Certainly not Java. Java is (IMO) practically defined by
stagnation as a result of poor management by the companies that have owned it.

And part of what I'm thinking here is getting away from the limitations
imposed by Javascript. For example, Javascript lacks big integers, and is a
dynamic language. My thought is that decoupling the HLL from the runtime would
allow for faster evolution, because it's hypothetically easier to add new
opcodes to a bytecode and VM than it is to make large shifts to a high-level
language. Consider the whole dynamic thing in particular - .NET has a great VM
with pretty good support for both static and dynamic languages. Java didn't,
but again we aren't strictly required to repeat the mistakes of the past. And
by just using "Javascript as the bytecode" we commit the reverse sin - that's
a 'bytecode' with poor support for static languages.

------
spatten
Here's a google cache link:
[http://webcache.googleusercontent.com/search?q=cache%3Ahttp%...](http://webcache.googleusercontent.com/search?q=cache%3Ahttp%3A%2F%2Felm-
lang.org%2Flearn%2FEscape-from-Callback-Hell.elm)

------
ams6110
_It [synchronous call] basically dodges the issue of time-dependence by just
freezing if it is waiting for a value. This blocks everything in the program.
Mouse and keyboard input just piles up, waiting to be processed, presenting
the user with an unresponsive app. This is not an acceptable user experience._

It depends on what else your user can realistically do before the call
completes. In many cases the answer is "nothing." He needs the result of the
call before he can proceed in his task. In simple web apps this happens a lot.
In those cases I will often just make a synchronous call and avoid all the
callback complexity.

~~~
Evbn
What about auto save in gmail, prefetching, loading resources in parallel,
updating buddy chat status, realtime stock prices, ...

------
vinayan3
Callbacks do lead to hell. The back traces half the lead you now-where. I have
been writing a scrapy crawler and sometimes when an exception happens it takes
some grepping around to figure out where the value that is wrong actually was
generated.

Has anyone touched the Google Chrome code base? It is quite difficult to start
debugging problems because of the sheer volume of callbacks. Add to that the
stack-traces are massive because of the use of templates and other C++
language features.

Async coding needs to be an abstraction within the language. I am curious how
languages manage the shared memory. What about the risk of dead locks?

~~~
SrslyJosh
> The back traces half the lead you now-where.

Yes. This can be solved by maintaining your own backtrace of callbacks (say,
as part of your request object/structure), but it's really something that
should be implemented in the language or framework.

~~~
Evbn
It is crazy. This is a huge problem that is so easy for the language writers
to solve (add a history object to each function call, when compiling/running
in debug mode), but no one solves it.

------
grannyg00se
"It is pretty much the same as using goto to structure your programs."

I don't see how a self contained block of code can be equated to goto where
the flow can bounce around all over the place.

The example callback "hell" code doesn't look any more complicated than the
solution Elm code to me. Maybe the improvement is going over my head and I
need to read it again. I just don't see it. Then again, I feel the same way
about Coffeescript. These javascript helper languages just seem like an
unnecessary added level of complication and cognitive load.

~~~
SoftwareMaven
This is the typical "blog sample" complaint: "Four lines of code doesn't prove
your conjecture". Of course, putting an entire application in would make for a
challenging read.

While I'm not sure I would go as far as saying goto==callback, after wading
through a fairly extensive browser extension, I know that callbacks are
inherently more difficult to reason about than imperative code.

I haven't decided if the tradeoff of more flexible code versus more difficult
reasoning is a good one. In some ways, I think the article got it right: we
are expecting the programmer to do a lot of book-keeping that a compiler could
do more cleanly. After all, it is possible to write goto-laden code that is
just as readable as function-laden code. It's just nice to let the compiler
manage all the baggage of frames, stack pointers and jumps for us.

------
schd
Does anyone recall the article a few months back that sort of dealt with
callback hell? It was a concept from a different language (which one, I don't
recall) that worked basically with returning 2 values and formatting your
functions according to a general model.

Yes, that's a bit vague, but that's all I've got to go on. :)

------
jberryman
Marginally related but IYI, I recently wrote a JS lib for writing loops with a
delay, to avoid one instance of callback hell:

[http://brandon.si/code/dilly-dot-js-a-library-for-loops-
with...](http://brandon.si/code/dilly-dot-js-a-library-for-loops-with-delays-
in-javascript/)

------
ambrop7
I think the author is missing the obvious and natural solution: let the
programmer write code in a completely synchronous (blocking) style, but have
the programming language execute it an an asynchronous and concurrent fashion.
Something like that:

    
    
      # this appears very synchronous
      function getPhoto(tag) {
          var photoList  = syncGet(requestTag(tag));
          var photoSizes = syncGet(requestOneFrom(photoList));
          return sizesToPhoto(photoSizes);
      }
    
      # Two getPhoto() "processes" are spawned. After this,
      # the language multiplexes between them via the (single) event loop,
      # in a single OS thread.
      job1 = spawn getPhoto('tokyo');
      job2 = spawn getPhoto('tokyo');
    
      # Wait for both of them to finish. This too happens in an asynchronous
      # fashion, i.e. calling job1.join() does not prevent the two jobs from
      # running. In effect at this point we have three "processes" running
      # (the main process doing the joins, the job1 process and the job2 process).
      photo1 = job1.join();
      photo2 = job2.join();
      drawOnScreen(photo1);
      drawOnScreen(photo2);
    

Yes, I know this may be very hard to implement in Javascript/Node, because it
fundamentally changes the way the JS engine needs to work.

NOTE: It seems this approach is not new; "green theads" seems to be the right
term, and there seem to be a lot of Python-based implementations. Go's
goroutines also appear similar (but you can have them run truly in parallel).

 _BUT_ note a crucial difference from the "green threads" approach - in my
suggested design, there would be no real scheduling. If you perform sequence
of operations and they are all guaranteed not to block, this sequence is
_automatically atomic_ , and cannot be interrupted by another "process".

I should also mention this programming language I'm developing, called NCD
[1], which employs the same idea. See the in-browser demo [2], click on the
Spawn example.

Note that NCD implements a unique extension of imperative programming.
Statements in general persist even after they have "returned", and they get
the chance to do stuff when they are about to be "deinitialized" (see what
happens when you click "Request termination" in the Spawn example). Plus, any
statement can trigger "backtracking" to its point within the process, causing
automatic deinitialization of any statements that follow (try Count example).

Also, IMO promises [3] are just a hack around the fact that the language is
not inherently asynchronous. Seriously, who would prefer:

    
    
      doFoo()
          .then(function (foo) {
              return doBar(foo);
          })
          .then(function (bar) {
              return doBaz(bar);
          })
          .then(function(baz) {
              console.log(baz);
          });
    

Over this?

    
    
      function myWork () {
          foo = doFoo();
          bar = doBar(foo);
          baz = doBaz(bar);
          console.log(baz);
      }
      spawn myWork();
    

[1] <http://code.google.com/p/badvpn/wiki/NCD> [2]
<http://badvpn.googlecode.com/svn/wiki/emncd.html> [3]
<https://gist.github.com/3889970>

~~~
kerneis
Green threads are indeed very useful, and probably the right way to go. In
Javascript, see Tamejs: <http://tamejs.org/>

NCD looks interesting, in particular the backtracking feature. But I'm a bit
concerned by your choice of developing a new language from scratch, with a
very awkward syntax (at least at first sight). Why not extend an existing
language?

See CPC for instance, which extends the C language with a spawn primitive
(disclaimer - this is my PhD thesis project): [http://www.pps.univ-paris-
diderot.fr/%7Ekerneis/software/cpc...](http://www.pps.univ-paris-
diderot.fr/%7Ekerneis/software/cpc/) and <http://www.pps.univ-paris-
diderot.fr/%7Ekerneis/research/> for more details.

~~~
ambrop7
Yes, Tamejs and CPC indeed do exactly what I had in mind. CPC is particularly
amazing for doing this to C.

Considering NCD: you might have noticed that NCD was never meant to be a
general-purpose language, but rather a simple scripting language for
controlling other programs and OS configuration. My thinking was that
extending a language with the features that define NCD (asynchronous
execution, backtracking and extended statement lifetime) would essentially
require a complete rewrite of a language implementation. However after seeing
tamejs and your CPC I'm not so sure anymore :)

P.S. Check out the Read File example I just added to the NCD demo page; it
shows how backtracking can be used to handle errors elegantly.

------
rgbrgb
An alternative method of decoupling callbacks is to make heavy use of events
within your program. However, perhaps an event is even more like a goto
because it doesn't encapsulate state.

~~~
Evbn
What is the different between events and callbacks? I though an event triggers
a callback.

------
nextstep
Can anybody recommend a way to avoid Callback Hell in Objective-C?

~~~
nubela
blocks. but they are dangerous when dealing with threading.

~~~
mpweiher
Huh? As far as I can tell, blocks _are_ callback hell in this case, certainly
the nested callback lambdas in the JavaScript examples are essentially
equivalent to blocks.

------
krob
Amen brotha! I totally agree. Callbacks, especially nested callbacks are a
nightmare to debug, and especially to refactor.

------
jschrf
The solution to all of this is incredibly simple but for some reason very few
people seem to utilize it: State machines.

~~~
pjscott
The reason people don't use explicit state machines more is because it's kind
of a painful way to write code. Suppose that I want to do a bunch of HTTP
GETs, munge the data, and then fire off corresponding HTTP PUTs. I would
_like_ to be able to write something like:

    
    
        def process(job):
            http.put(job.destUrl, munge(http.get(job.srcUrl)))
        
        print "Processing some jobs"
        workerPool(MAX_CONNECTIONS).map(process, jobs)
        print "All done!"
    

Wouldn't that be great? And in fact, you can do this easily with something
like Eventlet, so I'm not asking too much here.

~~~
spartango
Amusingly, in an environment with a level of parallelism, the above code poses
few problems even if http.get/put are synchronous & blocking. A scheduler
handles a bunch of blocked threads perfectly fine, and the series of calls per
job aren't inefficient when executed in series.

We hit pain points when we can't have parallelism in our programs, and are
forced to make elegant the work-arounds intended to unclog those programs.

------
padobson
Link seems broken, page fails to load.

~~~
com2kid
It seems that web site does not like IE9. I can view the source, but nothing
shows up.

Thankfully other browsers work fine. Makes me wonder what they are doing to
completely break rendering on IE though...

------
exabrial
So essentially, everything about node.js is wrong? Hmm. Didn't see that
coming!

~~~
ufo
To be less inflamatory, using callbacks to specify those low level APIs
certainly makes sense, since the main desire here is to make them simple and
interoperable, not necessarily human friendly.

You can then layer your favourite libraries or tools on top of that to make
things better to live with. (For example, a promise-based wrapper for the low
level libs or a compiles-to-js dialect)

I do agree that all those posts with people writing callback-based code as
though it were the future were kind of sad though.

------
tjholowaychuk
goto is sweet

------
martinced
Why is it that nearly everytime some coding blog writes about something
interesting you've got lots and lots of people criticizing the entry and
saying, basically, that "the old way of doing thing is just fine, it's just
you that are too stupid to understand it".

I'm sorry but that's not how it works. I've been coding for 20 years or so and
I'm always open to new things. The first time I heard about using immutable
objects in Java nearly everybody was laughing at the idea, making fun of it.
Nowadays it's the contrary that is true. Same thing for using composition
instead of concrete inheritance: everybody was there, ten years ago, saying
that there was no problem with Java's concrete (implementation) inheritance.
Or checked exceptions. Etc.

It's nearly always the same thing: a concept that is not mainstream but that
looks very promising is explained in great details and yet people come here,
bragging: "You're too stupid to understand ${common way of doing things},
there's no need for ${less commonly used technology}".

This really saddens me.

~~~
marshray
I've been coding for 20 years too and I'm also open to new things. But no one
can be proficient in everything anymore so there is a cost to learning
something new: we have to forget something we previously spent time to become
familiar with.

Don't be sad, it's clearly a natural reaction that has some utility. I agree
that there's a better way to phrase it, usually something like "is it really
_that_ much better than what we have now?"

~~~
mgkimsal
"we have to forget something we previously spent time to become familiar
with."

<http://www.youtube.com/watch?v=8dbDJzDV1CM>

------
Evbn
I don't get it. His example just uses "let ... in" syntax instead of putting
the next func as an argument to the first. It seemed to be exactly the same
thing "deeply nested functions". He just chose not to indent after the "in",
and he has a nicer syntax for nested functions.

