
Callbacks as our Generation's Goto Statement - matthewn
http://tirania.org/blog/archive/2013/Aug-15.html
======
crazygringo
"Await" is fantastic, and having using it for JavaScript (via TameJS and then
IcedCoffeeScript), it makes things a lot easier and clearer.

That being said, I don't think the comparison between callbacks and goto is
valid.

"Goto" allows you to create horrible spaghetti-code programs, and getting rid
of it forces you to structure your programs better.

"Await", fundamentally, isn't really anything more than syntactic sugar
(except for exception handling, which is a good thing). "Await" doesn't change
how your program is structured at all, it just changes the _visual_
representation of your code -- from indentations in a non-linear order, to
vertical and linear order. It's definitely a nice improvement, and makes code
easier to understand (and allows for better exception handling), but it's not
actually changing the way your program is fundamentally structured.

And finally, "await" is only applicable when a single callback gets called
once at the end. If you're passing a callback that gets used repeatedly (a
sorting function, for example), then normal-style callbacks are still
necessary, and not harmful at all. Sometimes they can be short lambdas,
sometimes they're necessarily much larger.

In sum: "await" is great, but there's nothing inherently harmful about
callbacks, the way "goto" is. To the contrary -- callbacks are amazingly
useful, and amazingly powerful in languages like JavaScript. "Await" just
makes them nicer.

~~~
danabramov
You're missing the point of `await`.

Await _does_ change how your program is structured, unless we're talking about
most trivial cases.

You can use `await` inside a `for` loop—can you do the same with callbacks
without significantly re-structuring your code?

What is the “callback analog” of placing something in a `finally` block that
executes no matter which callback in a nested chain fails? You'd have to
repeat that code.

Await has a potential of simplifying the structure a lot, because it befriends
asynchronous operations with control flow.

>And finally, "await" is only applicable when a single callback gets called
once at the end. If you're passing a callback that gets used repeatedly (a
sorting function, for example), then normal-style callbacks are still
necessary, and not harmful at all.

Indeed, await is only for the cases when we abuse functions (because we don't
know what we'll call, and we have to bend our minds). Passing the sorting
comparator is the primary _use_ of first-class functions.

~~~
crazygringo
I'm not disagreeing with you at all.

I guess we're using "structured" in different ways. Libraries like TameJS _do_
wind up creating structures to deal with for loops, in the same way you'd
otherwise manually have to deal with. Likewise with exceptions (which I said
are the main actual benefit to await, that can't be reproduced in normal
callback routines).

You obviously have to write a bunch of "plumbing" code to do with "raw"
callbacks, in complicated situations (like loops), which "await" does on its
own -- and writing that plumbing is annoying, although there are libraries to
help.

My only point is, the fundamental structure of your program, on a _conceptual_
level, is still the same. Everything's still running the same way, in the same
order. It's just more concise, with less plumbing of your own, using "await".
So the _micro_ structure is different with await, but the high-level structure
is no different. You can't "abuse" callbacks in the way you can abuse goto.
Maybe I should have made that clearer.

~~~
danabramov
I see your point. Closures also don't affect structure on conceptual level,
but I think they're pretty darn useful. By the way, async _can_ affect things
on conceptual level if you embrace[1] it (I posted this link somewhere below
as well, but just in case you haven't seen it).

[1]: [http://praeclarum.org/post/45277337108/await-in-the-land-
of-...](http://praeclarum.org/post/45277337108/await-in-the-land-of-ios-
scripting-users)

------
swannodette
Yes a thousand million times. This is the reason why people love golang and
why there's a lot of excitement about core.async in the Clojure community,
particularly for ClojureScript where we can target the last 11 years of client
web browser sans callback hell:

[http://swannodette.github.io/2013/07/12/communicating-
sequen...](http://swannodette.github.io/2013/07/12/communicating-sequential-
processes/)

Having spent some time with ClojureScript core.async I believe the CSP model
actually has a leg up on task based C# F# style async/await. We can do both Rx
style event stream processing and async task coordination under the same
conceptual framework.

~~~
benhoyt
Yeah, C# await/async is cool. But isn't Google Go's approach nicer? Keep
standard lib calls blocking as is, and use the "go" statement when you want to
run something async? It avoids all the extra DoWhateverAsync() API functions.
See
[http://stackoverflow.com/a/7480033/68707](http://stackoverflow.com/a/7480033/68707)

What's the Go equivalent of "await"? I.e., like the "go" statement but
asynchronously wait for the function to return and get its value?

~~~
chaosphere2112

      ci := make(chan int)
      go func(ci)
      <-ci

~~~
carbocation
If you're using `ci` as just a semaphore, you should use a struct, since those
use even less memory than an int in go.

------
lambda
You get a similar interface in Python's Twisted using the @inlineCallbacks
decorator:

    
    
        @inlineCallbacks
        def example():
            try:
                obtain_some_lock()
                ui_status("Fetching file...")
                result = yield fetch_file_from_server(args)
                ui_status("Uploading file...")
                yield post_file_to_other_server(result)
                ui_status("Done.")
            except SomeError as e:
                ui_status("Error: %s" % e.msg)
            finally:
                release_some_lock()
    

I must say that this style of writing async code is much friendlier than
descending into callback hell.

There is work to make a similar async interface native in Python 3, in PEP
3156
[http://www.python.org/dev/peps/pep-3156/](http://www.python.org/dev/peps/pep-3156/),
so this should become more widely available even to those who don't use
Twisted.

~~~
ak217
Yes! I really hope this use of yield will bubble up to the language spec and
become pervasive in Python. It really strikes me as the Pythonic approach to
solving callback hell.

~~~
masklinn
> Yes! I really hope this use of yield will bubble up to the language spec and
> become pervasive in Python.

And the time machine spaketh:
[http://www.python.org/dev/peps/pep-3156/#coroutines-and-
the-...](http://www.python.org/dev/peps/pep-3156/#coroutines-and-the-
scheduler)

(it has no reason to "bubble up to the language spec", the language provides
all the right primitives — especially with Python 3's `yield from`, it's up to
the libraries to use them. That is a Good Thing.)

~~~
ak217
Good point. Bubble up to the standard library, then.

------
shadowmint
I'm not buying the c# async/await kool-aid.

Async, sure, I'm down with that, but I've used the c# async stuff now, and
while it makes it the app somewhat faster, it has three major downsides (that
I encountered):

\- Infects everything; suddenly your whole application has to be async.

\- Debugging becomes a massive headache, because you end up in weird
situations where the request has completed before some async operation
completes, the debugger gets scared and stops working.

\- It's really hard to test properly.

The only good reason for using it is that because of the infection-property
back fitting async to your application is a major headache; if you might use
it, you have to use it from the beginning or you get a huge backlog of
refactoring and test fixes to do.

-___- 'I might use this for some performance bottleneck I don't yet know about, so better start using it now...' yeah, that's a thing: premature optimization.

~~~
ufo
But doing things by hand with callbacks is going to have all of those same
issues isnt it? If you dont want to infect everything then you basically need
to just write synchronous code instead...

~~~
raverbashing
The issue I see is that with some libraries you either go 100% async or 100%
sync with no middle ground.

~~~
danabramov
This is simply not true. You can use any C# library without using async (code
rewriter). You'd still be using Tasks but there is nothing special about them
(no compiler magic). They're just futures on which you can schedule
continuations.

~~~
raverbashing
I'm not talking about C#, but about libraries in other languages that support
async behaviour

~~~
angersock
PDFJS--the PDF reading package from Mozilla--is a gross offender in this
regard; the mix of sync and async, and the somewhat arbitrary nature of which
is which, is pretty annoying.

------
NathanKP
As someone who only recently switched to Node.js from PHP I personally haven't
had any difficulty switching over to the callback frame of mind, and I haven't
experienced the "callback hell" so many people complain about. At first I was
hesitant to start with Node because I saw blog posts by people bemoaning the
spaghetti callback code that they ended up with. But I haven't experienced any
of that, although I am a relatively newbie Node programmer with only a few
months of experience so far. My current project is quite non trivial as well,
running into the tens of thousands of lines so far.

The key I've discovered to nicely organizing callbacks is to avoid anonymous
callback functions unless absolutely necessary for a particular scope, or
unless the function is going to be so trivial and short that it can be read in
a single glance. By passing all longer, non trivial callback functions in by
name you can break a task up into clear functional components, and then have
all the asynchronous flow magic happen in one concise place where it is easy
to determine the flow by looking at the async structure and the function names
for each callback function.

Another major advantage to a code organization like this is that once you have
your code such that each step has it's own discrete function instead of being
some inception style anonymous function tucked away inside another function
inside another callback it allows you to properly unit test individual
functional steps to ensure that not only is your code working and bug free at
the top level functions, but also that each of the individual asynchronous
steps that may make up some of your more complicated logic are working
properly.

Most of the bad examples of callback hell that I see have anonymous callback
functions inside anonymous callback functions, often many levels deep. Of
course that is going to be a nightmare to maintain and debug. Callbacks are
not the problem though. Badly organized and written code is the problem.
Callbacks allow you to write nightmarish code, but they also allow you to
write some really beautiful and maintainable code if you use them properly.

~~~
kvb
I often write synchronous methods that include control flow that nests three
deep (say try/finally, if/then/else, and a for loop). Often it's easier to
read this code than it would be if everything were split out into separate
named methods.

Why would the same not be true of asynchronous methods, assuming that the
technology was there to enable it (as it is in C#)?

~~~
stcredzero
_> Often it's easier to read this code than it would be if everything were
split out into separate named methods._

This might actually be a shortcoming of our code organization/reading tools.

~~~
ufo
Why should that be the case? Creating all the extra methods is going to create
lots of new points of indirection, the new methods are likely to be tightly
coupled anyway and breaking the nesting might mean you have to hoist a bunch
of variables into an outer scope.

~~~
stcredzero
Exercise: list all of the implicit assumptions about how code organizing and
reading tools have to work from those two sentences.

------
davidhollander
Anonymous Callbacks != Callbacks

Callbacks have been around forever in C using named functions, and are not
specific to either the current generation of programming languages or
programmers. One can still use a _named function_ instead of a locally
constructed lambda to represent a callback in a high level languages.

The primary difference is that when declaring named functions non-locally, one
must explicitly share state through the parameter rather than implicitly
sharing state through lexical scoping. It seems more accurate to label the
problem of nesting lambdas to the point of ambiguity as "Lambda Abuse" or
"Lexical Scoping Abuse" rather than "Callback Hell".

~~~
haimez
Not totally, the problem being described as "Callback Hell" is in my
experience: non-locality of code. Things that relate to one another should be
spatially together for a programmer to write, read, understand- and NOT split
into success and error conditions around the axis of final callback.

Even with named functions, this is the problem- right? You pass in a named
callback and you have to find that function later when you're debugging to
figure out what's going on. Locality of code means no breaking context to find
something not already on screen and therefore easier programming.

------
informatimago
The problem is simply that those languages are not Lisp.

Once good patterns of use of GOTO were found, it was natural to critisize
random uses, and to wrap good uses in a lisp macro. Or in a new while or for
"instruction".

But then the next construct is discovered, and its bad uses considered
harmful, and its good uses need to be wrapped. In lisp, mere programmers will
just write the next macro to abstract away this new construct. Other
programming languages need to evolve or have new language invented with new
"instructions".

So now it's the callbacks. Yes, in lisp we'd just use closures, but this is
only a building block for higher level constructs. If those "callbacks" are
needed to represent futures, then we'd implement those futures, as a mere lisp
macro.

Yes, in the other languages you're still powerless, and need to wait for the
language designers to feel the pressure and implement a new "future"
instruction or whatever.

Any language construct can be considered harmful eventually. Concretely, a
repeative use is a hint of that: the construct becomes meaningless because
it's used all the time, or apparently randomly (just like those GOTOs). But
it's not that the construct is bad, it's that its usage is not abstracted away
in higher level constructs. And the only way to do that is lisp macros.

So unless you're programming in lisp (a homoiconic programming language that
let you write easily macros in the language itself), you will always reach a
point where some construct will be able to be considered harmful for lack of a
way to abstract its uses away.

~~~
ScottBurson
Ah, but the real problem is not that these languages aren't Lisp, but that
they aren't Scheme. People are being forced to write continuation-passing-
style code by hand, which anyone would agree is painful. To allow _user-level
code_ to abstract away the need to write CPS, you need call/cc (or something
like it).

~~~
informatimago
You would never use call/cc directly. Like goto, it is a building block to be
used in control abtraction macros.

~~~
ufo
Yes. The point is that while the idea of continuations exists, no one should
be forced to write continuation-passing-style code by hand!

------
danabramov
There is some creative use of C# async/await in this blogpost:

[http://praeclarum.org/post/45277337108/await-in-the-land-
of-...](http://praeclarum.org/post/45277337108/await-in-the-land-of-ios-
scripting-users)

Basically, the author implements a “first time walkthrough” kind of interface
a-la iWork very declaratively by using async:

    
    
        	async Task ShowTheUserHowToSearch ()
        	{
      		await Tutorial.EnterText (searchField, minLength: 3);
      		await Tutorial.Tap (searchButton);
      		await Tutorial.Congratulate ("Now you know how to search.");
      	}

~~~
MichaelMoser123
What if you have two buttons, search and 'be lucky'? I guess you will have to
await for a controller object that wakes up that if either one of the buttons
is triggered, and then test what button was pressed.

Now what if you have add a search option dialog that can be invoked at any
moment? I guess when the 'apply' button of the option dialog got pressed then
this is still going to be a callback that sets global variables according to
search option.

So, like any mechanism, this way of structuring code has its limits. If an
event can happen at any stage of the flow, then this still has to be handled
by a callback, otherwise you would have to check for this kind of event after
each await clause.

~~~
thallium205
"What if you have two buttons, search and 'be lucky'? I guess you will have to
await for a controller object that wakes up that if either one of the buttons
is triggered, and then test what button was pressed."

[http://msdn.microsoft.com/en-
us/library/hh194796.aspx](http://msdn.microsoft.com/en-
us/library/hh194796.aspx)

~~~
MichaelMoser123
... select or WaitForMultipleObjects in another form

------
fzzzy
I agree, although I think callbacks are more like COME FROM than goto. You see
a function being passed somewhere as a callback, and you know the block is
going to execute at some point, but most of the time you have no idea what the
codepath that calls you back looks like.

There's nothing more frustrating than trying to debug why a callback isn't
being called. Who calls it? How do I set a breakpoint somewhere to see why it
isn't being called? etc.

The one thing that is still missing from await and other green thread
approaches is cheap global contexts. Isolating every different green thread so
they can't implicitly share state is the obvious next step.

~~~
nickmeharry
Have you looked at the Erlang model? Each Erlang process (green thread) only
gets the arguments initially passed in and whatever else it asks for from
other running processes. The only shared state is long-running processes
created for the purpose of explicitly sharing state.

~~~
fzzzy
Yep. The Actor model that Erlang uses is exactly what I was referring to being
missing from green thread libraries for other languages (C#, python with
greenlet or generators, js with generators)

------
danenania
I generally agree that there are better ways to handle asynchronous control
flow than callbacks, but I think this is exaggerated. As in most posts like
this, the callback soup examples are difficult to follow primarily because
they are horribly written, not because of callbacks.

As long as you write decent code, the main impediment to asynchronous
programming is reasoning asynchronously, not syntax. If you require complex
asynchronous logic and don't use an appropriate algorithm, you'll end up in
the muck whether you use callbacks or await.

Taking go as an example: while I agree that the go statement is more elegant
than a callback approach, I see it as quite a minor win compared to channels.
The go statement is convenient syntax, but channels are what make concurrency
in go feel so robust, and it's a pattern than can be applied just as well in a
language that uses callbacks.

------
ryan-allen
I don't understand what the big deal is. Callbacks are OK. They're less
cumbersome if the language you're using has smaller function definitions.

Callbacks 'get crazy' when you've got more than one I think, and thankfully
someone smart has made a library you can use to manage them!

[https://github.com/caolan/async](https://github.com/caolan/async)

Saying that, I don't mind the way things look with the whole await/async stuff
in C# and etc. However I don't think we should be waving our arms around
saying callbacks are like goto, they so completely are not! I have written
heaps of stuff with callbacks and it's _not that confusing or unmaintainable_.
It's just different.

~~~
danabramov
How do you do this with callbacks?

    
    
        foreach (var player in players) {
            while (true) {
               var name = await Ask("What's your name");
               if (IsValidName(name)) {
                   player.name = name;
                   break;
               }
            }
        }
    

Assuming `Ask` is an asynchronous operation and must not block the UI thread.

Note that second player is only asked _after_ the first player has given a
valid name.

(And the code structure reflects that :-)

My point is of course it's _doable_ with callbacks, but I spent more time
indenting this code than writing it, and I darn well know _I_ 'm not smart
enough to spell out the correct callback-style code in a comment field on
Hacker News. And if I suddenly had to add error handling...

~~~
craigyk

      for player in players
    	get_name_for = (player) ->
    		ask "what's your name?", (response) ->
    			if is_valid_name response
    				player.name = response
    			else get_name_for player
    	get_name_for player

~~~
anonymoushn
This looks like it does the wrong thing. If "ask" gets to access the
scheduler's task list as a queue, then entering an invalid name in the first
response and only valid names thereafter will cause the first valid name to be
given to the second player, the second valid name to the third player, and so
on. If "ask" gets to access the scheduler's task list as a stack, then a
sequence of only valid input names will cause the last player to have the
first input name, the second-last player to have the second input name, and so
on.

Edit: I was optimistically assuming that the consumer of the many "asks" that
are created all at once would process them sequentially, dealing with one and
invoking the callback before dealing with the next. If you do not assume this,
my problem disappears and you get the simpler problem of spawning many prompts
simultaneously.

~~~
craigyk
Hmm, I wrote it so that the function closes over the player, so that shouldn't
happen. The real issue is, as pointed out, that this will ask for all the
names at once, rather than sequentially. Wether this is bad or not depends on
how the `ask` function gets its input.

------
cscheid
Holy Baader-Meinhof, just today, in frustration, I wrote something like
Haskell's sequence_ for ContT, in Javascript:

[https://gist.github.com/cscheid/6241817](https://gist.github.com/cscheid/6241817)

~~~
nkuttler
Why do you as an American feel the need to invoke some dead german left-wing
militants in a pseudo-religious phrase that's meaningless except maybe for
shock value? This seems highly inapproriate for any website and even more so
on HN.

~~~
cscheid
(for whatever's worth, I'm Brazilian)

~~~
manojlds
Aren't you still American? ;)

As an Indian I am confused why only people of the US are called Amerians while
two entire continents are called America.

And also, why we Indians are not considered Asians by the said Americans.

~~~
lazugod
Because "America" is part of the nation's actual name, and the only part that
isn't a modifier. What else could you call Americans? Unionized Statists?

~~~
nhaehnle
I actually use the term "USian" quite frequently, but then I'm weird that way.

------
dantle
(C/OS developer spiel)

I'm sick of these app developers assuming that using "goto" is bad practice.
The fact is that "goto" is used plenty in great production code you're
probably running right now.[1] I'd like to know a cleaner way to abort a
function into its cleanup phase when a function call returns an error. And
"goto" statements are extremely simple to handle for even the most naive of
compilers.

[1]
[https://www.kernel.org/doc/Documentation/CodingStyle](https://www.kernel.org/doc/Documentation/CodingStyle)
(see chapter 7)

~~~
pjmlp
Exceptions

~~~
Niten
Not terribly plausible in kernel code.

~~~
pjmlp
Not all kernels are UNIX.

~~~
dantle
I didn't know UNIX won't allow kernel exception handling. Source?

Of course exceptions are suboptimal; bad performance compared to "goto" can
have unintended side effects depending upon the OS (scheduling work may be
triggered off of the additional interrupt).

~~~
pjmlp
> I didn't know UNIX won't allow kernel exception handling. Source?

My statement was based on ideology as I doubt typical UNIX kernel coders would
ever allow for exceptions, given that C does not support them and is against
the UNIX way.

> Of course exceptions are suboptimal; bad performance compared to "goto" can
> have unintended side effects depending upon the OS (scheduling work may be
> triggered off of the additional interrupt).

Exceptions at the kernel level are possible, Windows does it for certain
classes of errors, for example.

Some other commercial or research kernels might do it as well.

------
worldsayshi
Evan Czaplicki (author of Elm lang) made the identical argument
(sometime?/years ago), with the same reference to Dijkstra's quote, but with
another suggested solution, Functional Reactive Programming, on which his
language is oriented:

[http://elm-lang.org/learn/Escape-from-Callback-Hell.elm](http://elm-
lang.org/learn/Escape-from-Callback-Hell.elm)

~~~
jcurbo
HN discussion of that article:
[https://news.ycombinator.com/item?id=4732924](https://news.ycombinator.com/item?id=4732924)

FRP is an interesting topic (I thought so anyway, I wrote a paper on it for my
MS). It doesn't seem to have caught on widely as a paradigm, with a few
exceptions I'm aware of (Elm, Meteor).

~~~
zcrar70
Rx (for C#), RxJS (for Javascript), Bacon.js.

Definitely not 'widely' but there are a few library implementations out there.

I reckon that the reason for this is that it's most useful when the
interactivity is high (i.e. there are a lot of events to react to), but most
applications (desktop, web) don't have a high enough number of events to make
learning a new paradigm 'worthwhile'.

------
solistice
I was pleasantly suprised that catoverflow is an actual site.

------
zzzcpan
Anonymous callbacks are very powerful and very important. They will make you
feel bad for unnecessary nesting. They will force you to learn how to abstract
better, especially state changes. They will show you how nice and reliable
code can be if it doesn't have shared states across multiple functions and how
easy it is to understand consistent code with explicit continuations and how
to write one yourself. They will make you a better programmer.

And "await" can only make it harder to visually distinguish which piece of
code is executed in parallel and which is executed sequentially. Nesting makes
it explicit.

------
adamconroy
Why do people insist on analysing things using analogies? Analogies are useful
for explaining a concept that might not be obvious. Saying callbacks are like
gotos, gotos are bad, therefore callbacks are bad is ridiculous.

And he gives some sample code where the 'problem' is nothing to do with
callbacks, its just nested lambdas. In fact I find that code quite easy to
read, and would be very interested in seeing the same functionality
implemented some other way, bearing in mind it is quite a difficult problem to
synchronize multiple async operations and usually requires horrible code using
multiple mutex.

~~~
zowch
I'm confused as to why you think the code presented in the blog post isn't an
example of 'the same functionality implemented some other way'. The await code
is almost undeniably more straightforward and the exceptional cases more
obvious to handle.

He also doesn't seem to be making a weird logical leap the way you claim he
is. He's not really using an analogy. He's saying callbacks are bad _the same
way_ goto is bad, in that they make the logical structure of a program's
execution non-obvious, particularly over time and when being modified.

~~~
adamconroy
I'm not saying the await syntax isn't useful or a better way to implement that
problem. The title and opening of the article is comparing callbacks with
gotos. It would be logical if he titled the article "C# Await syntax trumps
callbacks when multiple callbacks need to be synchronised".

Any syntax can be used to create spaghetti code. Callbacks only become
spaghetti like when nested / chained / abused. I've written hundreds of API
methods using callbacks that I believe are very clean. I've recently written a
large API which makes async service calls that follows this pattern :

void GetInvoiceHistory(int? customerId, Action<List<InvoiceItem>> callback,
Action<string, Exception> exceptionCallback);

The consumer of this API does not need await, there is no need to nest
callbacks, there is no need for try/catch and it is very clean to work with.
So IMHO, stating that callbacks are this generation's gotos is a pile of shit.

------
cbrauchli
This definitely is a problem in Obj-C. Using GCD and callbacks is usually
easier to understand than delegates and manual thread management, but it's
still not great. I would love to see something like async/await in Obj-C.
There are some great ideas on how to get something similar in this blogpost,
but none that I would use in production code unfortunately:
[http://overooped.com/post/41803252527/methods-of-
concurrency](http://overooped.com/post/41803252527/methods-of-concurrency)

------
eonil
It's right that callback model sucks, and the _task_ model is a way to go.

    
    
      Sadly, many developers when they hear the word "C# async" 
      ...
      All of these statements are made by people that have yet
      to study C# async or to grasp what it does.
    

But it's unpleasant to see the author is talking concept of task - lightweight
threading, coroutine, or whatever - is like a patent of C# (or F#). And
furthermore, treating _many developers_ are not able to understand this
concept.

Maybe true for the people around him.

I understand his position as a lead developer and an evangelist of Mono/C#,
but this attitude is ridiculous.

~~~
danabramov
Have you read the article? It's not about tasks per se, it's about a code
rewriter.

~~~
eonil
This article is about structural paradigm shift which foresaw for a long time
from Actor model, Erlang, and recently to Go and Rust.

How is this just only an introduction to a new syntactic sugar?

------
pfraze
ES6 generators combined with promises will bring this to the javascripters:
[http://taskjs.org/](http://taskjs.org/)

------
robomartin
In the right places and for the right reasons they are fine. A lot of code
today devolves into what I've come to call "callback spaghetti" and, well,
good luck. The toughest thing sometimes is getting your mind around what is
supposed to happen and, more importantly, what is not.

I found that sometimes it helps to build a state machine to effectively run
the show and try to limit callbacks to setting flags and/or navigating the
state tree. State machines make following code functionality a breeze, even
when dealing with really complex logic.

~~~
danabramov
That's a smooth point! As you probably know, async code is translated by C#
compiler to a state machine[1].

[1]:
[http://stackoverflow.com/a/4047607/458193](http://stackoverflow.com/a/4047607/458193)

------
tonetheman
Yup yup yup. I thought I was the only one who noticed that node had reinvented
the Windows 3.1 programming loop.

~~~
marshray
Windows 3.0 had it before Windows 3.1.

------
spiritplumber
Callbacks are basically COME FROM, epecially on a platform like a cell phone
where you at least in theory have limited processing resources ($40 android
phones need apps, too!). They are the devil.

~~~
dragonwriter
> Callbacks are basically COME FROM

No, they aren't even similar to COME FROM. COME FROM is "upon reaching label
X, jump to this point". Callbacks have _less_ in common with COME FROM than
with GOTO, and less in common with GOTO than with normal procedure/function
invocation.

> epecially on a platform like a cell phone where you at least in theory have
> limited processing resources

Platform is completely orthogonal to the relation between callbacks and other
programming constructs.

~~~
spiritplumber
I see your point, but they read a lot like come from when looking at code.

And platform is not orthogonal: sometimes I need to know exactly what is using
how many cpu cycles, and callbacks make it hard.

Admittedly I mostly do embedded dev, but there's a reason why I have the only
remote control / video app out there that still works on a HT G1 :)

------
AndyKelley
iced-coffee-script has a similar solution. [http://maxtaco.github.io/coffee-
script/](http://maxtaco.github.io/coffee-script/)

~~~
molf
Iced Coffeescript is brilliant! I think it is the closest you can get right
now to sane development with callback-oriented code in Javascript.

Unfortunately it doens't solve the exception problem yet. Exceptions passed to
callbacks (as `(err, value)`) are not thrown. So instead of try/catch there
will be lots of `return cb(err) if err` in your code.

------
seanmcdirmid
I was expecting to read something about FRP or other naturally reactive
programming models that dealt with the semantic complexity of callbacks, not
just their syntactic complexity. I don't think async constructs and others
that depend on CPS techniques are really going to save us from complex
programs that we barely understand.

------
btilly
Meh.

Callbacks are a very limited way to do asynchronous programming. However they
are a good way to create interfaces that let you call methods and insert your
own functionality in the middle.

So yes. Better async is good. But don't take away callbacks. They have their
uses.

------
halayli
lthread is a coroutine library that allows you to make blocking calls inside
coroutines by surrounding the blocking code with lthread_compute_begin() and
lthread_compute_end(). This is equivalent to async calls but without the need
to capture variables.

[http://github.com/halayli/lthread/](http://github.com/halayli/lthread/)

Disclaimer: lthread author

------
kudu
Instead of using callbacks, golang embraces synchronous-style calls and makes
them asynchronous by switching between goroutines (lightweight threads).
gevent (for Python) does something similar. It's certainly an interesting
approach IMO.

------
rogerbinns
I noticed that most of the methods awaited on had an Async suffix in their
name. Is that some sort of modern hungarian notation, and is it even
necessary? It also looks like you can't pass timeouts to await.

~~~
hokkos
The Async suffix is here to differentiate with the synchronous version of an
existing method returning T, the asynchronous return a Task<T>. With a Task<>
you can do :

    
    
      int timeout = 1000;
      var task = SomeOperationAsync();
      if (await Task.WhenAny(task, Task.Delay(timeout)) == task) {
          // task completed within timeout
      } else { 
          // timeout logic
      }

~~~
rogerbinns
According to the Microsoft doc[1] it is convention for async to return a Task
but not a requirement. Since the return type of Task tells you a method is
expected to be used asynchronously, also encoding that in the name seems
somewhat redundant. I guess you have to use a different name because of static
typing so Async is as good a suffix as any.

[1] [http://msdn.microsoft.com/en-
us/library/vstudio/hh156528.asp...](http://msdn.microsoft.com/en-
us/library/vstudio/hh156528.aspx)

------
vidarh
This is all simply sugar to hide behind-the-scenes threading behind very
narrow interfaces. Which isn't necessarily bad, but it's fun to see it
suddenly in favour again and presented as something new.

E.g. Simula67 had Call and Detach for the basic case, and Activate(object
representing async behaviour) and Wait(queue) that would both depending on
need often be used for the same purpose (as well as a number of other
operations). We had to write code using those methods in Simula67 in my
introduction to programming class first semester at university...

~~~
zwieback
That was my thought. I always get lost in these discussions because I have to
translate the bizarre lingo into basic threading primitives I can understand.
Once I've done that I don't understand what all the fuss is about.

~~~
vidarh
I think the issue is that most programmers actually _never_ touch threading,
or if they do, they get complicated threading models shoved in their faces and
run away in horror (which is a reasonable reaction). But these constructs
looks sort-of like they're just wrappers around callbacks.

Never mind that the only reason callbacks are all that interesting in JS is
because they allow the engine to sneak in threading behind your back - most
people have a very woolly idea about how JS execution happens and the fact the
javascript execution itself isn't threaded.

------
voidr
You can write any article like this about anything, here is the formula:

\- pick a language feature

\- write an article with the title "<language feature> as our Generation's
Goto Statement"

\- write an example where you misuse <language feature> and over generalize it

\- show a workaround that doesn't really save the trouble of actually thinking
before typing

The hard thing about callbacks is that you need to think about asynchronous
processes which can be hard, the callbacks are not the problem so replacing
them with something else won't help you too much.

------
tehwalrus
I am very happy that my current (PhD) code doesn't require me to deal with
blocking calls to things (it's just one massive calculation essentially). I
remember this nastiness from when I had a real job, and I'll no doubt have to
deal with it again when I escape academia.

This article is very interesting - I enjoy articles which spell out the
usefulness of a new language feature. I haven't used C# for a few years, and
this is a great advert for coming back to it one day.

------
javert
I have a basic technical question. I work in C for embedded systems, so I'm a
bit "behind the times."

How is "await" any different from a regular blocking system call? A regular
system call does exactly what is being described: The system call happens, and
then when it is finished, execution resumes where it left off.

(Yes, this makes the thread block... which is why you have multiple threads. I
think the answer will have something to do with this, though...)

~~~
marcosdumay
I'm not a beliver, so I may be missing something important...

As far as I can see, await makes it possible to do things like:

\- UI Thread starts Thread X to do something slow; \- Thread X is processing,
in the meantime the user clicks on something that requires X's result; \- Only
now the UI thread stops, waiting for X.

Your program stay responsive, unless it's completely not able to do so.

As I said, I still don't think it's a so important feature. It is quite rare
that one needs such kind of coordination in UI programs, and there are better
mechanisms for no-interactive software (that optimize for throughput, not
responsiveness)... As a second thought, it may be very relevant for games, I
don't know.

I also don't get why the node.js people consider it so important to have
concurrence at a web server. For me, it only makes sense for sites that have
less concurrent requests than server cores, AKA: nobody.

~~~
danabramov
No, you got it wrong. Await _never blocks_ the UI thread.

Instead, the compiler rewrites your sequential code into a state machine. When
you await on a task, the compiler turns this into scheduling a continuation.
No blocking.

~~~
marcosdumay
Yep, I got it wrong. Thanks for pointing that.

Now I'm also wondering how I'd use something like that in an imperative
language... Well, I have some studying to do.

------
lists
Completely OT, but when Dijkstra says:

>My second remark is that our intellectual powers are rather geared to master
static relations and that our powers to visualize processes evolving in time
are relatively poorly developed. For that reason we should do (as wise
programmers aware of our limitations) our utmost to shorten the conceptual gap
between the static program and the dynamic process, to make the correspondence
between the program (spread out in text space) and the process (spread out in
time) as trivial as possible.

He's touching on a crucial point in Immanuel Kant's philosophy. Kant theorized
that though humans received their sensations as a constant stream of input in
time (which is an internal condition of human beings, not a feature of bare
reality), we can't actually do anything with that stream without applying
concepts so as to form concrete (or abstract) objects, i.e. chairs, black
holes, mothers, etc. But how would our minds know when to apply this concept
or the other? Kant's reply was that our minds look for little clues called
'schematisms' which tell us what the most appropriate fundamental concept to
apply to a part of the stream is, upon which others could be combined to
produce objective representations we can think and act upon.

Almost a hundred years later, Nietzsche will claim (paraphrasing) that a
measure of strength in a human being is the extent to which to which they can
'consume' phenomena in time, weakness being how direly one needs to apply a
static idea to phenomena (like morals, stereotypes, prejudices, cause and
effect, etc).

I'm just noting an interesting entry point into an old philosophical
conversation. If it's understandable then I hope someone finds it interesting.

------
kirab
The only thing I’ve encountered in modern day programming which is really as
evil as goto is aspect oriented programming (AOP). Maybe there are different
implementations of AOP but in the one I’ve used you were basically able to
hook into every method from everywhere and it was impossible to have any grasp
on the flow of the program. That is besides using a debugger.

------
dpweb
JS gives you the tools to cope..

    
    
      function sequence(fns){
       var fn = fns.pop(); 
       while(fns.length) fn = fns.pop().bind(this, fn); 
       fn();
      }
    
      sequence([
        function(k) { funcy(1, k) },
        function(k) { funcy(2, k)  },
        function(k) { funcy(3, console.log)  }
      ]);
    
      function funcy(v, cb){
    	console.log(v);	cb(v);
      }
    
      // ==> 1 2 3 3

------
cpprototypes
ES6 generators will be the solution for callback hell in node.js. Node 0.11
already has generators support hidden behind a flag (--harmony-generators) and
eventually it will be enabled by default. Generators + libraries like this

[https://github.com/jmar777/suspend](https://github.com/jmar777/suspend)

will make node.js code more readable.

------
RHSeeger
Callbacks are a tool in my toolbox. General event based programming is a tool
in my toolbox. Various threading models are a tool in my toolbox. Just because
there are situations where a tool is not the best choice does not make the
tool bad... it means you use a different tool in that case.

------
kbradero
Miguel is always fun/good to read.

------
10098
Can someone explain to me the difference between this and futures
(specifically, futures in c++11)?

~~~
platz
I can't comment on futures in c++11, but async/await is more like promises +
coroutines, rather than just promises (as defined by Promises/A+)

------
racketeer
This is what continuations are for.

~~~
rgbrgb
Continuations don't help with the problem that the visual structure of
callback-oriented programs doesn't reflect the order of execution. As a heavy
JS programmer, that's the most compelling point for me in this post.

~~~
anonymoushn
Actually, they do. C#'s await (and any similar scheme based on ES6 generators)
is a feature that can be built using continuations.

~~~
dragonwriter
So are Java/C#/everything-else exceptions. And, for that matter, pretty much
every control flow construct imaginable.

OTOH, I think that the big problem with continuations is that it gets very
difficult to build efficient implementations of them (and this tends to impact
not just efficiency of code that _uses_ continuation, but usually efficiency
of any code in a language which _supports_ them), and it is much more
efficient to implement specialized weaker (but good enough for most key use
cases) forms of the most important applications of continuations.

~~~
anonymoushn
Supporting call/cc and dynamic-wind has a significant performance impact in
some languages, even for code that does not use the features.

Supporting coroutine.create+coroutine.clone, shift+reset, or
setcontext+getcontext+makecontext+swapcontext seems to have no performance
impact on code that does not use the features.

------
orp
Akka adds something similar in Scala land (and Java) called Dataflow
concurrency.

[http://doc.akka.io/docs/akka/snapshot/scala/dataflow.html](http://doc.akka.io/docs/akka/snapshot/scala/dataflow.html)

------
gnufied
How is this different from Fibers in Ruby? One can accomplish same thing with
Fibers.

------
jostylr
To appreciate whether callbacks are Goto and what to do about them, it is
probably good to read a good perspective on Goto from back in the day:
[http://cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammingGoT...](http://cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammingGoTo.pdf)

When skimming it, I noticed the appeal to events and the precursors to
literate programming (Knuth eventually came up with literate programming a few
years after this paper was written).

------
sukaka
futures are so nice to use in scala. warping back to my planet..

~~~
wheaties
Scala has both Futures and Promises which are inherently much nicer than that
async call. That's what happens when you allow things to compose. Glad c# gets
something.

------
hderms
I love when Node.js advocates try to convince you that promises are as good a
concept as anyone would need to handle asynchronous programming

~~~
ricardobeat
Promises were removed from node core a long time ago, and remained unpopular
until very recently. They are making a comeback due to lobbying in standards
committees, forward-compatibility with ES6 generators, and the jQuery effect.

~~~
hderms
Fair enough. I guess I'm confusing IRC with reality again.

------
OldSchool
As soon as you have anything involved that's async to your process or thread,
you're going to operate most efficiently with something along the lines of a
callback. I don't see them as a Goto at all; they're much more like interrupt
handlers or at least event handlers if you want jump ahead a generation from
there.

------
itengelhardt
Having had the pleasure to work with node.js for the past months, I upvoted
this submission on title alone.

------
swiil
I actually believe that code should be synchronous unless instructed to
operate asynchronously - just my .02

Await should not be required - it should be more like...

\--

regularWork(); //im waiting till this thing is done

driveHome();// not executed till thing one is done

background orderStatus = orderPizza();

turnOnXbox();

while(orderStatus == 'not ready') {

playXbox();

}

turnOffXbox();

eat();

\--

Like I said - just my humble opinion that the code written would become more
expressive.

------
olegp
Agreed, that's why we're using fibers and common-node
([https://github.com/olegp/common-node](https://github.com/olegp/common-node))
at [https://starthq.com](https://starthq.com)

------
etler
I always liked event based systems the most. I find them to be clean and
flexible. Sometimes you want to run some more code after you run an async
operation, or you want to run multiple operations at once and deal with them
out of order. Await seems pretty linear.

------
Tichy
Does Await convert those async calls back into synchronous calls, or what does
it do? Because that would be kind of defeating the purpose of doing things
asynchronously?

And you don't have to nest all those callbacks and write them inline.
Rearrange your code a bit.

~~~
danabramov
No, it rewrites the method code into a state machine[1].

See Async/Await FAQ[2].

[1]:
[http://stackoverflow.com/a/4047607/458193](http://stackoverflow.com/a/4047607/458193)
[2]: [http://blogs.msdn.com/b/pfxteam/archive/2012/04/12/async-
awa...](http://blogs.msdn.com/b/pfxteam/archive/2012/04/12/async-await-
faq.aspx)

~~~
Tichy
I'll have to look at that in the morning, thanks!

------
Tloewald
The ease with which callbacks can be created leads people to create them
carelessly and excessively. While I like what the article has to say, there
are ways to write callback heavy code that do not get ugly so fast. Looking at
the iOS nested block example from Marco Arment -- the first step is to not do
everything inline. Then the code suddenly becomes clear and the argument
becomes one of syntax sugar.

Comparing callbacks to goto is a tad unfair. They don't merely solve async
issues, but also event handling and dynamic systems to name two common uses. I
don't see a better solution on the table. Using callbacks to write deeply
async code is the real problem. And while async/await may help with this
problem, it still won't tell you why step 3 never finishes, because it's still
waiting for a come from.

~~~
molf
First: code that performs simple sequential steps should look simple. With
callbacks, it always ends up looking complicated.

Second: refactorability in callback oriented code is a lot worse than linear
code. Even refactoring code with a single callback can be annoying. Async code
with callbacks that looks and feels like imperative synchronous code is an
enormous gain.

~~~
Tloewald
First: you're hammering the async point, which I don't disagree with. I am
just pointing out that you can write less horrible async code than the example
cited with callbacks.

Second: wait until people write ludicrous numbers of async routines as if they
were imperative because it's so easy. Calling one from another (and hooking
them up to event handlers). You'll have the same damn problem one level
removed.

------
tome
Here's a cool way to reinvert control

[http://blog.sigfpe.com/2011/10/quick-and-dirty-
reinversion-o...](http://blog.sigfpe.com/2011/10/quick-and-dirty-reinversion-
of-control.html)

------
dap
"I have just delegated the bookkeeping to the compiler."

That's not obviously a good thing. Debugging the compiler (or just figuring
out why it did something, even if correct) is far more difficult than
debugging application code. Given the choice between implementing behavior
with application code (or a library function) or adding semantics to the
language, I prefer the former because it's much easier to reason about code
written in a simple language than to memorize the semantics of a complex
language.

[edited to replace sarcasm]

~~~
danabramov
This is a nonsensical comment, and I voted it down. The same point can be made
about any time languages got a level higher. This kind of rejection of
powerful in favor of complex-but-familiar is precisely what Bret Vector warns
against in the Future of Programming talk[1].

If anything, `await` makes debugging easier because you don't have to untangle
callbacks and jump back and forth. You're not supposed to “debug the compiler”
because, well, you know, there are test suites and everything.

 _Yes_ , this is something that takes getting used to. Just like `for` loops,
functions, classes, futures, first class functions, actors and many other
useful concepts and their implementations.

As for your edit, I still can't agree with you.

You're saying:

>I prefer the former because it's much easier to reason about code written in
a simple language than to memorize the semantics of a complex language.

The point of `async` is making the semantics _more obvious_. Is it much easier
to reason about Assembler than C? I say it's not. Would it be for somebody
with years of experience in ASM and none in C? Yes it would.

I think it just comes down to that. Callbacks seem simpler to you not because
they _are_ simpler (try explaining them to someone just learning the language,
and you'll see what I mean), but because you got used to them. Even so, error
handling and explicit thread synchronization make maintaining callback-ridden
code painful. I think setting `Busy` to `false` in `finally` block is a great
example (in the blog post). You just _can 't do that with nested
callbacks—they are not that expressive_.

Async allows you to think in structure (`for`, `if`, `while`, etc) about time,
that's why it's powerful.

[1]: [http://vimeo.com/71278954](http://vimeo.com/71278954)

~~~
dap
The sarcasm in my post was unnecessary, so I've replaced it with a better
explanation.

~~~
danabramov
Thanks for taking time!

I still don't agree though, I edited my post as well to explain why I think
this is exactly the moment you need to tweak the language, and not the
libraries. (And this is the point Miguel was trying to make when he
differentiated `async` from “futures” libraries, even from the one `async`
uses, because they are irrelevant to the discussion.)

------
jheriko
callbacks are like goto in that you can create terrible code by using them
badly, but also they are vital to implementing good code. if, for, while and
co are all syntactic sugar for correct and standardised use of goto with hints
to help the compiler make optimisation.

in both cases though we don't something universally evil or bad - just
something that bad programmers can and will abuse.

------
blacksqr
I love it -- another dramatic unveiling in a cutting-edge language of a
feature Tcl has had for decades (google "vwait").

~~~
danabramov
Oh, can we use Tcl on iOS maybe? No? How about Android? Xamarin runs on both.

------
xedarius
I can see how this could be useful for Javascript and web stuff. But it isn't
Async, this is in effect a blocking call.

------
ilaksh
In ToffeeScript you can just do e, data = readFile! fname

Or take a look at CoffeeScript or IcedCoffeeScript or LiveScript back calls.

------
ttflee
One instant cure:

Make rules in IDE/editors, for each async/anonymous closure, undo the indent
for one level.

------
strictfp
So... This article is basically saying that blocking style programming is a
lot easier to read and write, and proceeds with demoing a lib which makes
async calls look sync. So instead of doing this in $lang, why not invest time
in making blocking style faster on the kernel level? Perhaps introduce actors
or tasks in the kernel, so that every lang can benefit.

------
rbanffy
I think the problem is not the callback itself, but the nested inline
callback.

In any case, I prefer futures.

------
boopbeep
shenanigans. Pyramids (at least in js) can be easily avoided by simply naming
your functions. Treating them like the first class objects they are. Naming
the function means you are no longer going to an arbitrary code block, but
instead going to a concept whose name, doc string, and (through hoisting)
position on the page illuminate its purpose.

Callbacks aren't bad. Pyramids are bad. Stop writing pyramids.

Nodejs also establishes a nice api for callback functions-- in particular,
callbacks are defined with an `error` and a `data` argument. You handle the
error if it is non-null, otherwise execute `data`.

If you want to avoid callbacks, Node also provides event emitters, and
streams. Streams in particular provide a nice api for dealing with event based
programming.

~~~
fauigerzigerk
I disagree. Callback based programming forces you to modularize around
asychronous/IO calls instead of semantic cohesion.

Naming functions doesn't solve that problem. On the contrary, you get seperate
lexical units that are not separately reusable and make no sense on their own.
They are simply fragments of code that's supposed to be executed before or
after some IO operation inside another function.

------
Murk
await PostPicToServiceAsync(mFile.GetStream (), tagsCtrl.Tags);

Seems to be equivalent to a blocking call of old, or am I missing something?

~~~
danabramov
Yes, you are :-)

The compiler rewrites your code to schedule all next lines as a
_continuation_. There is no blocking.

------
stupejr
At least for java, when your software library exists in the cloud, I don't see
how you could avoid using callbacks.

~~~
kansface
This is possible if the language supports coroutines like python (see
twisted's inline callbacks) or my port to Lua for Luvit:
[https://github.com/kans/luvit-inlineCallbacks](https://github.com/kans/luvit-
inlineCallbacks).

Personally, I think raw callbacks are the right thing 95% of the time.
yield/async/coroutines are needed for branching async logic where otherwise
you'd be forced to make a new function for each branch and deal with the
spaghetti at the end.

------
coreyja
I think this article misses a main point and that's the fact that all await
does it take a function that used to be asynchronous and makes it synchronous.
While yes, there are definitely use cases where that is nice, in general I
think that if you want to use an await command, why are you making a call that
was meant to be async? You are defeating the whole point of async calls.

Yes I know a lot of standard libraries have calls that are async and you may
not really need for them to be async but I don't think that this is the case
often enough that we should abandon callbacks and the like and go back to an
age where all code must be synchronous. I know the author isn't saying it to
that extreme necessarily, but his comparing callbacks to gotos is extreme as
well.

~~~
kvb
That's not how async/await work at all. They let you write the code _as if_ it
were synchronous, but it's still asynchronous, which is the whole point.

