As someone who only recently switched to Node.js from PHP I personally haven't had any difficulty switching over to the callback frame of mind, and I haven't experienced the "callback hell" so many people complain about. At first I was hesitant to start with Node because I saw blog posts by people bemoaning the spaghetti callback code that they ended up with. But I haven't experienced any of that, although I am a relatively newbie Node programmer with only a few months of experience so far. My current project is quite non trivial as well, running into the tens of thousands of lines so far.
The key I've discovered to nicely organizing callbacks is to avoid anonymous callback functions unless absolutely necessary for a particular scope, or unless the function is going to be so trivial and short that it can be read in a single glance. By passing all longer, non trivial callback functions in by name you can break a task up into clear functional components, and then have all the asynchronous flow magic happen in one concise place where it is easy to determine the flow by looking at the async structure and the function names for each callback function.
Another major advantage to a code organization like this is that once you have your code such that each step has it's own discrete function instead of being some inception style anonymous function tucked away inside another function inside another callback it allows you to properly unit test individual functional steps to ensure that not only is your code working and bug free at the top level functions, but also that each of the individual asynchronous steps that may make up some of your more complicated logic are working properly.
Most of the bad examples of callback hell that I see have anonymous callback functions inside anonymous callback functions, often many levels deep. Of course that is going to be a nightmare to maintain and debug. Callbacks are not the problem though. Badly organized and written code is the problem. Callbacks allow you to write nightmarish code, but they also allow you to write some really beautiful and maintainable code if you use them properly.
I kinda have to disagree with you here. The problem of callback hell has nothing to do with the funcions being anonymous. In fact, you kinda want to have anonymous functions if you want to keep things as similar as possible to traditional code.
For example, when you have code like
var x = f();
print(x);
only a hardcore extremist like Uncle Bob would write it as
var x;
function start(){
x = f();
onAfterF();
}
function onAfterF(){
print(x);
}
because now your code logic is split among a bunch of functions, the variables had to be hoisted to where everyone can see them and the extra functions obscure control flow. In the first case its obvious that its a linear sequence of statements but in the second you cant be sure a-priori how many times onAfterF gets called, when it gets called and who calls it.
Coming back on topic, callback hell is not just about nesting and your current code still suffesr a bit from it. The real problem is that you cant use traditional structured control flow (for, while, try-catch, break, return, etc) and must instead use lots of explicit callbacks. Additionally, for this same reason, callback code looks very different from how you would normally write synchronous code and its a PITA if you ever have to convert a piece of code from one style to the other.
Of course its complicated for no reason. Its an example! The same logic would apply if I had 5+ nontrivial lines of code instead of just a print statement.
I'm sorry, hardly ever had an issue with callback-based programming. If you're used to imperative, maybe the problem is that you're making a mess because you're adapting from a different style and complicating it with workarounds, you need to be functional.
I dont think its a matter of functional vs imperative. In fact, functional languages give some of the best tools to avoid having to write callbacks by hand. For example, in LISPs the language tends to have explicit support for converting non callback code to CPS (call/cc and thigns like that) and in Haskell you have do-notation to get rid of the nesting and hide the callbacks behind some syntax sugar.
I'm tired to the utmost degree of all these posts about people (supposedly) coming from PHP/C#/Ruby/Python background and seeing "absolutely no problems" with JS syntax, object model and programming paradigms. There are problems. They are objectively there. If you don't see them, you have to check your critical thinking skills, rather than imply that everyone outside of elite JS circles are simply too ignorant to understand its awesomeness.
The simplest example of callback hell is trying to analyze workflow of some chunk of code in a debugger. If the code is linear, you place a breakpoint at the beginning of the method you're interested in and go through the code one line at a time. If there are nested statement of method calls, the debugger happily redirect you to them without fail.
With extensive use of callbacks, this becomes impossible. Since callbacks are merely registered in the original method, you need to place a breakpoint at the beginning of every callback function you might encounter in advance. Named callbacks actually make this worse by physically separating the place where a function is registered from its body. Did I mention that you're loosing ability to do any kinds of static reasoning, since callbacks are inherently a runtime concept? And the fact that you loose ability to look at the stack trace to "reverse engineer" why something was called?
Which reminds me of something. Have you ever seen code that reads a global variable, and you have no clue where the value came from? Callbacks create the exact same problem, except they aren't just data, they are code, so the problem can be nested multiple times.
IMO, this is more of a problem of indirections than its a problem of callbacks.
If you use anonymous function as the callback there is no indirection and you know perfctly well where to set the break point.
At the same time, you can also have the sort of debugging problem you mentioned in regular code whenever you call a method in some polymorphic object. (the "listener" pattern is just one example of this)
Well in addition to not having a problem with callback hell I also haven't used a debugger in more than three years, so I guess I'm just weird.
As I said I like to write unit tests with my named callbacks. This allows me to test the callbacks as well as the root level functions that make use of these callbacks to ensure that everything is working perfectly.
When I follow this model it is extremely rare that I ever encounter any issues that would need a debugger, and if a problem does arise somewhere the relevant unit test can quickly expose which callback is having a trouble, and precisely what is wrong.
I'm not saying callbacks are perfect. My goal is just to share my technique for organizing my code in Node which I feel has led to some very well organized, testable, and maintainable code.
When someone gives you a sufficiently large codebase written by other people and asks why when they click A they get B, you have two options:
1. Read the code and try to reason about it.
2. Fire up the debugger and replicate user actions.
Guess what? Callbacks in JS make option #1 significantly harder, since they are, essentially, runtime weakly typed mechanism for code composition.
I often write synchronous methods that include control flow that nests three deep (say try/finally, if/then/else, and a for loop). Often it's easier to read this code than it would be if everything were split out into separate named methods.
Why would the same not be true of asynchronous methods, assuming that the technology was there to enable it (as it is in C#)?
I agree. Sometimes inline asynchronous callbacks work, just like inline code blocks for if statements or for loops. You just need to train your eye to read them as if they were inline code blocks for an if/then/else block or a for loop.
But sometimes when you get many levels deep in if statements or if a synchronous function starts to reach the hundreds of lines it makes sense to break it up into multiple functions that each have a sensible semantic meaning and which fit on a screen or so. This makes the synchronous code easier to read.
The same goes for asynchronous callback functions. The callback hell that I see most often happens when people have hundreds of lines of inception style anonymous callback functions inside of anonymous callback functions. In this case, just as with the synchronous function that got excessively heavy it makes sense to break things up into multiple functions.
It's all about finding the right balance, and when you do the results are very readable and easy to understand whether you are writing synchronous or asynchronous code.
Why should that be the case? Creating all the extra methods is going to create lots of new points of indirection, the new methods are likely to be tightly coupled anyway and breaking the nesting might mean you have to hoist a bunch of variables into an outer scope.
this use case is inherently more complicated than any of the control flow structures you mention here. You are introducing a new closure, and you don't know when the function is going to be executed.
The key I've discovered to nicely organizing callbacks is to avoid anonymous callback functions unless absolutely necessary for a particular scope, or unless the function is going to be so trivial and short that it can be read in a single glance. By passing all longer, non trivial callback functions in by name you can break a task up into clear functional components, and then have all the asynchronous flow magic happen in one concise place where it is easy to determine the flow by looking at the async structure and the function names for each callback function.
Another major advantage to a code organization like this is that once you have your code such that each step has it's own discrete function instead of being some inception style anonymous function tucked away inside another function inside another callback it allows you to properly unit test individual functional steps to ensure that not only is your code working and bug free at the top level functions, but also that each of the individual asynchronous steps that may make up some of your more complicated logic are working properly.
Most of the bad examples of callback hell that I see have anonymous callback functions inside anonymous callback functions, often many levels deep. Of course that is going to be a nightmare to maintain and debug. Callbacks are not the problem though. Badly organized and written code is the problem. Callbacks allow you to write nightmarish code, but they also allow you to write some really beautiful and maintainable code if you use them properly.