I've criticized Node.js in the past on this site but I actually only recently realized what is really wrong with it in a strongly-grounded computer science way rather than an intuitive way. Or rather, wrong with the asynchronous event-based programming style in general that Node.js adopted, rather than Node.js in particular (which is a fine implementation of the bad idea).
Does anyone remember the structured programming arguments that occurred, oh, ten years before I was born? There was a lot of arguing back and forth, but one of the arguments made was that when your program was structured, your position in the program means something. Just making up some quick psuedocode:
def x(y: int, z: int):
for i in 1 to 10:
--> print i * y + z
def a(b: int):
for i in 1 to 5:
print i
x(b, 3)
At the line that I marked, the very fact the program counter is pointing there, and by extension the stack trace up to the point given, means certain things. We know we have an x and a y which aren't just "undef", we know we're in a call from a (in this case), and therefore, any preconditions or invariants that are provided by those functions are also in effect at this point in time. One of several reasons spaghetti code is bad is that the program counter means much less; we don't know how we got there, we don't know what invariants are in effect, in the old school assembler case we know almost nothing at all, for all we know we just lept here with a goto.
As is almost always the case, this is a small trivial example, but when you start layering many things on top of each other, layering in functions that provide safe file handling or other resource management or state machines or any of the other tools we've built on structured programming or its smarter child Object Orientation, it becomes difficult to in some cases impossible to follow all this in your head. Or deal with the work to make this stuff compose together properly without layers actively stomping on each other. After spending some time with the modern functional world and their increasing focus on composability, working with async event programming feels like stepping back in time 20 years.
Asynchronous event-based code is not as bad as old-school assembler, because the event handlers themselves are still using the ideas of structured programming internally. But the code as a whole is spaghetti code. I'm not the first to say that, but it turns out upon reflection it's not a slur or a metaphor, it is actually descriptive and fair. Async event code actually does share many of the critical properties of spaghetti code, as the term was first used. You don't have a call stack, you don't have the invariants, you're just adrift in the code.
Of course with massive amounts of discipline you can function anyhow. You can program structured code in assembler with massive amounts of discipline. But A: you are spending valuable developer mindpower maintaining that discipline which is better done by a language/runtime/VM, no matter how smart you are you're still better off spending your smart on something other than raw plumbing and B: it's actually harder than you think. We've so thoroughly, utterly internalized structured programming since even before I was born that we can't hardly even see what we're getting out of it, and consequently we don't easily realize what we're giving up when we adopt this style. We aren't any better people than our assembler ancestors, we're not particularly more disciplined than them, and they switched to structured programming for a reason.
I'm this critical because I'm actually trapped in this style at work, fortunately just in one of several subprograms but it's still annoying as hell and the one that always takes far longer to work with than I'd like. It's an excess of experience, not a deficit, causing me to be this critical. And I've really come to loath this style. Compiling is for computers, not humans.
That's what Erlang and similar languages that can take care of the asynchronousness at the VM/language level bring to you; they bring you at least back up to structured programming in power and safety, and possibly beyond. And the arguments about how wonderful async event based programming is and how you've got it all under control and how it's performant and not a problem sounds to me like an absolute repeat of assembler programmers ranting against structured programming back in the day, to an almost scary degree... and every bit as correct and likely to win the future.
Very good points. I wrote the article and have struggled a lot with how right node is as a general purpose tool. My interim conclusion, for many of the reasons you point out, is that to the extent that it can be made relatively safe, it is a good tool for those situations where my other choice would have been to write a massively scalable, non-blocking C program. This class of problems does not come up every day, but it is coming up more and more as the web and mobile arena move to more of a fully connected mesh of clients and the switchboards that arbitrate their exchanges need to become more flexible and efficient.
On the flip side, building massively scalable systems based on blocking IO is rife with problems and I've found that enforcing architectures that make it manageable have a way of obscuring the code and making it difficult for people to intuitively get right as well. I personally find the explicit functional style to be just as intuitive as structured and/or OO and therefore find that when I need it, a platform like node that makes the knife's edge that is asynchronous programming explicit and manageable with a good functional style is a good trade-off in the world where I am trying to find less bad options to a hard problem.
Sometimes you have to step back 20 years in order to break through the layers of assumptions that were added in the interim. I doubt node is the last word on the topic, but it is a refreshing interlude to what was becoming an unwieldy calcification of the theory of how to program computers. The water will find the right course eventually as we experiment with the different styles.
"it is a good tool for those situations where my other choice would have been to write a massively scalable, non-blocking C program."
Agreed. I'd take a Node.js implementation over straight C any day.
"On the flip side, building massively scalable systems based on blocking IO"
Please remember that's an implicit false dichotomy. "Blocking" has to do with the runtime and the VM, not the visual appearance of the programming language syntax. Erlang is non-blocking, but it is also structured (and functional). In fact, by my personal standards Node.js is still a blocking language; you have to jump through hoops to get non-blocking behavior and you only get it when you ask for it. In Erlang or Haskell, you simply get it. Go ahead and do a lengthy math computation if you'd like. Take several minutes. You won't block anything else in the meantime. And you don't have to manually break the computation up yourself. Just do it.
I say Node is stepping back 20 years not because it feels primitive compared to C#, but because it feels primitive compared to the "really existing, I use it in production code" Erlang runtime. Also Haskell, except I can't say I use that in production code. And probably Go (still waiting for someone to confirm), and Stackless Python, and several other things.
The real reason I speak up so often on Node.js articles is not that I hate Node.js, it is that I hate the hype because it is shot through with falsehoods, which I'm rapidly upgrading to "lies" as the same blatant falsehoods continue to get around without the community fixing them. Your alternative is not async event based or synchronous; there's a third option where the async is managed by the compiler and runtime and it works. In fact it's Node.js that has to catch up to even the previously existing async frameworks, and Erlang is simply miles beyond Node.js in every way, except it isn't Javascript. (Which I freely admit is a problem. Erlang-the-language could stand to be improved, even as it is hard to borderline impossible to match the total environment.)
(If you know the choice exists and you don't choose it for some reason, hey, great. Like I said in my first message above, I've got my own async-event based program I have to maintain, and I'm the original author, it was my choice, because when I took all the issues into account, it was the right choice. But you ought to choose with an understanding of the full picture and all the relevant choices, and understand all the tradeoffs, not because you think that async event-based programming is better than everything else at everything. It's got some really serious drawbacks, and there are really-existing, production-quality, "web-scale" things that can be used that do not have those drawbacks.)
I can certainly see your point, and I am aware that I am playing a little fast and loose with some of the terminology around blocking vs non-blocking. To be precise, in node's case, it presents a consistent, callback based API to non-blocking IO. Given that that API is reflected exclusively in JavaScript the language, this has impacts the style and appearance of Node programs. The resultant style is imprecisely referred to as asynchronous, non-blocking, etc.
Using Erlang vs Node on my current project was a serious consideration. It is well thought out and its message passing and lightweight process based design is more evolved than anything node has or likely will have. It's also Erlang and I have found productively programming in it to be incomprehensible. Maybe its a personal problem, but it is what it is. Ditto for Haskell. I was surprised to find that my brain didn't bend that way.
I don't get it. Erlang is a language, Node is not. By necessity then, isn't your Node code as spaghetti or as structured as you write it? The great (and bad) thing about JavaScript is that you can write in any style, object-oriented, functional, "imperative"/spaghetti or Python-style/intendation-driven with CoffeeScript. You can mess it up easily, you can stay disciplined.
I think Node is coming from / targeting a situation where a web developer may write a lot of client script with jQuery. That's callbacks all over the place for your flow of code. But if it gets too much you need to get organized and rearrange things. So it brings the same paradigm to the server that on the client side is a necessity devs have to deal with. Event-driven callbacks, timers, web workers -- you already have that mess in your code base and will have found / to find a way to stay organized there, so whatever you do towards that end on the client side, you can do on the server side too.
"Of course with massive amounts of discipline you can function anyhow."
In any meaningful project, you will need to have that discipline, anyhow.
Does anyone remember the structured programming arguments that occurred, oh, ten years before I was born? There was a lot of arguing back and forth, but one of the arguments made was that when your program was structured, your position in the program means something. Just making up some quick psuedocode:
At the line that I marked, the very fact the program counter is pointing there, and by extension the stack trace up to the point given, means certain things. We know we have an x and a y which aren't just "undef", we know we're in a call from a (in this case), and therefore, any preconditions or invariants that are provided by those functions are also in effect at this point in time. One of several reasons spaghetti code is bad is that the program counter means much less; we don't know how we got there, we don't know what invariants are in effect, in the old school assembler case we know almost nothing at all, for all we know we just lept here with a goto.As is almost always the case, this is a small trivial example, but when you start layering many things on top of each other, layering in functions that provide safe file handling or other resource management or state machines or any of the other tools we've built on structured programming or its smarter child Object Orientation, it becomes difficult to in some cases impossible to follow all this in your head. Or deal with the work to make this stuff compose together properly without layers actively stomping on each other. After spending some time with the modern functional world and their increasing focus on composability, working with async event programming feels like stepping back in time 20 years.
Asynchronous event-based code is not as bad as old-school assembler, because the event handlers themselves are still using the ideas of structured programming internally. But the code as a whole is spaghetti code. I'm not the first to say that, but it turns out upon reflection it's not a slur or a metaphor, it is actually descriptive and fair. Async event code actually does share many of the critical properties of spaghetti code, as the term was first used. You don't have a call stack, you don't have the invariants, you're just adrift in the code.
Of course with massive amounts of discipline you can function anyhow. You can program structured code in assembler with massive amounts of discipline. But A: you are spending valuable developer mindpower maintaining that discipline which is better done by a language/runtime/VM, no matter how smart you are you're still better off spending your smart on something other than raw plumbing and B: it's actually harder than you think. We've so thoroughly, utterly internalized structured programming since even before I was born that we can't hardly even see what we're getting out of it, and consequently we don't easily realize what we're giving up when we adopt this style. We aren't any better people than our assembler ancestors, we're not particularly more disciplined than them, and they switched to structured programming for a reason.
I'm this critical because I'm actually trapped in this style at work, fortunately just in one of several subprograms but it's still annoying as hell and the one that always takes far longer to work with than I'd like. It's an excess of experience, not a deficit, causing me to be this critical. And I've really come to loath this style. Compiling is for computers, not humans.
That's what Erlang and similar languages that can take care of the asynchronousness at the VM/language level bring to you; they bring you at least back up to structured programming in power and safety, and possibly beyond. And the arguments about how wonderful async event based programming is and how you've got it all under control and how it's performant and not a problem sounds to me like an absolute repeat of assembler programmers ranting against structured programming back in the day, to an almost scary degree... and every bit as correct and likely to win the future.