"While we all know and love JavaScript as a language ..."
I know it but I don't love it. It's basically the only game in town when it comes to client side scripting.
Also, the concurrency model of Node.js does not eliminate concurrency related bugs. It's entirely possible to have race conditions, for example, with single threaded code that uses asynchronous callbacks. I guess a lot of people don't realize that. A real solution is no shared state, like Erlang or Clojure.
My gut feeling is that the Node.js library model (no call is blocking) is eventually going to lead to a mess. As I understand it, any API that can potentially block has to take a callback function. What happens when you realize some API that used to not block now may block (due to some increased functionally, platform changes, etc)? Any code that uses that API, and worse, any non blocking API that uses that API, now has to grow a callback too.
It can be done. The Twisted library has done a similar thing for years already. I just don't think it's a very good model for a general purpose server side language.
All that said, Node.js will probably be successful. The fact that browsers are not entirely broken WRT DOM and Javascript means that there is going to be ever increasing amounts of Javascript code around. It makes sense to re-use that development effort on the server side.
I think your assumption about no call blocking libs leading to a mess is under the assumption that people see a need for a blocking lib with Node. The assumption by default is that every call is async. Even calls which happen instantly sometimes offer a callback pattern because that's what people who use node often expect.
I can't personally envision a situation where an NPM module would update to change a call to blocking spontaneously. This is the reason why we're using Node in the first place.
Non-blocking is the default. The community assumes and expects non-blocking calls by default.
I find it very easy to imagine myself writing some code that assumes a function doesn't need to do I/O, and then lo and behold the spec changes and we have to refactor the entire call stack of anything that assumes it's nonblocking.
I haven't had much difficulty avoiding such cases. It can come up if you have a simple fixed function that returns its value, but later decide to make it customizable through the database. When that happens, it's not very hard to make the callers use a callback.
Some changes are breaking changes. If a call requires new parameters and there are no sensible defaults then clients have to change to supply those parameters.
I am sure this will be misunderstood but anyway... The initial Java world made IO exceptions checked exceptions, this implied that the addition of IO was a breaking change in that model also.
Java shops could use javascript via jdk6 for 7 years now, however no popular frameworks or libraries for the backend emerged. And it is a javascript which is embedded and basically could be promoted as a 'simpler and faster (development-wise) java' but never happened.
> Servers built with Node.js are typically super fast and can handle thousands, tens of thousands, even hundreds of thousands of concurrent connections — something very difficult to achieve with threaded servers. How can that be?
I've read similar sentiments in a bunch of places but whenever I push any real traffic at the nodejs version of our API server it falls over pretty quickly (even with ulimit increases). What exactly are people doing to handle tens or hundreds of thousands of connections?
Incidentally, if any nodejs gurus are interested we're looking for a remote freelancer to audit our nodejs stuff and help us with a deployment strategy (assuming we can get to the 100s of thousands of connections mark).
There are a bunch of factors in scaling up a Node service. Like any other technology memory per user is a factor, in Node this tends to be connection state, and some application state. Node currently has a ~2gb JavaScript limit per process for JavaScript object (not Buffers).
Since Node bounces much work to the kernel then kernel and TCP factors can play a part. So ulimit, and ephemeral port tuning is important.
Finally at some point CPU load for doing keep-alive on open lots of connections has a cost. So that is also a limiting factor.
In general it depends what you are doing. Using Joyent's SmartOS (a Solaris varient) I was able to hold just under a million TCP connections on a single Node process. However the ability to service those Node is extremely limited.
When I worked at Joyent I had access to root level params you can't configure on individual SmartMachines by hand. Joyent ops might be able to do it for you, but some of those changes are global per server (affecting all the SmartMachines it hosts).
Could you please tell a few words about your application, if it is a million connections, it is a million * N database queries on the backend (for a typical web app)?
Would every request still stay in a queue, not on the front-end layer, but on database/storage/messaging/other middleware layer?
I am just wondering what are the benefits of moving a bottleneck one layer up?
Not an application, just a benchmark. There was some keep-alive, and a small amount of state being passed from memory, but nothing heavy. At that level of contention you'll easily saturate a core making the application useless.
My point is more that Node can support a very large amount of connections. Once you scale down to the tens or possibly low hundreds of thousands you can still do real work.
This is mostly useful when you have a server that is looking after a lot of things that don't communicate very often. Sensors is one application for example. While totally passive sensors can poll a server with their data, being able to hold open many very cheap connections allows you ask sensors for their data when the server wants it rather than on an interval. This can cut down the amount of unnecessary data sent a lot.
The threads vs. event loop thing doesn't always pan out to either one being 'better' - who do you think is doing the work when your browser, or Node.js does an async call? They use threads too, and there are certain workloads where it's more efficient to just use multithreading instead of event loop + thread pools. I can't find my references atm b/c I'm traveling, but if you google around you will find numerous case studies in the Java world about threads vs. event loops (the Java world has been wrestling with threads vs. event loops too, for the past DECADE)
Remember as an engineer: spike, profile, benchmark, and determine the best solution for the problem.
I know it but I don't love it. It's basically the only game in town when it comes to client side scripting.
Also, the concurrency model of Node.js does not eliminate concurrency related bugs. It's entirely possible to have race conditions, for example, with single threaded code that uses asynchronous callbacks. I guess a lot of people don't realize that. A real solution is no shared state, like Erlang or Clojure.
My gut feeling is that the Node.js library model (no call is blocking) is eventually going to lead to a mess. As I understand it, any API that can potentially block has to take a callback function. What happens when you realize some API that used to not block now may block (due to some increased functionally, platform changes, etc)? Any code that uses that API, and worse, any non blocking API that uses that API, now has to grow a callback too.
It can be done. The Twisted library has done a similar thing for years already. I just don't think it's a very good model for a general purpose server side language.
All that said, Node.js will probably be successful. The fact that browsers are not entirely broken WRT DOM and Javascript means that there is going to be ever increasing amounts of Javascript code around. It makes sense to re-use that development effort on the server side.