

Node.js is genuinely exciting - simonw
http://simonwillison.net/2009/Nov/23/node/

======
Maciek416
I just jumped on node a few days ago.

People who are already comfortable with programming with events and closures
in Javascript are going to try Node and say things like "where has this been
all my life?". In those few short days Node has highlighted to me just how
dramatic of an impact a tool that is close to your language/model of choice
can have on your ability to rapidly turn ideas into working code. For a cross
section of coders that work with jQuery all day, Node will be a big deal.

Node has some really friendly and highly productive people working behind it,
chief among them Ryan Dahl ( <http://github.com/ry> ). To learn Node's ins and
outs and have some fun, I spent this weekend building out a simple node-based
MapReduce-like work unit processor and needed a lot of random help along the
way. When I needed gzip ( <http://github.com/waveto/node-compress> ), or to
communicate with Unix command processes, or a redis client (
<http://github.com/fictorial/redis-node-client/> ), the Node community was
there to help me very quickly.

Simply said, Node.js is a very productive and fun environment for weekend
hacks, and I expect it to become a primary fixture in my toolbox in the coming
months.

[edit: added links]

------
xal
It's really awkward to write evened code without a good implementation of
futures ( or go's channels ). The problem is that you end up nesting a lot.
Look at the example for the posix rename here:
<http://nodejs.org/api.html#_posix_module>

Now imagine implementing a normal http controller. You have to do say 5 MySQL
call (or redis, whatever) to get all the data you want. This means that you
have to nest together 5 closures. Now if any of the methods fails and there is
no global error handler that disconnects the client then that means that the
connection will hang forever until the browser user hits the stop button.
Spotting these situations becomes really hard with complex code.

Futures ( Go channels ) solve this because they block the thread when someone
accesses the data. If you want to do 3 db calls you can just write familiar
code:

var a = db.select(...); var b = db.select(...); var c = db.select(...);

print c #=> blocks until c has returned

~~~
tolmasky
This is precisely my concern, and I have talked to a lot of people regarding
this exact point. I think node.js is really cool but I would love if this was
just built into the language, such that code would perform asynchronously but
not move to the next line.

var a = something_async(), b = something_else_async();

Upon calling something_async, the code starts waiting and blocks further
execution until it is ready, going to the next statement. This is not a
foreign concept at all, and is used in javascript as well with alert, prompt,
confirm:

fries = window.confirm("Do you want fries with that?")

confirm is an async method (has to wait for the user), but the next line is
not executed until it returns. I personally see _no_ advantage to creating a
system of callbacks:

do_x(function () { do_y(function() { do_z() }) })

when this would be much more readable:

do_x(); do_y(); do_z();

and would have the exact same perf characteristics. In fact, you could
probably simpy code transform the latter into the former (I know this is what
stratified JS does), although this is probably the worst way to do it. What
you really want is to just auto-call "wait" on every promise (I believe wait
still gives you the asynchronous benefits).

Of course, the main "drawback" to this is that you need language support to
pull it off, so you need to implement a preprocessor of some sort.

~~~
_sh
I don't understand your concern. When you say 'Upon calling something_async,
the code starts waiting and blocks further execution until it is ready',
you're describing a _synchronous_ call. Indeed window.confirm is a synchronous
function--nothing is executed until it returns.

And yes, both asynchronous and synchronous operations have their place, _umm_
, except perhaps in Haskell where things get funky. Chaining async operations
together in javascript can be tedious, but its easy enough to write a helper
function to handle it for you. Every javascript app I've written that uses
AJAX always has some form of createCallback(context, function, arguments)
function.

~~~
xal
Re-read my example. All 3 db queries are run in parallel however the code does
not continue if you access one of the values until the value is available. In
a typical web application you usually get 3-5 things from the db, then you
start rendering the templates. Each DB query waits for completion, then it
starts the next which is a huge waste of time. In fact the system can easily
start the parser and template renderer while the DB queries run and all the
data will likely be available once it's accessed. This means you have _all_
the benefits of synchronous processing with the majority of the advantages of
async. The vast majority of web applications would likely very rarely block a
thread in such an architecture.

I'm very surprised that not more languages implement this pattern. Maybe i'm
missing something but this is the major feature of GO, except that almost
everyone missed it and focused on the fact that it compiles fast and looks
like c

~~~
Hexstream
Seems like you the dataflow paradigm is exactly what you'd need. With
dataflow, trying to read a variable that doesn't know its value yet blocks
until the value becomes known.

------
onewland
I'm excited about it. I've been attempting to blog, and using node.js as my
subject matter. (Don't want to link-pimp as I need to enhance/edit, but you
can check my profile if you're interested).

It's my opinion that JS is the new lingua franca of programming, and it's
about time we replaced C/Java. It has consistent syntax, some great functional
aspects, dynamic typing, and it's now become a top target for optimization by
some of the world's smartest engineers.

Breaking into systems coding is the next step, IMO, in JS becoming a
legitimate option for "anything" programming. I think node.js is going to be
the library for it. It's really idiomatic and concise, and it's got just the
right amount of opinionatedness I think to become the systems and JS version
of Rails.

~~~
anonjon
What exactly is the driving force behind the desire to do everything in
Javascript? Is it just because we have a surplus of Javascript programmers? It
doesn't seem to me that this task is particularly well suited to Javascript.
(I think) Something like Erlang would be more applicable.

And to me it seems that learning a new programming language (even one using a
relatively obscure syntax), is a much simpler task than designing an entire
project (or piece of a project), around a language that doesn’t really fit the
needs of a project.

To that extent, I think that the idea of a ‘lingua franca’ language is an
inherently broken metaphor. Programming languages are closer to human-computer
interaction tools than they are natural languages. ‘Lingua franca’ implies
that every language can communicate approximately the same thing (it is just a
matter of who has influence). I think this is a mistake, as different
languages allow you to think in hugely different ways about programming.
Different languages have different intrinsic concepts.

It may be that this argument boils down to ‘if all you have is a hammer,
everything looks like a nail,’ but even that is an oversimplification, as I
don’t even use the same hammer for all types of nails (tack hammer, framing
hammer, etc).

I am not saying that node.js is necessarily a bad thing to have. It may turn
out that Javascript is indeed particularly well suited to this type of
application. Superficially, however, nothing convinces me that this is some
sort of exciting home-run-win situation (or that it is any better than any
other functional-style programming language).

Could you explain the excitement and appeal to me?

~~~
simonw
"Could you explain the excitement and appeal to me?"

There's not much I can say that I didn't already say in my blog entry.
JavaScript is designed for event-driven programming in the browser, and my
brief experience with node.js makes me believe that it also works well for
event-driven programming on the server. And V8 is really fast.

As for Erlang... when you're programming for the web, it's useful to have a
programming language that treats strings as more than just an array of
integers.

~~~
anonjon
With regard to Erlang, are you sure? Joe Armstrong seems to think its fine for
string manipulation...

[http://www.erlang.org/cgi-bin/ezmlm-
cgi?4:mss:47293:200911:d...](http://www.erlang.org/cgi-bin/ezmlm-
cgi?4:mss:47293:200911:dmkmepbchkebogigcnaf)

(You can either think that he is biased or that he would be an expert on it).

I mean, it seems to me that javascripts main advantage in programming for the
web would be DOM manipulation. And that is good, but a lot of languages can do
it. (Honestly I've never really done (much) direct string manipulation for web
programming).

It doesn't seem to me that javascript really has a monopoly on event driven
programming either. It is relatively easy to do that sort of thing in almost
any other functional style language...

------
davidw
> Twisted and EventMachine are hampered by the existence of a large number of
> blocking libraries for their respective languages. Part of the difficulty in
> learning those technologies is understanding which Python or Ruby libraries
> you can use and which ones you have to avoid.

This is what's interesting, I think. The reason Erlang does so well at this is
that stuff written in Erlang basically just doesn't block because a scheduler
sits under the whole thing.

> Node creator Ryan Dahl has a stated aim for Node to never provide a blocking
> API—even filesystem access and DNS lookups are catered for with non-blocking
> callback based APIs.

So that's the tricky bit, I suppose, especially as your language/environment
grow, and people start doing random stuff with it.

BTW, it's also worth mentioning that Tcl has had a nice event system since
well before Twisted came out, built into the language. However, like Python,
you can't count on random commands not to block, so you have to be careful.

~~~
tdavis
I don't find that particularly true with Twisted because for every common
blocking library there is a Twisted equivalent. Whether you need http, ssh,
db, or a multitude of other protocols and interfaces, Twisted has them. Sure,
you have to be a _little_ careful, but chances are if you're using an event-
based networking engine you have a good reason for doing so and a pretty
specific purpose.

------
Slashed
I've compiled it and had a brief look at this. I have got only this to say, it
is a very important evolutionary step in server-side coding and coding in
general. Really impressed! Though I have one more question, how do you handle
concurrency? I mean, mutex, locks, etc. Don't get me wrong, I'm not being
picky, just wondering what could be the best solution for this.

Edit: Or Events in EventEmitter are being executed one at a time, so there is
no need for locks?

~~~
simonw
Node is single threaded, so you shouldn't need to implement any locks
yourself.

------
allertonm
(Comment repeated from proggit, where this was also posted.)

Don't get me wrong here, I love coding in Javascript and have often
entertained the idea that server-side Javascript would be fun. I also totally
buy the need to get away from thread-per-request for many web serving
scenarios, Comet being a perfect example.

However when I see someone going so far with this as to say "all APIs must be
non-blocking" I have to wonder if they would be better served by using a
language where threads were less expensive. I know closures make this kind of
thing a lot less of a pain than back in the day when we were programming
against OSs with no threads and async APIs were often a necessity, but to
mandate this style is basically asking developers to manually rewrite all of
their logic into continuation passing style. It's a bit, well... naive is the
kindest word that comes to mind.

BTW, I don't think it's a coincidence that all of these event-driven web
frameworks are for interpreted languages with crappy thread support.

------
ra88it
The 'Hello World' example (at the top of the front page of the Node.js
website) is a server that listens to http requests, waits 2 seconds, then
responds 'Hello World'.

The server is supposed to continue responding to incoming requests even while
it is waiting 2 seconds for previous requests. So I figured that if I quickly
queued up a bunch of requests (say, over the course of one second), they would
all respond just as quickly after the two seconds was up.

However, each response came a full 2 seconds after the previous response. I
increased the wait argument to 10 seconds just to be sure.

This seems functionally equivalent to blocking. Is this the way the example is
supposed to work?

~~~
simonw
Was your client sending the requests concurrently? If I do that and run "ab -n
100 -c 100" (for 100 requests run with concurrency 100) the benchmark takes
almost exactly 2 seconds to execute all 100 requests.

~~~
ra88it
I guess not - it worked when I followed your suggestion. Thanks!

I was naively cranking out a bunch of tabs in Firefox really quickly. :) I
guess Firefox doesn't make concurrent requests when it's the same URL...?

~~~
pilif
Firefox only makes two (by now maybe four) concurrent requests to the same
server as mandated by the HTTP protocol.

~~~
pmjordan
It's technically not mandated, but recommended. Servers need to be able to
deal with more anyway, as proxy servers are recommended to open up to 2 * [no
of clients] connections to upstream servers.

------
richcollins
It is unfortunate that none of the JS VMs have support for concurrency. Event
driven programming can get messy (lots of state machines).

~~~
ZitchDog
It's usually pretty trivial to create an event bus where you can publish and
subscribe to events.

~~~
richcollins
How is that equivalent to new threads (stacks) of execution?

~~~
ZitchDog
It's nothing to do with concurrency - the event bus pattern will alleviate
much of the pain around event-driven programming (the FSM's you were referring
to) and decouple the event handling machines. In my experience, moving to an
event bus model makes event driven programming very manageable.

If the event bus has executors (which it could in v8), it will be parallel to
boot.

------
intellectronica
"Exciting" is an understatement. Cloned and compiled it yesterday and since
then I can't bring myself to stop playing with it.

~~~
megamark16
I also recently cloned and compiled it myself, but I haven't had much time to
play with it yet.

I have been doing a lot of Javascript/jQuery UI stuff at work lately and have
become VERY comfortable with the anonymous callbacks and such. Now I'm ready
to jump on Node and see what I can do. This looks awesome!

------
Slashed
Sorry, can somebody explain me how this works? Obviously, I'm missing
something. Is the code being interpreted on the server-side? Then my question
is, how fast it is and if it's capable to serve large number of connections?
Actually, it's very exciting if it runs no slower than Java.

~~~
simonw
Yes, it's being executed on the server. It's very fast - 3,500 requests a
second for returning a "hello world" string with 100 simultaneous connections,
and because its all event based it should be able of handling thousands of
concurrent connections without much trouble (I haven't tried yet because my
laptop runs out of file descriptors).

~~~
fizx
You know you can just bump allowed file descriptors up with ulimit, right?

------
AndrewO
Glad I read this. I'd assumed Node.js had something to do with the Nodebox
visualization environment on Javascript (I think a project like that has been
featured here before). While that would be interesting and pretty, it wouldn't
be my cup of tea right now.

This, however, looks really cool. I've been waiting for a compelling JS server
side framework for awhile—definitely going to have to check it out. I hope
other people aren't making the same mistake.

------
huyegn
For those of you who have taken a look at this, is there any library or
construct in python that would be equivalent to the Events in Node.js?
<http://nodejs.org/api.html#_events>

That's the deal maker for me... To have custom defined events. Correct me if
I'm wrong but even Twisted doesn't have this facility?

~~~
simonw
I'm pretty sure Twisted's deferreds serve the same purpose.

------
tlrobinson
It's great to see node.js starting to support CommonJS
(<http://commonjs.org/>) modules.

------
simplify
Very exciting! What sort of persistent storage options are there for something
like this?

~~~
martian
There are several options already available, including file i/o and Postgres
db connections. cf. <http://wiki.github.com/ry/node>

------
AlexTheFounder
What's the main one advantage of Node.js over php (or ruby)?

~~~
simonw
As I said in the article, it's event based which means it can handle massive
concurrency and serve pages that don't necessarily reply straight away -
perfect for implementing things like comet. It gives you similar capabilities
to Ruby's EventMachine. I don't think PHP has an equivalent at the moment.

------
c00p3r
Does v8 know supports cross-platform aio, epoll, kqueue or at least
sendfile()?

~~~
jacobolus
Node uses libev: <http://software.schmorp.de/pkg/libev.html>

