

What it's like building a real website in Node.js - travisglines
http://www.travisglines.com/web-coding/what-its-like-building-a-real-website-in-node-js

======
bitdrift
Agreed. I spent the last few months building my first Node.js app (and
MongoDB) and I resonate with a lot of what's being said here. I wouldn't want
to discourage anyone from exploring Node or Mongo, because I think both are
amazing projects and have huge potential, but I do think anyone considering
either should stop and think carefully before taking the plunge.

Things that stick out in my head:

1\. What the author says about the Node community moving fast is absolutely
true. Be prepared to spend time upgrading to the latest versions often or deal
with dependency hell as you try to ensure you've got matching legacy versions
of libraries.

2\. Coding async is weird at first, but you may grow to love it (as I have).
Over the months, I've discovered many cool tricks and patterns that help keep
the nesting to a minimum.

3\. Be prepared to fill library gaps, and submit patches if necessary. This
means not being afraid to dig into the source code of the libraries you're
using. Most libraries have limited documentation so often the only way to
figure out how to use them is to get in there and see how they work.

4\. Best practices may not apply. In the case of MongoDB, I had to redesign my
schema in order to run certain queries now that MongoDB won't support until
sometime in the future.

All in all, I'm really glad for the experience and will definitely consider
both Node.js and MongoDB for future projects. What will really be interesting
is seeing what I miss about Node/Mongo after jumping back into Python/MySQL.
:-)

~~~
nailer
> 2\. Coding async is weird at first, but you may grow to love it (as I have).
> Over the months, I've discovered many cool tricks and patterns that help
> keep the nesting to a minimum.

Can you share a few? Either here or as a blog post?

~~~
mjs
If you have a function foo() that takes a callback as an argument i.e.
something like:

    
    
      foo(..., function (n) {
         // closure over q, r, s
      });
    

Then what you can do is create a separate function, bar() that takes q, r, s
as an argument, and returns a function:

    
    
      function bar(q, r, s) {
        return function(n) {
          // still has dependency on q, r, s
        }
      }
    

Once you have bar(), the foo() call above can be written as:

    
    
      foo(..., bar(q, r, s));
    

and you save one level of nesting. (This also makes the dependencies explicit,
which is helpful.)

The main loop of

[https://github.com/ithinkihaveacat/node-
fishback/blob/ce5f9c...](https://github.com/ithinkihaveacat/node-
fishback/blob/ce5f9c308db6a12f15fb0d790c53dfb2f60eaf9c/lib/fishback.js#L313)

does quite a lot of this, if you want a real example.

~~~
svetlyak40wt
Why not to write it in a much simplier way? Like this:

    
    
      function bar(n) {
        // still has dependency on q, r, s
      }
      foo(..., bar);

~~~
mjs
Yeah, that's nicer if bar() can be defined at the same "level" as foo() but if
it makes more sense to move it somewhere else the closed-over variables need
to be passed.

------
dbro
This would have been helpful to me when I was starting out building
<http://domainjig.com> \- which is a simpler site than the author's. That
said, I'd offer these responses to those looking for some advice:

1) Yes, wrapping every database call with an asynchronous callback is a drag.
I found a helpful library that made some of my workflow easier to express and
maintain: <https://github.com/caolan/async> And I'm pretty sure I could
simplify my code quite a bit more if I understood functional program patterns
better already.

2) Libraries can be challenging, but I was able to find good ones for
everything I needed. The author discussed version issues. To deal with that, I
punted on all those node library managers and just jumped into the library
code, checking it in to my repository and freezing it in time (in contrast to
the authors point #9 about NPM being awesome for him). The other issue I found
with libraries was that there are TOO MANY of them for any single thing. For
example, there are at least a half dozen separate libraries to help with
asynchronous workflow, and trying to choose which one would be "best" was
difficult, and depended on freshness, documentation, and then actually test
driving them. Github's encouragement of forking made this extra challenging,
with many slightly tweaked variations for each separate library.

I found node.js lived up to its promises (for me): coding in javascript on
both server and client was helpful in avoiding context switching inside my
head, and encouraged me to invest more effort in writing better quality code
everywhere. And node.js is plenty fast.

Another thing I'd add is that I used redis and tokyo cabinet as data stores
and found the node.js interfaces to them to be mature and easy enough. It was
convenient to use python's interactive shell to create and experiment with
redis, and then port the routines to node.js.

To summarize, consider this a positive vote for node.js.

------
peregrine
This is a great article but again, like the last article posted here about
your MVC setup, I'd personally like to see examples. For instance whats an
example of good asynchronous code?

~~~
travisglines
Hey peregrine, I'm working on a solid example here:
<https://github.com/tglines/throughexample>

I'm taking a bit of time with it to make sure its actually useful to people
and not just a stub. Feel free to check out what I have up there so far.

~~~
TY
Thank you for a good article, Travis.

Quick side note - you might want to add simple validation for sign up form for
exipe.com - it currently accepts empty string as a valid value.

------
i386
I was hoping to know more about what corner cases he hit where he had to write
subjectively ugly code and what part of the ecosystem hasn't matured yet
(libraries for certain problems missing, etc). For me, building real
applications and solving real problems in any new stack comes down to those
two things not being much of an issue.

I remember a friend and I hacking on a RoR app a few years ago where we spent
more time trying to get Ruby Gems to play nice rather than actually writing
the app. Now, those problems have been mostly shaken out and are a thing of
the distant past. Is Node past this productivity immaturity hump yet?

------
penguinboy08
Great article. Writing my node.js project has exposed me to all of the things
listed.

One thing I have to say is that NPM is good, but good be better with it's
distro integration. The ubuntu install has broken permissions by default,
which forces me to install stuff with sudo. I haven't had the time to try and
fix it.

~~~
justincormack
I dont think its worth using the Ubuntu install its just too old. At the
moment easiest to install by hand, and script something for production.

~~~
nailer
His issue (though not too obvious) is that NPM assumes /usr/local is writable
by unprivileged users, which isn't secure or common in Linux distros or OS X.

It's a bug in NPM, it should be fixed someday.

------
alexbosworth
I have been running Node.js in production for a while, it is kind of a wild-
west, Apache/PHP holds your hand for you and takes care of a bunch of things
you might not realize.

If you come from a background of already writing javascript heavy apps, the
browser is async so there shouldn't be as steep a learning curve.

Node is moving super fast, with some critical breaks between point releases.

I don't find the community crazy good, but Github is saving the day on node
for me - being able to quickly browse through forks is a lifesaver

~~~
gruseom
_If you come from a background of already writing javascript heavy apps, the
browser is async so there shouldn't be as steep a learning curve._

People say this, but it's not really true. Yes, browsers are async in the
sense that your code gets invoked by the browser as events occur. But once
your client code gets control, it nearly always proceeds in a completely
synchronous fashion. If all you're doing is changing a few dom elements in
response to a button click, that happens quickly and all is well, but woe
betide if you need to do some long-running computation in the browser... what
are you going to do? Build your own event loop with setInterval and chop
things up into short-running pieces? That's certainly uncommon. So client-side
JS programmers typically don't have much experience with organizing their own
code asynchronously, and the learning curve for doing so is still steep.

(There is of course one big exception to what I'm saying, namely communicating
with the server via XHR. But not so big that it teaches you how to write a
complex program in a non-blocking asynchronous style.)

~~~
alexbosworth
What I really mean is JS Apps that use XMLHttpRequest or JSONP -

I found that prepared me pretty well for asynchronous disk reads and
asynchronous http requests, I was already doing that in my browser code and it
ported quite cleanly to node. (I use a wrapper method that chooses whether to
use XMLHttpRequest or http.request based on the environment)

I should note one thing that tripped me up in the beginning is forgetting that
I was in a shared environment for my scripts. I typically design server stuff
to be 'shared nothing' by default, and you have to be a little careful in node
as that is not the default way of running node services and at first I was
sometimes sharing objects by accident.

------
yannickmahe
Great article.

In your opinion, is Node.js mature enough to use in production code ? Or
should I wait a little longer ?

Also, what is the sate of the community (i.e. is it easy to find answers and
tutorials) ?

------
augustl
I just wrote TLS client certificate authentication for an internal systems API
with node, in about 20 lines of code. Node's biggest power is a great
abstraction for low level HTTP and socket stuff. You get direct access to
everything, with a sane and elegant API.

For plain old websites, I'd still use something like Rails or Django.

------
Kilimanjaro
I wish we had s-node (s for synchronous) so we do:

    
    
      session.start()
      data.load()
      template.render()
    

instead of the cretan labyrinth it becomes coding for a-node

~~~
randylahey
That completely defeats the point of using Node.js; that is, to use an evented
model so your app isn't sitting around waiting for IO. s-node wouldn't be node
at all.

FWIW any other runtime/framework will do what you suggest. For example, Rhino.

~~~
joeshaw
It would be nice of Node (or rather, v8 underneath it) supported generators
and the "yield" keyword like Mozilla's Spidermonkey. Unfortunately, yield is a
Javascript 1.7 extension for now, but there's hope for it in ES Harmony.

Anyway, with generators and promises you can write asynchronous code which
looks like synchronous code but which under the covers is still completely
async and callback-driven. We use it in gjs (a GNOMEy JS runtime built on top
of Spidermonkey).

Instead of workflows like:

    
    
      function onServerRequest(req, res) {
          db.query("get filename", function(result) {
              fs.readContents(result, function (data) {
                  res.write(data);
              });
          });
      }
    

which can get pretty nasty if you're running things in serial, or want to run
a couple of things in parallel but then run something serially, you can have a
workflow more like this:

    
    
      function onServerRequest(req, res) {
          var queryPromise = db.queryPromise("get filename");
          var filename = yield queryPromise;
    
          var data = yield db.readContentsPromise(filename);
        
          res.write(data);
      }
    

The main difference between the two is that the async functions in the latter
return promises instead of nothing. The db.queryPromise() function itself
would probably look something like:

    
    
      function queryPromise(query) {
          var promise = new Promise();
          db.query(query, function(result) {
              promise.putValue(result);
          });
          return promise;
      }
    

I imagine you could probably write a function to automatically generate
promise wrappers for existing async functions as long as they followed a
certain schema, and Node's functions tend to do well with that.

Anyway, the onServerRequest() function would itself be wrapped in a function
which took the generator values from it and returned its own promise. This way
you could stack promises all the way up.

The gjs implementation of this is here:
<http://git.gnome.org/browse/gjs/tree/modules/promise.js>

This is pretty similar to Twisted's inlineCallbacks if you're familiar with
them in Python.

