Hacker News new | comments | show | ask | jobs | submit login

The problem is that the author simply didn't notice the refactoring and abstraction opportunities available. (One question to ask yourself: "How do I test this?" If you can't answer that question, the code is wrong.)

We'll start with the synchronous example:

    myThing = synchronousCache.get("id:3244");
    if (myThing == null) {
      myThing = synchronousDB.query("SELECT * from something WHERE id = 3244");
This is verbose and tedious. We should really make the API look like:

    myThing = database.lookup({'id':3244}, {'cache':cache_object});
Let's apply this idea to his asynchronous example. We want the code to look like:

    database.lookup({'id':3244}, {'cache':cache_object}, function(myThing) {
        // whatever
So instead of writing this:

    asynchronousCache.get("id:3244", function(err, myThing) {
      if (myThing == null) {
        asynchronousDB.query("SELECT * from something WHERE id = 3244", function(err, myThing) {
          // We now have a thing from DB, do something with result
          // ...
      } else {
        // We have a thing from cache, do something with result
        // ...
We need to refactor this. Remember, node.js is a continuation-passing-style language. So let's set a convention and say that every function takes two continuations (success and error).

Then, to compose two functions of one argument:

   function f(x, result, error)
   function g(x, result, error)

   h = f o g
You write:

   function compose(f, g){
       return function(x, result, error){
           g(x, function(x_){ f(x_, result, error) }, error);
(Data flows right-to-left over composition, so "do x, then do y" is written: "do y" o "do x".)

Now we can cleanly write a complex program from simple parts. We'll start by creating a result type:

    result = { 'id': null, 'value': null, 'not_found': null }
Then, we'll implement cache functions that take keys (as results of this type) and return values (as results of this type). Looking up an entry in cache looks like:

    cache.lookup = function(key, result, error){
        new_key = key.copy();
        cache.raw_cache.lookup(key.id, function(value){
            new_key.result = value;
            new_key.not_found = false;
        function(error_type, error_msg){
            if(error_type == ENOENT){
                new_key.not_found = true;
            else {
                error(error_type, error_msg);
Looking up an entry in the database looks about the same. The key feature is that the "return value" and the "input" are of the same type. That makes composing, in the case of "try various abstract storage layer lookups in a fixed order", very easy. (Yes, the example is contrived.)

    dbapi.lookup = function(key, result, error){ ... };
Now we can very easily implement the logic, "look up a value in the cache, if it's not there, look it up in the database":

    cached_lookup = compose(dbapi.lookup, cache.lookup);
    cached_lookup(1234, do_next_step, handle_error);
You can, of course, generalize compose to something like:

    my_program = do([cache.lookup, dbapi.lookup, print_result]);
Writing clean and maintainable code in node.js is the same as writing it in any other language. You need to design your program correctly, and rewrite the parts that aren't designed correctly when you realize that your code is becoming messy.

Continuation-passing style is pretty weird, but you do get some benefits over the alternatives. Writing a program with coroutines involves deferring to the scheduler coroutine every so often, littering your code with meaningless lines like "yield();". Using "real" threads is even worse; your code looks like single-threaded code, but different parts of your program are running concurrently. (Did you share any non-thread-safe data structures, like Java's date formatter? Hope not, because you won't know you did until the production code dies at 3am.) Continuation-passing style lets you "pretend" that you are executing multiple threads concurrently, but the structure of the code ensures that only one codepath is running at a time. This means that libraries that don't do IO don't have to be thread safe, since only one "thread" runs at a time.

All concurrency models involve trade-offs over other concurrency models. But when comparing them, make sure you're comparing the actual trade-offs, not your programming ability with each model.

I think you've actually demonstrated how something that would be really simple in python, gets horribly complicated in javascript. Or maybe it's just me...

If you use a continuation passing style in Python, then the code looks about the same. Most Python programmers use threads (and let the GIL give them a bit more thread safety than C++ and Java programmers get) or Twisted (with Deferreds).

I think you'll write better JavaScript if you know Python because Python encourages you to use named functions instead of lambdas. JavaScript fanbois get very excited about anonymous functions and overuse them; Python doesn't let you use anonymous functions for anything useful, so you tend to name things. (object.method is also nice syntax for working with callbacks.)

Anyway, Python and Node feel about the same to me, except for the fact that Python has nicer syntax.

No, it's definitely not just you. This seems way too difficult for trying to accomplish something so simple.

Great takeaway, jrockway. For his other example, I encourage people to take a look at caolan's excellent async library:

    function(callback) {
      async.map(ids, db.getById, callback)
    function(posts, callback) {
      callback(null, posts.map(templating.render))
  ], function(err, results) {
EDIT: Fixed example.

Personally I've gotten more use out of the Futures library, though both are great. https://github.com/coolaj86/futures It contains a form of 'waterfall' known as merely a sequence. I use a lot of sequences and promises, it makes me not go insane from NodeJS programming. In the end I still think it was a mistake for Node to not support any sync features because there are times it's nice to have. TameJS doesn't solve the problem completely either (and I hate the compile step).

Impressive example. However, as hinted at in other comments related to composing, this style of composition is really quite hard to hold in your head all at once.

It's certainly not a path I would relish having to follow, and I would not expect everyone to be able to program at this level.

I'd certainly not go so far as to criticize their programming ability for not being able to intuitively do this.

Dude... are you converting from Perl/Catalyst? :)

Nope! But I play with programming languages for fun.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact