
Re-using Backbone.js Models on the server with Node.js and Socket.io - ericflo
http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-server-with-node/
======
chapel
I wanted to cross post this here as I already posted it in the comments.

\-----

I am glad you wrote about this, but I feel you are missing the real 'magic'
that can be done with node.js and Backbone, or any client code you choose.
Socket.io is great, fantastic even, and I think it is something that really
shows the power of node.js; there is something built on top of socket.io that
is even more wonderful, and it is called dnode[0]. It is really simple to use,
but has a lot of potential for server process to server process and server to
client communication. It is in effect RPC, and I really can't do it justice.

For the example you give up here, you could easily overwrite the Backbone.sync
function and have it instead send the appropriate objects you need to fetch
models on the server, while having the server call back to the client with
said models. Not only that but you could easily remotely invoke the model
changed functions in Backbone directly from the server. There is so much more
you can do, but I lack the knowledge at this point to put it in play just yet.

[0]: <https://github.com/substack/dnode> &
[http://substack.net/posts/85e1bd/DNode-Asynchronous-
Remote-M...](http://substack.net/posts/85e1bd/DNode-Asynchronous-Remote-
Method-Invocation-for-Node-js-and-the-Browser)

~~~
tlrobinson
How's this different than JSON-RPC, for example?

~~~
substack
You can pass functions around in your JSON.

~~~
tlrobinson
That's pretty cool. Like an asynchronous functional version of Cocoa's
Distributed Objects.

Are there ever problems with one side referencing objects that no longer exist
on the other side?

I've run into these kinds of problems due to the lack of weak references in
JavaScript. Hopefully a future version of JavaScript [1][2] will fix that.

[1]
[http://wiki.ecmascript.org/doku.php?id=strawman:weak_referen...](http://wiki.ecmascript.org/doku.php?id=strawman:weak_references)

[2] <http://wiki.ecmascript.org/doku.php?id=harmony:weak_maps>

~~~
substack
Referencing objects isn't a problem since dnode doesn't do replication, it
just makes copies, but referencing functions when the client disconnects can
occasionally be bothersome. You do get a hook when the client disconnects:

    
    
        conn.on('end', fn)
    

so that helps. At browserling.com all of our backend and frontend spanning
lots of processes over several servers is tied together with dnode, and we
haven't had any big problems tracking references that I can think of.

------
KrisJordan
Symmetry between code on client and server has long been the promise of
server-side JS. I'm working towards a similar goal of reusing Backbone's
model/collection structure server-side. It does not attempt to pub/sub events
to all clients with an instance of a model/collection, at least not yet.

My project simply proxies client fetch/save/destroy calls via Backbone.sync to
the server. Through a RESTful URL convention and Express.js the model and
collection of the client's request are recreated, any transformations applied
and persisted, then sent back down the pipe. Server-side, models and
collections extend their client-side bases and introduce additional
functionality (i.e. per-user access control). Models and collections are not
kept in memory server-side beyond the duration of a request.

~~~
stdbrouw
Yeah, I've been using a similar approach. Not as fancy as using DNode or
Hendrik Joreteg's approach, but it's closer to traditional web app development
and thus easier to wrap one's head around. If real-time isn't a requirement,
that little bit of extra simplicity can mean a lot when testing, debugging and
scaling.

------
moe
This is a very interesting look at things. I'm currently building a backbone-
based frontend in a very similar way, only without the node.js counterpart -
yet.

One problem I immediately see in this hybrid approach is the requirement to
keep the state for all [active] users in memory on the server. This does not
scale well - which is why conventional web development tries to be as
stateless as possible.

It's not hard to remedy this _partly_ by adding basic lazy-fetching and cursor
capabilities to Collection. But you lose some of the most interesting features
in the process (push-notify on changes), and preserving those at the same time
becomes much more tricky then (bordering on classical ORM problems).

There's also still ground-level stuff missing in Backbone, i.e. the support
for recursive updates of nested collections (e.g. from one big, nested json
blob) is just not there yet. A few approaches have been discussed, but afaik
not implemented yet. The models need to be beefed up with some sort of schema
(e.g. json-schema) to truly solve this one.

However, none of that is insurmountable. All it takes is some more of the
smartness that the Backbone developers have already demonstrated. And elbow
grease.

For example I'd love to see backbone paired with 'dnode' or a similar RPC
approach. That might open amazing possibilities.

~~~
HenrikJoreteg
Thanks for your feedback. I've just been playing with this stuff and wanted to
put it out there despite its warts.

Regarding the scaling issue. Yes. This is definitely a weakness (I actually
mention it as a weakness in the last paragraph or so). We're working on a
solution for that using redis as a sort of shared memory. But it's just not
ready for primetime and I just figured I'd write up what I had going so far.

Regarding recursive updates of nested collections, the "mport" function in the
post works well for re-inflating the entire state. But not quite as well for
updates, but for this type of thing, I prefer piecemeal updates anyway, so we
don't have to send as much data each time.

I'm definitely gonna check out dnode some more as well. Cheers.

~~~
moe
_I've just been playing with this stuff and wanted to put it out there despite
its warts._

And thanks for that!

In hindsight my first reply came out more nitpicky than intended; all my
concerns are aimed at (still to be fixed) shortcomings in the backbone/node
ecosystem, not at your particular implementation.

Major kudos for not only coming up with (at the very least) an intriguing
Proof Of Concept but then also writing a great blog post to describe it. This
will be learning material for myself and many others.

~~~
HenrikJoreteg
Thank you. No offense taken :)

------
chanks
All the hubbub about the hashbang URL system got me thinking about sharing
client/server javascript just this morning, and then lo and behold, a post
about it pops up a few hours later.

Instead, though, I was thinking that perhaps controllers and views could be
re-used across the server and client. When a request comes in to the server,
it renders a page just like a typical MVC framework would, but also directs
the client to download the view/controller javascript. Any links the user
follows could then be rendered client-side. When the framework is running
server-side, the model layer talks directly to the database - when it's
running client-side, it talks to the server through JSON.

It could give you the best of both worlds - an initial request returns a
quick, fully-formed HTML page, so it's speedy on the first load and crawlers
will know what to do with it. Later requests use AJAX, so things are
responsive and the server isn't weighed down with HTML rendering.

~~~
wizard_2
Take a look at liftweb. <http://liftweb.net/> Its Scala which I don't know,
but I've got some very intelligent friends trying to get me to back it at our
company. (Scala is a lisp that runs in the JVM. - Someone's going to hang me
for confusing what lisp is.) Its a framework that can leverage
websockets/comet/longpoll to extend your classes to work client side.

Think message passing between client objects (in js) and server objects (in
scala). You can do jqueryish things server side to the client side pages.
(That blew me away when I saw it in action.) Your submit even can pass params
to the server object. It's really simple and strait forward. I'm looking for
lift for NodeJS.

Check out the ubiquitous "chat server" example in the simply lift book.
<http://simply.liftweb.net/>

------
jessedhillon
Is it me or is this proposing, for the sake of symmetry, that you expose
server-side implementation details on the client? Unless you have client-only
and server-only versions of the models, I would think that's what you have to
do. It seems like that's exactly what's happening when you have, in client-
side code, a boolean called "server" that indicates whether it's running on
the server or not.

Now, off the top of my head I can't think of a particular way that this could
screw you. But I can't say that it looks like the Right Thing to do either.

I don't know how far the Asana team got with Luna, but I think their approach
for a unified client-server model is much closer to how I'd want things to
work: <http://asana.com/luna/>

~~~
moe
_that you expose server-side implementation details on the client?_

This particular impl does it with javascript conditionals, but there's no
reason the client _must_ see code that it never executes anyways. The
equivalent of an #ifdef (ideally buried under a framework rug) goes a long way
here.

Moreover a function does not even need to resolve to the same code on the
client and the server. For example a model-validation function could evaluate
to the actual validation logic on the server (incl. DB lookups etc.) and to a
mere ajax RPC (calling that function on the server) on the client.

I could very well see a smart node serverside-library that let's you write
_one_ js file and then transparently mangles it as necessary before sending it
to the client. The trick is to keep the magic down to a sane amount (slippery
slope!) so that the end-result remains always predictable.

 _Luna_

Yes, it seems luna is basically taking this approach.

What I don't like about luna is that it's not javascript and that it's, for my
taste, taking the abstraction too far. It claims:

    
    
      the client-side JavaScript, the server-side SQL, and everything in between
    

That's too much. Sure, for the developers it's comfortable to sidestep a whole
cascade of hard problems simply by assuming control over everything ("My way
or highway"). But this approach has not proven very successful in the
javascript-world. We love small and pluggable components like jQuery or
Backbone. The monoliths like Dojo or ExtJS tend to be too limiting on the _My_
part of the way.

~~~
jessedhillon
I agree with your comments about Luna. About the #ifdef proposal, I think a
better solution might be that your server-side model object has a method like
getSlaveObject() that gives you a version of the model that is acceptable to
be seen on the client side.

~~~
moe
Yes, you are probably right, my examples were just meant to deliver the rough
idea, not fully shaped solutions.

Lots of room for experimentation and exploration in this space - exciting
times ahead.

------
mxavier
I _really_ could have used this tutorial a few months ago. I made a bit of a
toy node/backbone app. My first iteration attempted to unify the models across
frontend and backend, unit testing them, etc, but it ended up being so kludgy
and hard to debug that I went with frontend only and just stored the JSON on
the backend. The benefit of the OP's strategy however is that its much safer
to have the parsing and validation on the backend so people can't put bad
stuff in your database as easily.

In the end I ended up redoing the project as a CLI app in Haskell.

~~~
ericflo
That's quite the course correction, going from backbone/node to CLI/Haskell.

