

Diaspora re-writes its front to Backbone: why and what it means - plunchete
http://devblog.joindiaspora.com/2012/01/10/client-side-re-write-oh-my/

======
jashkenas
The way that this post describes porting and existing interface from server-
side/ajax rendering to client-side rendering is particularly interesting -- I
imagine that there aren't many apps that have made that jump _after_ first
being built around server-side templates.

The speed benefits that Diaspora is seeing from distributing their HTML
rendering is probably only the first step. There are _so many_ more
interesting interactions you can accomplish once your data is being modeled in
the browser.

~~~
windsurfer
Like what? Fluid layouts?

~~~
jashkenas
Live re-rendering of data -- imagine if the (now) "24 minutes ago" timestamp
next to your handle was always accurate, instead of just reflecting when I
loaded the page.

Easier implementation of otherwise difficult features. Doing a client-side
autocomplete of people's names is trivial when you already have all of the
other accounts in your organization modeled in JavaScript ... but not as fun
when you have to ajax for server-rendered HTML for it.

Live updates of pushed data. When another reader +1's a post that you're
currently looking at, if the server pushes that data to you, and you have a
model for that post -- it's easy to increment the counter. If the server has
to push the re-rendered HTML for the entire post, it's much more difficult.

... and those are just the tip of the iceberg.

~~~
devs1010
This can be done without the radical switch to modeling everything client-
side. I mainly do java web apps with JSP's (which everyone seems to think is
stodgy and past its prime) but its rather simple to do something like what
you're describing with keeping things up-to-date. I simply have broken my
project down into "widgets" that self-load so a page actually is made up of
many widgets that use javascript to call various server-side controllers, you
can then set them to do whatever you want (such as refresh every 30 seconds)
and it doesn't require some massive client-side data model.

~~~
hippich
Your solution is more like VNC, where backbone solution is more like X11. Both
can live side by side and used in different situations.

~~~
devs1010
I agree, there are cases where you would want a more robust client side data
model, but, IMO people are a bit too quick to jump to using this and it can
make more work in the end and make a project more costly to maintain. I think
the cases where it is useful are mainly for highly interactive sites like
those that would have previously done in flash, with animations (such as
games, etc), for just regular CRUD webapps I feel its a bit of overkill

------
AznHisoka
This is the biggest win for Diaspora? Really? Sounds like they've been deluded
by all those NYU CS professors who think knowing how to add 32 bit floating
point numbers is the key to success.

Just dump everything, spend 1% of your time rewriting everything in PHP, and
the rest actually doing some marketing, not imaginary work.

~~~
kmfrk
I'm also getting a "premature optimization" vibe from this[1].

[1]:
[https://en.wikiquote.org/wiki/Donald_Knuth#Computer_Programm...](https://en.wikiquote.org/wiki/Donald_Knuth#Computer_Programming_as_an_Art_.281974.29)

~~~
Joeboy
They're running a fairly large scale app, which has a lot of users and
recently experienced serious capacity issues (after Ilya's fairly well
publicized death). Optimizing it doesn't seem premature at all.

------
tripzilch
Maybe this is a really stupid question, but wasn't Diaspora supposed to be
decentralized? In the sense that anybody could set up their own hub? (so
people would not have to accept features pushed by some big corporation and
things like that)

Then what is this whole database backend for? Is it like the "main" Diaspora
hub? And could anyone set up their own secondary one? Would they run into the
same difficulties they're trying to solve here?

Though I get the feeling I'm probably misunderstanding the entire Diaspora
project, here.

~~~
icebraining
Each hub has its own database to store the data about its own user(s). Then
the hubs can communicate with each other so you can "friend/follow" users from
other hubs.

~~~
tripzilch
Okay, then I did understand it correctly (phew!).

But doesn't that mean that the problems the Diaspora team has to solve now
will also pop up for other hubs as they grow in scale?

Or is the solution to create loads of smaller hubs and have them communicate?
Then why don't they do that?

~~~
icebraining
The problems Diaspora is solving will be solved for the other hubs too, since
they all use the same codebase.

~~~
tripzilch
That makes sense, thanks!

------
ferrofluid
Sorry, I don't know what "Backbone" is. I see links to New Relic and Pivotal
Labs, but no links to anything called "Backbone".

I guess this blog post was maybe not meant for purely technical people, but it
would be nice to understand what "Backbone" is, and exactly why it would solve
their problems.

~~~
Helianthus
I hate to be That Asshole, but <https://www.google.com/search?q=backbone>

The first link is to a javascript library widely known in the web development
community.

~~~
ferrofluid
I actually did a google search for
<https://www.google.com/search?q=backbone+web>

Which gave me the first hit of www.backbonemedia.com/

In fact, backbone.js is no where on the first page for that search.

That's amazing it comes up first for "backbone". I did actually find that link
eventually, and try out the "example" todo list, but that gave me no
information about how this would fix a garbage collection problem. Does anyone
know how it fixed their GC problem?

~~~
scottschulthess
The article was confusing a bit confusing about the Garbage Collection so I'll
try to explain.

1\. Dispora is slow, why?

2\. Requests take a long time on the server, why?

3\. Most of the time is spent in ruby doing processing.

4\. Why is ruby slow?

5\. Ruby is slow because of the garbage collector.

6\. How can we get around this ruby being slow problem?

7\. Reduce calls to server side by writing replacing server side calls with
javascript rendering of templates and whatnot.

------
JoachimSchipper
500ms+ for rendering a template seems ridiculous. Were they just Doing It
Wrong, Ruby-wise?

~~~
cheald
If I had to guess, they're running with the stock Ruby GC parameters, which
are fairly horrible for a Rails app, and can result in multiple GC cycles per
request. Ruby's GC is painfully slow, so if you're kicking into it, you're
going to slaughter your response times.

REE and Ruby 1.9.3 both offer GC tuning parameters, which let you instruct
Ruby that you're going to shove a huge app at it and to not garbage collect
every time someone sneezes, which can have a pretty massive impact on response
times.

------
charliesome
It sounds like they were jumping to conclusions regarding the templating being
responsible for creating a large number of objects.

ERB, Haml (which is what Diaspora uses), and any other templating engine I've
seen use either concat or << when rendering a template. These never create a
new object, they mutate (and perhaps resize) the original string.

Maybe next time they should profile better before following their gut feeling
and rewriting their front end ;)

~~~
Estragon
They got a 3-fold speed up. Seems like they were doing something right.

------
peterhunt
I'm coming from a Python perspective; no idea if this is easy/possible in
Ruby.

Maybe this is a really stupid idea, but what if we _gasp_ disabled the GC
during the course of each request and did a collection run after the request
is completed and before the next one is accepted? With a sufficient number of
workers, wouldn't this solve some of the problems they were having?

~~~
SamReidHughes
So you'd only handle one request at a time?

~~~
mnutt
Rails usually has a request lock so most rails servers end up forking off a
number of processes that each only process one request at a time.

------
memoryfault
This post made me curious to learn more about new relic and their
architecture. I found this post pretty interesting:
[http://highscalability.com/blog/2011/7/18/new-relic-
architec...](http://highscalability.com/blog/2011/7/18/new-relic-architecture-
collecting-20-billion-metrics-a-day.html)

------
gaius
Does it mean they won't blow away all their user accounts when they get bored
and start again? No? Well that is why Diaspora is a joke.

~~~
aarondf
I don't understand the "blow away all their user accounts when they get bored"
statement. I'm guessing I'm missing something in Diaspora's history?

------
charlesju
Is there a main deployment of Diaspora somewhere? Or is this a purely academic
project?

~~~
icebraining
It's the main domain: <https://joindiaspora.com/>

Public content is indexed by Google, like the 'cake' tag:
<https://joindiaspora.com/tags/cake>

------
emehrkay
okay <http://i.imgur.com/jHu1N.png>

this is from webkit nightly after I clicked a user name and got a "you messed
up" 404 page and went back

------
marcosvm
Rewriting the front-end using Javascript and Backbone won't fix the backend
problems necessarily. How about reducing the objects creation and tuning the
backend too?

~~~
jroll
They discussed this in the post. They found that the templating was the source
of many of the objects, as Ruby creates new objects when performing string
concatenation.

------
bionicbrian
Rad! That graph is awesome!

Also cool that the source is public on Github. thanks for that. Always fun to
peek in on source.

