

Backbone Events vs. Ember Bindings: A Benchmark - jashkenas
http://jsfiddle.net/jashkenas/CGSd5/

======
tomdale
As I noted in my lunch talk at Carbon 5 the other day, Ember.js optimizes for
developer productivity--which is not to say that you can't have multiple
goals, but at the end of the day, one thing has to be your priority.

For the past few months, we've prioritized creating an API that is friendly,
conceptually simple, and correct. Our view implementation, for example,
handles many edge cases that other frameworks might punt on.

We believe that nailing the API first is important for a number of reasons. We
can increase performance without you having to change your app; the same is
not true if we spend time making it fast, then require you to change the API
once we realize it is not a good one.

The cool thing about declarative systems is that once you express your intent
in a way the framework understands, we can apply heuristics and optimizations
that every app developer gets for free.

Some people pooh-pooh the idea that better abstractions require a
"sufficiently advanced compiler," but I think the history of the field has
proved that it's the case. V8 and LLVM are examples of projects where we
continue to eke out performance increases well past what I think the average
person would have expected. I saw the same thing in Handlebars-- a lot of
people criticize Handlebars for being separated into multiple steps, as it
increases the size of the library somewhat, but being able to reason about
each of the steps in the compilation pipeline has given contributors the
ability to reason about possible slowdowns and turned it into one of the
fastest template libraries around.

So, all of that being said, I would never recommend using bindings for
animation. I'm not aware of _any_ framework or library that recommends this.
Bindings were designed for infrequently changing values that need to be
displayed to the user. Since the smoothness of animation is critical to the
user experience, it's hard to argue that any optimization is premature
optimization. All of the animation we've done in our Ember apps is done via
either CSS transitions, or using requestAnimationFrame and modifying the DOM
element directly.

While Ember.js will presumably always be technically slower than Backbone,
slower doesn't have to mean slow. That being said, there are obvious
performance gains to be made here, and I'm glad we've got Jeremy to keep us on
our toes. :P

------
jashkenas
To respond in advance: Yes, this is an entirely artificial setup. The point is
to start to get a feel for the overhead that different binding strategies can
incur -- a question that folks often ask about.

If you're never rendering complicated UI, or never have more than a couple
models, this will _never_ make a difference. But if you have a large list of
models, or need to run in IE8, or need to run on mobile, it very well may.

 _Edit_ : For folks wondering about the cause behind the difference --
Yehuda's looked into it more, and knows more about it than I do -- but there
are a number of factors that could be at play:

* bulk innerHTML updates vs. granular DOM twiddling (especially in different browsers).

* Handlebars vs. Underscore templates -- although Handlebars templates are quite fast.

* Fine-grained bindings needing to be kept in sync inside of Ember for Metamorph.js purposes.

* Other miscellaneous overhead.

Some quality time with the Chrome CPU profiler should be able to tell you
more. FWIW, Yehuda seems to link it's the latter: overhead that should be
possible to eliminate in time.

 _Edit 2_ : Looks like @ericf's forked and added a new YUI version:
<http://jsfiddle.net/ericf/NrpcQ/>

~~~
wycats
I can address some of these, as I've been staring at the profile quite a bit:

"bulk innerHTML updates vs. granular DOM twiddling (especially in different
browsers)."

A part of the cost is definitely in making these small changes, but not a huge
part. For what it's worth, we're in the process of implementing a feature that
would allow you to specify the re-rendering granularity in the template
itself, but we don't want it to be heavily used by end-users.

"Handlebars vs. Underscore templates -- although Handlebars templates are
quite fast."

I don't see the cost of rendering handlebars templates at all in the top of
the profile, which is not surprising, as the templates we're rendering are
very small, and Handlebars templates are very fast.

"Fine-grained bindings needing to be kept in sync inside of Ember for
Metamorph.js purposes."

The cost of bindings are not ongoing, but rather when a `set` happens.
Internally, this code is somewhat equivalent to the Backbone event system (we
even trigger `change:foo` events) with some more chrome on top to manage
things like computed properties. Most of the cost here is not in the internals
of the binding system, but rather in the effects of the binding system, like
re-rendering the `{{content}}` and updating the style.

In short, the internals of Ember know about as much on an ongoing basis as
Backbone's internals know: namely, that there is an observer on
`change:content` that needs to be triggered when the underlying property
changes. Again, we do more work once that event is triggered than Backbone
does, but the vast majority of the cost is on the receiving end of the events,
not in the event system itself.

"Other miscellaneous overhead."

From my analysis of the profile, this looks to be the primary cost. The view
layer itself could stand to be more optimized. For example, we use a state
machine internally to control what events are legal in different states
(preRender, inBuffer, hasElement, inDOM). It was our first state machine
implementation inside of Ember, and we didn't do some obvious optimizations
(like cache legal transitions).

The ember-states code, which ships with ember and is used in ember-data, is
far more optimized, and moving to use it should eliminate some of the cost I'm
seeing. At the very least, it provides us with a central state machine
implementation that we can continue to optimize over time, resulting in
performance improvements all around.

There are other areas where micro-optimizations will almost certainly help
(such as in the event system itself and how it's used).

~~~
krisselden
A bit more of an apples to apples comparison
<http://jsfiddle.net/krisselden/bYPPT> and just to illustrate, Ember doesn't
force you to micro bind and one would never suggest it for animations.

------
Lazare
In case anyone cares, here's a jsfiddle for KnockoutJS:
<http://jsfiddle.net/2pLk3/5/>

I was curious how it would stack up - the answer seems to be "pretty well,
actually". Of course, all the normal warnings about synthetic benchmarks
apply. :)

Edit: I'm updated the JSFiddle to _attempt_ to properly clean up the Knockout
bindings. Probably. As I said, I'm pretty new to Knockout still, and may well
have made a mistake.

~~~
schinckel
You are not actually using ko.computed objects in your fiddle.

My take on it: <http://jsfiddle.net/HusVw/1/>

And guess what, it's _much_ slower. (And I'm a happy KO user).

~~~
krosaen
There's some discussion on the ko google groups about this :

[https://groups.google.com/forum/?fromgroups#!topic/knockoutj...](https://groups.google.com/forum/?fromgroups#!topic/knockoutjs/xb0oS00ndlE)

you are right, the correct version is perhaps slightly faster than the ember,
but way slower than the backbone example.

However, by leveraging an optimized ko.computed implementation

<http://mbest.github.com/knockout-deferred-updates/>

the results are much more impressive.

<http://jsfiddle.net/2pLk3/10/>

~~~
Lazare
I'm don't agree that that the version with unused intermediate computed
observables is the only correct version (or that there even _is_ only one
correct version).

Avoiding the use of intermediate computeds makes the Knockout version roughly
as fast as Backbone for this specific benchmark, and more closely replicates
the functionality of the Backbone example. That sounds like the correct
version to me.

------
dts
One thing that jumped out at me pretty quick in comparing the two
implementations in jsFiddle is that with ember every property is set with an
individual call rather than passing in an object of properties to change. Here
is a version with backbone using individual set calls with a notable dip in
performance: <http://jsfiddle.net/6rvkC/>

Is this limitation in ember because of the computed properties? Would changing
this even help?

~~~
wycats
Interestingly, not really, because the Ember run loop implementation coalesces
multiple calls to set, so using setProperties (which allows you to pass in
multiple properties at once) doesn't really affect performance much.

It's one of the first things I tried, in case there was a bug in the run loop
code that caused the re-render to happen multiple times.

~~~
bcrescimanno
I think it's worth noting, that Backbone (as expected, I imagine) gives you a
little more rope to hang yourself with here. To me, that's a great highlight
of how the frameworks differ in their approach. "Perfectly written" Backbone
code is significantly faster; but, if you (perhaps unknowingly) take a slower
path, Backbone isn't going to do anything to make it faster. Not advocating
one approach over the other; just pointing out the juxtaposition.

~~~
jashkenas
Quite, and it's also worth noting that by relying on a "runloop" to defer your
renders, you're giving up synchronicity in your data changes -> ui updates
when you don't necessarily need to (a very common source of dev bugs in
SproutCore 1).

~~~
bcrescimanno
Not having worked with SproutCore, I honestly can't comment on whether it
introduced more bugs. Working with anything async is always going to introduce
some complexity; but given the poor DOM performance on low-end devices,
deferring rendering is often a very helpful concept.

I'd be interested to hear what type of bugs were common in this scenario; my
understanding of the runloop is that after the DOM update (at the end of each
loop) the UI should reflect the application state.

------
joubert
For those interested in, or learning Ember, I've taken Jeremy's Ember code and
rewrote it in idiomatic Ember style, with commentary -
[http://joubert.posterous.com/application-architecture-in-
emb...](http://joubert.posterous.com/application-architecture-in-emberjs)

------
winteriscomming
Both make my Core i7 CPU go to 100% for a simple animation of 100 small
circles.

I thought HTML5 was going to solve all performance problems and make me
breakfast too. Wonder if this will even run on mobile.

------
mhd
They're both pretty slow considering the ridiculous hardware of today's
computers. We're really running a lot of our stuff on layers and layers of
performance-draining abstractions... I remember the days when my friends were
complaining about the unneccesary slowdown of MUI apps for their Amigas and I
was reading Michael Abrash's Black Book to improve my line-drawing routines.
Granted, I surely don't want to go back to the days of Mode X DOS programming,
but the whole web stack is really one baroque mess, despite the heroic
achievements of browser and Javascript VM developers.

~~~
cageface
I completely agree. So much clever engineering is being devoted right now to
making HTML do things it was never intended to do. I'm not happy about the
shift to proprietary native platforms but selfishly I'm so much happier coding
close to the metal in iOS with a well-designed UI toolkit.

~~~
mhd
Close to the metal? Well, in your case that might be true, but I see more and
more applications on mobile devices that are pretty small wrappers around
Webkit views, as this makes portability easier and harnesses the existing
skills of the developers.

------
moe
Micro-benchmarks are useful, but please keep in mind that you'll most likely
not be updating hundreds of objects multiple times a second in a real
application...

~~~
oscilloscope
You might be, it depends on the application. That's exactly what's going in in
this NY Times visualization:

[http://www.nytimes.com/interactive/2012/02/13/us/politics/20...](http://www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-
proposal-graphic.html)

Though it uses d3 rather than Backbone or Ember.

~~~
n_time
That is an amazing visualization.

------
rgarcia
Could the difference be mostly due to handlebars vs. underscore? I'd be
curious how it'd look normalizing for templating engine.

~~~
braddunbar
At first glance it seems that template rendering is a key point here (render-
as-a-string vs. dom-manipulation). However, if you change the backbone
implementation to use dom manipulation (<http://jsfiddle.net/jqSK2/>) it
performs just as well (or better, depending on the browser). Backbone's event
system is much more important in this comparison.

------
sid6376
Out of curiosity and apologies if this is a lazy question but why is the
difference so pronounced?

------
bootz15
They both work well when N = 2 ;)

~~~
ypcx
exactly, and that's why i will use neither

