
How Basecamp Next got to be so damn fast without using much client-side UI - pors
http://37signals.com/svn/posts/3112-how-basecamp-next-got-to-be-so-damn-fast-without-using-much-client-side-ui
======
jashkenas
This is all fascinating ... but does it feel like a sustainable approach going
forward, or more like a turbo-charged horse and buggy?

Developer happiness aside -- there are plenty of folks who like JavaScript
just as much as Ruby -- and assuming that we're talking about an non-public-
facing app (like Basecamp Next), is there still an argument that can be made
in favor of doing your UI on the server side?

Even just in principle, it can't possibly be close to as fast (client-side can
render optimistically, where possible), and it can't be close to as flexible
(you don't have a model of state on the client to perform logic with).

I think that the truth of this is already admitted in that the really fancy
bits of UI are being implemented in JS:
[http://37signals.com/svn/posts/3094-code-statistics-for-
base...](http://37signals.com/svn/posts/3094-code-statistics-for-basecamp-
next)

~~~
dhh
I'm not sure what qualifies as sustainable or not in your book. We've achieved
our speed goals (sub 100ms pages for the most part), our development goals
(Ruby, Rails, server side pleasure), and our UI goals (app that feels like a
web page, not just a single-page JS app).

I'm sure Flash or Silverlight or other RIA people would argue that anything
that's not a compiled native experience isn't as flexible or as fast as
whatever they're peddling. Meh.

Yes, it's great to occasional dip into advanced client-side when that's
needed. Just like Flash had its legitimate use cases here and there. But for
the bulk of the UI interactions we're giving people, it's just not needed (or
desired).

~~~
arturadib
DHH - if you guys keep working so hard on your custom platform one day you
just might be able to reproduce the client-side JSON+JavaScript stack!

Keep up the hard work!

~~~
dhh
Is that supposed to be a slam? I've been working on "custom" platforms for the
better part of a decade.

~~~
arturadib
Hence why you're trying at all costs to make it work with things it wasn't
designed for?

~~~
smokinn
I'm pretty sure he knows what it's designed for better than you given the fact
he's the one who designed it.

Rails was literally lifted out of Basecamp into a standalone framework that
works great for CRUD-style apps. You might personally prefer another approach
or think this design failed but Rails was definitely designed for this use
case.

------
edw519
I love posts like this, not just for what OP has done, but for what you guys
have to say about it. I'm continually asking myself 3 questions:

    
    
      1. Should I do it on the client or the server?
      2. What should be travelling between the client and the server?
      3. Based on the answer to #2, what else do I need on the client?
    

Even though I've read all your great discussion points, I'm not sure I'm that
much smarter. But it's nice to know I don't suffer alone. :-)

~~~
irahul
> 1\. Should I do it on the client or the server?

I think most applications use both client and server components. If your
question is more on the lines of which one plays a major role, it depends
mostly on the app. GMail won't be half as good without all the JS magic, but
again, I assume it also has a heavy server component.

> 2\. What should be travelling between the client and the server?

If you need any client side processing or are using a framework which works on
raw data(backbone.js), then json/xml/...; or else if your goal is to run the
app without reloading unnecessary parts, html is a better idea.

> 3\. Based on the answer to #2, what else do I need on the client?

I am not sure what you are looking for here, but if your goal is to make a
responsive app, there are some basic things to take care of.

Have bookmarkable links and don't break the back button. Doing that is
breaking the base paradigm, and plain ajax does that.

Use something that provides bookmarkable links, and doesn't break the
back/forwrad button; and at the same time, doesn't reload the whole page when
only a small portion of the page needs reloading.

pjax is one such solution - the 37signals guys investigated it, then went with
their home grown solution, but I think pjax will work just fine for majority
of use cases.

The trick to pjax is serving content without layout if it is a pjax reques;
with layout otherwise. There is nothing you can't handroll, but 1) why would
you want to? 2) pjax uses pushState to preserve links and back button
behavior.

Here is the pjax code:

<https://github.com/defunkt/jquery-pjax>

And here is a gist in Flask/Python which shows conditionally including the
layout:

<https://gist.github.com/830fc28c680e77759b4b>

~~~
mark_l_watson
Thanks you for the links. I found a simple Rails pjax app and I am going
through the code right now: <https://github.com/edison/pjax-rails-sample>

~~~
irahul
Looking at the rails pjax_rails gem <https://github.com/rails/pjax_rails> I
see it automatically patches layout to return false when the request is pjax.

[https://github.com/rails/pjax_rails/blob/master/lib/pjax.rb#...](https://github.com/rails/pjax_rails/blob/master/lib/pjax.rb#L5)

    
    
        layout ->(c) { pjax_request? ? false : 'application' }
    

So in the sample app you pointed out:

    
    
        def show
          @post = Post.find(params[:id])
    
          if pjax_request?
            render layout: false
          else
            respond_to :html
          end
        end
    

is the same as:

    
    
        def show
          @post = Post.find(params[:id])
        end
    

Also, pjax-rails converts the following selection to pjax:

[https://github.com/rails/pjax_rails/blob/master/lib/assets/j...](https://github.com/rails/pjax_rails/blob/master/lib/assets/javascripts/pjax/enable_pjax.js.coffee)

    
    
        $('a:not([data-remote]):not([data-behavior]):not([data-skip-pjax])')
    

I was a bit baffled by the example app since it didn't appear to be doing any
pjaxy stuff, but rails being opinionated, I see all links are pjax(except the
not classes show above), and layout is turned off for pjax requests.

~~~
mark_l_watson
I put some debug printouts in his example notes controller and it seemed to be
setting the no layout rendering option correctly, but as you say, the checks
in the controller may be redundent. Anyway, I am planning on giving pjax a
very good look: I find doing a lot of client side coding to be tedious and is
something to be minimized.

------
mattmanser
Hmm, degraded development experience of doing everything on the client?

Bit of a bizarre claim there given the state of templating these days. Sounds
more of a prejudice than an actual problem.

Also json tends to be much lighter than HTML and if you're taking an
optimistic view of updates you can assume the action's been a success and even
immediately update the display without waiting for the server to create some
simple HTML, so it's arguably a worse user experience due to the lag.

~~~
dhh
We've found that the additional overhead of sending HTML vs JSON is negligible
in most cases. The additional speed gained by just sending JSON across is not
worth the programming enjoyment hit of doing everything client-side.

When you get below 50-100ms per action, things are generally fast enough that
this is not a key problem any more. The user cares greatly if you can go from
800ms to 100ms, but not so much if you can go from 100ms to 70ms.

~~~
zaidf
I have to agree here. A major problem with client-side json is the need for
additional documentation and likelihood of things breaking because of simple
html edits.

For example, if you are populating a bunch of divs inside a parent div with
the data returned using json, you would describe this using something like
jquery selectors. Now if someone else comes and moves around the div during a
redesign, he must go through whole bunch of js to make sure his shuffling of
the divs around won't break the jquery selectors populating the data from
json.

Now think about having a parent div that is simply populated with returned
HTML from server-side and it seems way easier than using json to populate the
data.

~~~
jonknee
That's nonsense, the same issue would apply on the server side templates. If
you go around messing with DIVs without an idea of the consequences, you will
get in trouble regardless of your templates being on the server or client. If
you try to have the designer do a redesign without testing the application
afterwards, you're always in for trouble.

~~~
zaidf
_you will get in trouble regardless of your templates being on the server or
client_

My view is that you will get in _significantly_ less trouble because
manipulating an HTML template that is rendered on server side is much easier
to manage and adapt to a new design than finding nitpick jquery selectors
across the app that depend on a specific div structure.

Btw, my post doesn't imply anywhere that a designer should redesign an app
without testing. Therefore, your argument about testing is garbage.

~~~
agilebyte
Rather than "finding nitpick jquery selectors" I use a client side framework
(backbone.js) with a lot of Views which makes my development faster vs using
server side templating, so I cannot share your opinion.

------
latch
I have to say, reading this made my day. I've been feeling such a push towards
client-side templates lately, that I felt like it was just me that didn't get
it. I really don't understand what's so compelling - unless those exact same
actions are actually (YAGNI) used as endpoints of a web service. render
:partial => '..' is such an sweet, simple and natural paradigm to web
programming.

~~~
roc
> _"I really don't understand what's so compelling - unless those exact same
> actions are actually (YAGNI) used as endpoints of a web service"_

Well of course it's not going to appear all that compelling after you rule out
_the most compelling use case_.

~~~
latch
Hey, I totally agree that if your actions are gonna be consumed by clients
other than your own UI, it makes _total_ sense. However, I've seen people
[strongly] advocating this approach even when this isn't the case..that's what
I don't get.

~~~
roc
Well there are other benefits. [1]

Though I suspect you're just talking about cases that are best chalked up to:
there's no accounting for the logic of the illogical.

[1] If you want to ensure a separation between display code and business
logic, it's not a bad way to draw a line; some heavily data-driven problems do
tangibly benefit from fetching JSON and formatting on the client; solutions
that have multiple views on essentially the same data sets benefit in much the
same way that solutions with multiple clients benefit; etc.

------
ayanb
Cross posted from a comment by DHH on this post

>> We use nodejs for local development via Sam’s Pow server, but there’s no
nodejs in Basecamp Next itself. It’s all Ruby on Rails. We’re running the
default stack using Rails 3.2+, MySQL 5.5, Memcached, a tad of Redis (mostly
for Resque), ERB , test/unit, CoffeeScript, Sass, and whatever else is in the
default box you get with Rails.

------
iamleppert
Awesome, very interesting article that shows you don't always have to go full
client side to get a snappy app.

However, I do agree with another poster about this being a turbo charged horse
and buggy. You can't, for example, deploy the app to a static CDN, or serve
templates from a CDN. Templates can't be cached on the client, so you've
developed what is essentially an elaborate partials caching infrastructure
that is likely very brittle and must be watched over closely to maintain speed
benefits (keeping all those things in mind when content changes -- what to
throw out and what to keep, etc).

Also, by delivering HTML you're limiting your presentation to browser-only
devices, at least with this app. I know basecamp has a JSON API, and at that
point, why didn't you just do a traditional client side app I wonder? I think
the answer rests in the fact that you preferred the "niceness" of server side
development, which is arguably more mature than client side at the present
time. If that's the only benefit, in a few years, do you think that advantage
will hold true?

------
hello_moto
I find GWT to be the best platform to do full-blown fat client-side paradigm
due to several reasons:

1) UI Templating (like XAML)

2) MVP Pattern + JUnit = test automation as part of your build with minimum
investment to infrastructure (most JS based solution requires investment on
infrastructure)

3) Resource Bundle (similar to Asset Pipelining) cut down HTTP requests

4) Resource splitting (follow up from #3) to reduce the size of the responses
when it comes to resources

5) JS binding (like C binding, C++ binding, JNI, etc) to support popular JS
library

6) Top Notch Compiler (arguable... but it's there) that can also prune dead
code

7) JS switching based on browser's request (IE browser will get IE-specific
JS)

8) CSS variable substitution (like Sass)

9) History framework.

10) Flexibility: end point can either return JSON, XML, or use the built-in
XML-RPC mechanism (Java only).

The disadvantages are: it's Java and requires compilation.

I like GWT the most so far because I don't have to find libraries or tools
that support all of the above (Sass, mustache, various js unit-testing tools,
library to merge and compile css/js, etc).

YMMV.

~~~
bradleyjg
I like GWT as well, but its development history makes me a bit nervous about
its future.

It is open source, but doesn't have a developer ecosystem outside of google
employees (in fact they outright state that no one else can be a committer).
That in itself wouldn't be the end of the world, but unfortunately google has
a) reduced resources devoted to GWT dramatically in the last year or so (many
team members were reassigned to the dart project), and b) has a history
rapidly changing 'paradigms' and semi-abandoning the code and documentation
for the old way of doing things.

~~~
hello_moto
I think the biggest danger of using GWT is when it breaks on newer browsers
and this has happened once in the past with the release of Safari 4 (or 5?)
there was a minor hiccup in which they quickly fix it the next day.

That is definitely a major glare. Other than that, if the community decided to
fork it (and have the time and capability to do so) it'll be a safe choice for
a long time.

GWT as of today is definitely very polished and well maintained and fulfil the
needs of Single Page App smoother than having to hunt different tools and
libraries and to set them individually to be a part of the build tool.

------
adeelk
I think in the whole client-side rendering debate, a lot of people miss the
fact that there is a one-to-one correspondence between JSON and a certain
subset of HTML, that can be illustrated by the following example:

    
    
        {"user": {"first_name": "David",
                  "last_name":  "Hansson"}}
        
        <div class="user">
          <div class="first_name">David</div>
          <div class="last_name">Hansson</div>
        </div>
    

If, as 37signals basically seem to be doing, you restrict yourself to this
subset of HTML and use CSS to do _all_ presentational styling, then honestly
it’s all the same either way.

------
rkalla
Are you guys taking care to send as-plain-as-possible markup from the server
to the client and then style it up to keep most the UI development in CSS or
are you just sending through raw, styled HTML that gets inserted as-sent
directly into the page?

This reminds me of the early days of Java Servlets when people would stream
back HTML from inside their Java Servlet code to JSP pages to build it out...
made it impossibly hard to edit the templates.

Then the Java world moved to templating engines, but it was still the same
idea.

Just curious how you are managing this pain point (I am not a Ruby dev so
maybe there is a well-known 'best practice' templating approach?).

~~~
dhh
I'm not sure I understand the pain point you're describing. What's "styled
HTML"? We're sending HTML across the wire that looks exactly like the HTML we
used to render the initial page. It has classes, ids, and custom data
descriptors.

We have not experienced any pain doing this.

~~~
rkalla
You answered my poorly-worded question. Thanks.

------
_johnny
Interesting. We've just had a discussion in our team about moving as much as
possible to the client becasue... developers enjoy writing CoffeeScript!

------
espeed
Is this similar to Quora's livenode/webnode2 (<http://www.quora.com/Quora-
Infrastructure/What-is-Webnode2>, [http://www.quora.com/Quora-
Infrastructure/What-limitations-h...](http://www.quora.com/Quora-
Infrastructure/What-limitations-has-Quora-encountered-due-to-
Webnode2-LiveNode), <http://www.bigfastblog.com/quoras-technology-examined>,
[http://www.quora.com/Quora-Infrastructure/How-does-
LiveNode-...](http://www.quora.com/Quora-Infrastructure/How-does-LiveNode-
work))?

------
va_coder
I don't think push state is supported in IE (except ie 10):

<http://caniuse.com/#search=pushstate>

This makes it a non-starter for me.

~~~
akavlie
The app still works in IE, it just requires page refreshes on every page load.

------
robinduckett
Aren't they are using a __lot __of client side UI code in order to replace
parts of the UI with updated HTML from the server?

They have caching on the server side but that can only account for some of
this.

Although in my couple of years of using Basecamp in my previous job, I never
found it particularly "snappy" but then I was one user out of a team of five,
out of how ever many users they have.

------
joshontheweb
I can't help but feel that you are sacrificing the flexibility of your UI in
order to stay in your 'safe place'. IMO its easier to implement complex UI if
you can deal with it all in the front end. I have used a similar approach
before and felt like my hands were tied many times by it.

------
jshen
This ia a bit of a tangent, but Stacker feels like breadcrumbs that require
more real estate than necessary. Maybe the physical metaphor is important, but
my first thought was 'this is just bread crumbs'.

------
fleaflicker
This is very similar to what Google+ does with Closure Templates.

------
kristianp
I'm showing my ignorance of rails here, but is it hard to do the 'russian
doll' style of fragment caching that DHH describes? Are there any gems that do
this?

------
deepkut
I just realized how "next last year" is an oxymoron... Or is it? A reference
to the present per chance?

------
gabyar
Is Stacker a proprietary 37 Signals thing? It doesn't seem to pop up after
several search attempts.

~~~
latch
yes, the blog post was quite clear that it's built in-house, with no plans to
open source it since it isn't a generic solution (it's purpose-built with
Basecamp in mind).

------
dreamdu5t
Summary: Use window.pushState() to reduce HTTP requests and cache _every
thing_.

------
bjeanes
> Alexander, I think the Law of Demeter is shit and never follow it.

Classic DHH

------
tapertaper
What's the point of putting the tl;dr at the bottom?

------
jarnix
advertising

------
abdelm
or, you know, you can also stop using Rails.

