
Eliminate HTTP requests using JavaScript diffs and LocalStorage - joshfraser
http://nodeknockout.com/teams/waving
======
ricardobeat
Google Chrome has had native support for SDHC
([http://en.wikipedia.org/wiki/Shared_Dictionary_Compression_O...](http://en.wikipedia.org/wiki/Shared_Dictionary_Compression_Over_HTTP))
for a long time now, but it has had near zero adoption.

CloudFlare also does something similar within their own network, they call it
Railgun: [http://blog.cloudflare.com/railgun-in-the-real-
world](http://blog.cloudflare.com/railgun-in-the-real-world)

~~~
diurnalist
One of the authors of the dynoSRC hackathon project here.

Thanks for the link! I wasn't aware of this white paper. Would be great to
integrate this as an automatic feature in the dynoSRC express middleware. I've
added an issue for us to look into (and implement if feasible) SDCH here:
[https://github.com/diurnalist/dynosrc/issues/3](https://github.com/diurnalist/dynosrc/issues/3)

------
cromwellian
If you have any kind of JS minification in your pipeline, then just applying
git diff is likely to produce large patches for small changes as renaming and
compiler optimizations on Javascript can vary wildly.

Google has had diff tools like this for awhile, as such, the Closure Compiler
contains a feature called stable name maps to prevent a small change from
causing all of the symbols globally to be minified differently. It's outputs
two files of variable and property names, and subsequent compilers can attempt
to minimize diff-cheese. They also help SDCH.

[https://groups.google.com/forum/#!topic/closure-compiler-
dis...](https://groups.google.com/forum/#!topic/closure-compiler-
discuss/MrENQdmovSE)

~~~
diurnalist
DynoSRC will actually insert white space into minified files before doing the
diff so it can still do line diffs. We were and are concerned about the
overhead of a diff on the minfication use-case, but it's promising that some
minifiers like Closure have settings you can use to ensure that two versions
of a file will match up relatively well when minified. Thanks for the link!

------
mikegioia
The problem here is that this could remove your ability to use localstorage in
your application for anything but your JS source. We were hitting issues with
some browser's 2-3MB limit when storing data for offline use.

~~~
eli
I believe it's at least 5MB on all major browsers (that support it).... but I
think text is stored UTF-16, so that gets used up more quickly than you might
expect.

~~~
Fishkins
I've been using LZString for data I put in LocalStorage. Obviously you can
still hit the limit, but it takes quite a bit longer.
[http://pieroxy.net/blog/pages/lz-
string/index.html](http://pieroxy.net/blog/pages/lz-string/index.html)

~~~
recuter
Quite a bit being basically twice as long. So you're up to localstorage being
usable with 6-10MB now which is nothing to sneeze at.

~~~
Fishkins
For the data I was compressing (JSON) it was getting it to about 17% of the
original size. Obviously that will vary depending on your data.

------
riskable
Gate One already does this (though it sends whole files not just the diffs).
Once a client has an asset (JS, CSS) there's no reason for it to be re-
downloaded unless there's a change.

Another improvement in Gate One's implementation is that it uses IndexedDB
instead of localStorage. This is significantly more performant--especially as
you add more/larger assets (localStorage blocks--always--whenever it is used).

It also downloads all assets over the WebSocket connection which has
significantly improved performance over multiple GET requests. It's quite
fast.

~~~
ricardobeat
Caching with localStorage is not new, the whole point is sending diffs for
changes instead of re-downloading everything. This improves performance even
for very dynamic data that is constantly refreshed.

------
frio80
I remember seeing something similar to this a while back called Diffable
[http://googlediffable.blogspot.com/](http://googlediffable.blogspot.com/).
Looks like the same thing but maybe I'm wrong... Still cool though.

------
adnrw
This is a neat idea, but a problem I didn't see discussed in the article is
that on iOS (at least), LocalStorage is disabled when running in private mode.

There are ways to detect it and work around it obviously but considering this
is pitched as a solution for mobile users, it's probably something to
consider.

~~~
diurnalist
Good point. I believe if the cache is disabled the fallback behavior will be
that all the JS assets just get inlined in full on the page, which is still
arguably better than requiring an extra HTTP request. I haven't tested this
yet, though, so it may not work out of the box yet.

------
ddebernardy
Doesn't this end up eliminating URIs and suffer from the same problems as
hashbangs? (Or perhaps those problems have been solved and fixed since?)

------
nornagon
> computes diffs by forking a git process on the server

sounds expensive. are the diffs cached?

------
gruseom
After applying a diff, how does it load the new code? Eval?

~~~
aaronem
I wondered that myself, but the answer is no; it appends a script element,
whose content is the updated code, to the head element. [1]

[1]
[https://github.com/diurnalist/dynosrc/blob/master/dynoSrc.js...](https://github.com/diurnalist/dynosrc/blob/master/dynoSrc.js#L119)

------
vezzy-fnord
This appears to be a rudimentary implementation of a delta encoding scheme?

I am unsure as to whether this will contribute to a significant performance
gain, or if most websites even require one like this in the first place.

------
JetSpiegel
"a lot of people have 4G" Welcome to the real world. What do they mean by a
lot?

~~~
hayksaakian
a lot of people (who won't close your site because it's slow) have 4G

------
ape4
premature optimization, anyone

~~~
voltagex_
When web "apps" are over 1 megabyte initial load, no, I don't think this is
premature. I think it's long overdue.

------
iLoch
Stopped reading after I saw the meme. :(

