

Sprockets: JavaScript dependency management and concatenation (SvN) - jakestein
http://www.37signals.com/svn/posts/1587-introducing-sprockets-javascript-dependency-management-and-concatenation

======
Hexstream
Sounds like great stuff! I think we shouldn't limit ourselves to source code
directory structure so much and this is a step in the right direction.

One problem I foresee with this approach is that it could make debugging
harder, if you have to look at the "concatenated" javascript file in
production to make sense of what code's included and not, it's harder to know
what comes from where than if you just included the original source code
files.

One solution would be to have a way to switch between "concatenated" and
"normal" modes easily (maybe it's already possible, I don't know), so that
normal development would be unimpeded. But you'd have to resort to the big
concatenated file for debugging in production.

Another solution would be to somehow preserve the information about what comes
from where, so that via a browser plugin you could browse the concatenated
file as if it were the original modular files.

Lately I'm getting really interested in ways to preserve information about
"what comes from where" so that you could point to something in a generated
output and ask the system to explain the sum of all inputs and considerations
(decisions, transformations) that resulted in that "atom" being generated in
the output through the various "semantic levels". You usually generate a low-
level representation from a higher-level representation and it can be useful
to proceed carefully from one transformation to the next so that you preserve
meaningful semantic information about intermediate stages.

------
jsdalton
Unless I missed something, this is exactly what django-compress already does
for the Django framework:

<http://code.google.com/p/django-compress/>

We just started using it in a project I'm working on. Most excellent.

~~~
sstephenson
Cool, django-compress looks similar to the sprockets-rails plugin. A few
differences:

django-compress makes you define your source files and concatenation order in
an array in a configuration file. Sprockets lets you define your dependencies
with annotations in the source files themselves, meaning plugins can have
their own dependencies and integrate themselves with your applicaton's build
system.

Sprockets doesn't do minification yet.

Sprockets doesn't do CSS concatenation, either. But it does have an asset
provisioning feature, where plugins can specify a collection of files (e.g.
css, images) that should be copied into the document root. (Rails itself
provides basic CSS and JS concatenation.)

It's nice to see other similar but independent implementations of the same
idea.

------
tlrobinson
I dislike the comment "require" directives way of specifying dependencies.
They should be real JavaScript function calls, which could either be analyzed
ahead of time and stripped, like Sprockets, or used with a JavaScript loader
system at runtime. This would remove the CGI/Ruby dependency and make the
whole system more reusable (for example, server side, where concatenation
isn't necessary/desired).

~~~
sstephenson
Respectfully: no, it shouldn't. :) There's plenty of libraries that do this
already. Sprockets solves a different problem.

~~~
tlrobinson
Can you elaborate? I assumed the problem was "JavaScript dependency management
and concatenation".

What I described is essentially what we do in Objective-J. During development
it just resolves the dependencies at runtime, but when you deploy you can
flatten them into one big file.

------
apgwoz
I love how the options are Rails Plugin or CGI. If they write their marketing
sites in PHP, why didn't they go with PHP instead of the CGI route? Or, is it
the case that the CGI was just a "hey, you don't need to use Rails to take
advantage of it" ploy?

EDIT: after reading the CGI source, they use some mod_rewrite trickery to
avoid running the CGI if theres a cached file.

~~~
sstephenson
I originally wrote Sprockets 1) as a replacement for Prototype's build system
and 2) as a library that could be used within Rails. Just before releasing
1.0, I added a simple CGI wrapper around the library so we could use it in our
marketing sites and so people could use it relatively easily without Rails.

I like the idea of distributing a PHP wrapper. I'll try to add one for the
next release.

And there's no "ploy." Interesting theory, though.

~~~
apgwoz
After finding that the CGI was smart, I can see the value in it. The "ploy"
comment was written "pre"-enlightenment.

------
cpr
Wonder how it interacts with browser cacheing and parallel JS resource
downloads?

~~~
RyanGWU82
JavaScript resources are never downloaded in parallel. They are downloaded
sequentially because they are interpreted while they download, and (existing)
browsers always run JavaScript in a single thread.

This will have very little effect on caching. You will still need to configure
your web server for optimal caching, by setting the Cache-control and Expires
headers, for example.

Even so, this still has two main benefits: The JavaScript is minified, which
will reduce the amount of raw data sent to remote users. Plus, by
concatenating all the files together, you avoid the need for multiple separate
HTTP requests.

~~~
axod
Incorrect. In the latest browsers, js can be downloaded in parallel, but
executed in the correct order - sequentially.

AFAIK, that includes the latest safari, firefox, and IE8.

However, it's still obviously a good idea to concat and minify all your
scripts.

~~~
RyanGWU82
Cool, thanks for the tidbit.

I had previously been told that a JavaScript download basically halts the
entire processing of the remainder of the page -- including any downloads that
might occur there (CSS, images, and more JS) -- because the browser can't
start parsing the rest of the page until it has interpreted the JavaScript.
That's why it's best to move your JavaScript down to the bottom of the page.

Is this just the way older browsers used to behave? Are there still any
restrictions like this that I should be aware of?

~~~
jacquesm
My rule of thumb with javascript is to put everything that is critical to page
functionality right at the top and everything else as far to the bottom as I
can (just before </body>), that way all your buttons/sliders and other active
components will work as soon as the page is rendered and any other javascript
will never hurt performance.

------
bdittmer
We use ant to accomplish the same thing.

~~~
jorgem
Yeah, I was gonna say "make" combined with some new tool to read the metadata
from the js files would probably be doable.

~~~
moe
We are using make for exactly that purpose, works like a charme.

