

CommonJS Is The Future For Rails JavaScript Packaging - maccman
http://alexmaccaw.co.uk/posts/2011/06/30/rails_js_packaging.html

======
sstephenson
As the creator of both Sprockets, which underlies Rails 3.1's asset pipeline,
and Stitch, which the poster of this article ported beautifully from Node.js
to Ruby, I feel obligated to reply here :)

While CommonJS modules are undeniably elegant, they carry with them a high
conceptual overhead. It would be unreasonable to expect Rails developers to
rewrite all their existing JavaScript in order to take advantage of the asset
pipeline. Most people do not want to manually import every dependency into
every source file (and in fact Rails goes to great lengths to make use of
Ruby's "require" method largely unnecessary).

Sprockets solves this with the "require_tree" directive that automatically
pulls in all source files in a given directory. If you've been keeping your
JavaScript source files in public/javascripts, it's trivial to upgrade your
app to 3.1 and take advantage of the asset pipeline — just move your source
files into app/assets/javascripts. Then when you do need more specific control
over source file ordering, the "require" and "include" directives are there
for you. Combine this with the implicit closure around each CoffeeScript
source file and Sprockets gives you 80% of the benefit of CommonJS modules
with 20% of the complexity.

Stitch (and CommonJS in general) does not address the problem of CSS
preprocessing or concatenation, and does not provide a mechanism for managing
other assets like images and sounds. Sprockets supports Sass and SCSS, and
provides a global load path which lets you depend on JS, CSS and images from a
single logical package — even a Ruby gem — with essentially no configuration.

I hope this clarifies why we chose to go with the Sprockets approach rather
than CommonJS for Rails 3.1.

~~~
rasmusrn
Sprockets can be used for JS, CSS, images, and so on, because it is so simple:
it is "just" concatenation. I understand and agree on the benefit/complexity
argument.

But the fact that this method works universally, is also the reason it cannot
accomodate individual needs for each "asset type" (css, js):

* For Javascript we cannot use CommonJS (or something similar)

* For SCSS we cannot use variables, mixins

For me, this means a lot. Traditionally, Rails has been giving me the best web
environment experience I could imagine. With Rails 3.1, I can't help thinking:
Why did they leave some of the best parts out?

I know I actually CAN use SCSS variables and mixins - I just have to use SCSS'
@import rather than Sprockets own =require. But that leads to my stance in
this matter: If we need to bypass Sprockets in order to get the cool features,
then why use it in the first place?

We should use a system designed specifically for JS to management our JS, and
a system specifically for CSS to manage CSS. Stitch and SASS comes to mind of
course.

While I understand the benefits of the "Sprockets way", I'd like to see Rails
use specific tools for specific asset types in order to give developers easy
access to all of the available features instead of just 80% of them.

------
bronson
Didn't Ryan Dahl say he wish he used something else for packaging Node
modules?

(I don't know what he had in mind other than CommonJS and I don't remember
where I heard it so this is pretty hazy...)

~~~
telemachos
Yes, it's in this interview:

[http://bostinnovation.com/2011/01/31/node-js-
interview-4-que...](http://bostinnovation.com/2011/01/31/node-js-
interview-4-questions-with-creator-ryan-dahl/)

BostInno: Is there anything you wish you had done differently with node?

Dahl: Yes – many things. For example, I wish I had not used the CommonJS
module system. It’s far too complex and wildly different from how the browser
works.

In another interview (which I can't find), I recall him worrying a bit that
CommonJS is not async, though admitting that a few lapses in async purity
might be fine. In any case, I'm not aware that Node actually has any plans to
move away from CommonJS. At this point, a whole hell of a lot of code relies
on that style of _require_. Perhaps it could be swapped out in some backward
compatible way, but it seems like a lot of work. (Also from the point of view
of someone who just uses CommonJS style rather than implements it in something
like Node, I have to say, it's dead easy and intuitive. Why fix what isn't
broken?)

Wow, wtf? I got voted down for finding someone a reference? I'm not making an
anti- (or pro-) CommonJS argument here. I just found the reference.

~~~
tlrobinson
There's this huge misconception that because "require" synchronously returns
the module object it must use synchronous IO to load the modules as well,
which is not true. The root module and it's transitive dependencies can be
loaded asynchronously before any execution of the code (and thus require
calls) takes place.

To do that the dependency module IDs must be extracted from the module text
via static analysis, but that's fairly easy since module IDs are specified to
be string literals. A simple regex is usually good enough, but using one of
the many JavaScript parsers is safer. For cases where the module ID needs to
be computed or loaded based on a runtime decision, an asynchronous form of
require should be used.

The expectation is ECMAScripts modules will map pretty cleanly to CommonJS
modules, but with real syntax rather than function calls. In the meantime
CommonJS modules are a pretty good compromise I think. Simple implementations
can do the loading using synchronous IO as "require" is called, while more
advanced loaders (especially for browsers) can implement a slightly more
complicated system to do the loading up front before excution.

~~~
substack
> To do that the dependency module IDs must be extracted from the module text
> via static analysis, but that's fairly easy since module IDs are specified
> to be string literals. A simple regex is usually good enough, but using one
> of the many JavaScript parsers is safer.

I did exactly this for browserify, yet another node-based browser-side
require() bundling system. Another benefit of this approach is that you can
use modules made for node just by require()ing them in your browser-side
javascript, so long as your static analysis bundling is recursive and the
modules can be `require.resolve()`'d by node.

------
shubber
If only it were true. I started in on Sprockets to get my feet wet for Rails
3.1, and have to say I was disappointed. Sprockets is a textual concatenator,
which makes it appropriate when you're writing short snippets throughout a
project. By contrast, CommonJS (and requireJS and a slew of other
implementations of asynchronous modules) implements well scoped modules, which
have all kinds of boons.

------
exogen
"If you've ever use Node or Python you've used CommonJS modules"

Huh? Python, really?

~~~
joeyespo
I could be wrong, but I imagine he's referring to the module concept. Not
CommonJS itself. (Or perhaps referring to skulpt.org, but I doubt that.)

~~~
maccman
Yes, that's correct. Importing is similar to require(), although there's no
explicit exporting.

~~~
MBlume
setting module-level __all__ can modify what gets pulled in a wild import, but
that's about it.

~~~
masklinn
It's also used by `help()` (and pydoc): if there's an __all__, help will only
pull the docstrings of the stuff in __all__. `dir()`, on the other hand, does
not care.

------
bobfunk
Looks very cool, will play around with it. Seems like a big missed opportunity
that Sprockets doesn't speak CommonJS.

~~~
shubber
Sprockets, as it exists, can't do module loading: it's a Javascript
concatenator. All it does it copy the contents of <filename.js> in place of
//= require 'filename.js'.

Gotta, say, I just spent a chunk of time converting a project from one to the
other (actually RequireJS), and I really think it was the right move.

