
Why does a new Rails app need 106 MB of JavaScript? - rileytg
https://gist.github.com/rileytg/6f2a1a8d982cbcee8cdcab37bfe344bb
======
schneidmaster
Rails 6 by default includes webpack to bundle assets for production. Aside
from webpack, several of the top 10 on your list are libraries that ensure a
great user & developer experience across browsers:

\- babel transpiles ES6/7+ JavaScript syntax to the least-common-denominator
of the browsers you designate as supported by your application

\- core-js provides implementations of features that are not supported in
older browsers -- things like array methods that cannot simply be transpiled

\- caniuse-lite is a database of supported browser features to determine what
needs to be transpiled and when

This toolchain allows developers to write code with the latest language
features (which usually results in simpler, more elegant code) while
seamlessly supporting legacy browsers as needed. If you decide to drop support
for an older browser, you can simply remove it from your browserslist instead
of going through all your code to figure out which hacks you put in just for
that browser. It takes a ton of cycles out of development googling "can I use
feature X in browser Y".

Prior to Rails 6, asset precompilation was powered by Sprockets by default.
Sprockets still bundled many analogous libraries, just hidden in ruby gems
instead of node_modules. For example, if you want to use SASS then you need a
library that compiles SASS to CSS -- whether it's node-sass on your list or
the sassc gem.

As noted in other comments, very little of this code is actually loaded in the
frontend by end users.

~~~
userbinator
Regardless of whether you think all those components are necessary, there is
still the question of whether the amount of space they require is justified;
JavaScript is a high-level language, and in fact one of the higher-level ones
in comparison to things like C/C++. 1KB of JS should be able to do far more
than 1KB of C or even binary.

Thus, 100MB of JS is _astoundingly huge_. It's roughly the same size as _all
the binaries in a full install of Windows 95_ , with all its features and
drivers.

In the spirit of
[https://prog21.dadgum.com/116.html](https://prog21.dadgum.com/116.html)
perhaps we need a list of "Things an empty Rails app is smaller than".

 _It takes a ton of cycles out of development googling "can I use feature X in
browser Y"._

...or you could just avoid the trendchasing and use a minimal subset that you
are sure will work everywhere, but that's a different discussion.

~~~
lacker
At $50 for a 1TB hard drive, that 100MB is costing you half a cent. It doesn’t
get sent to users over the network so it doesn’t affect performance. Why does
that matter again?

~~~
Arnt
106MB of source code is perhaps five million lines of code, if written in a
typical way. Some people use longer lines, not so many "{" lines, but anyway.

Storing five million lines of code is cheap, sure, no question. But don't you
feel curious about how why you need that much code in the first place?

------
bhhaskin
Looks like it is just the asset pipeline, so not anything that is loaded on
the frontend.

Little bit click baity. A lot of other frameworks do the same thing.

~~~
svnpenn
> A lot of other frameworks do the same thing.

God that is an awful justification.

The heading is the opposite of clickbait. 106 MB is not exaggerated, the link
provides proof showing that number. I get pretty tired of people handwaving
away bloat these days. Its a real problem across programming languages and
communities, and it doesnt just go away by pretending it doesnt exist. It goes
away by people testing for it, making it a priority for software to be
streamlined and pruning bloat where it makes sense.

~~~
thrwwy90876
It's build tools. This is like complaining that Xcode is many gigabytes.

~~~
tedunangst
I never had to install a new copy of xcode for every project.

~~~
pistoriusp
So the problem is not the size, but that you want to have a single cache of
all the npm modules?

------
AnIdiotOnTheNet
For reference, a full install of Windows 95 is about 50MB.

There is something deeply wrong with this entire industry.

~~~
crazygringo
But those are compiled binaries, not source code.

I guarantee you the source code for Windows 95 is _vastly_ larger than the 106
MB we're talking about for Rails.

And considering the ridiculously large number of things Rails does that
Windows 95 didn't... the sheer number of different technologies, standards,
formats and versions it's able to interface with... I don't see what's wrong
at all.

~~~
userbinator
_I guarantee you the source code for Windows 95 is vastly larger than the 106
MB we 're talking about for Rails._

I wouldn't be so sure. Win95 has a lot of pieces written in Asm (source >
binary), mostly the low-level parts, but also a lot of C/C++ (binary >
source).

~~~
crazygringo
I mean, I know that for a small "hello world", you'll get an executable much
larger than the source code.

But with a decent-sized program, full of header files and comments and config
files and descriptive variable names and all the rest, there quickly comes a
point where the executable is vastly smaller, no?

I really can't believe that the C/C++ source code for Windows (or Word or
Photoshop or whatever) winds up compiling to a _larger_ file size, can it?

~~~
flqn
It really depends. Small amount of source code doesn't necessarily mean a
small binary if you have a ton of template instantiations.

------
pistoriusp
Right, what I love about this is that it completely ignores the installed gems
because they're installed somewhere else?

    
    
      $ bundle --path vendor
      $ du -h -d 1 .
      64M ./vendor
    

After building the rails site you can delete node_modules because you only
need the built bundles. Those shouldn't be very big at all.

~~~
rileytg
When building container images, yes. However in heroku I'm not sure how to do
this without custom buildpacks (e.g.
[https://elements.heroku.com/buildpacks/istrategylabs/heroku-...](https://elements.heroku.com/buildpacks/istrategylabs/heroku-
buildpack-node-cleanup)). Custom buildpacks reduce the value add of heroku for
my small team.

Ultimately my solution will likely be build the assets in CI and deliver via
CDN or check into the repo on some release branch.

Edit: I intentionally omitted the gem sizes as that was a prior cost of using
rails

~~~
pistoriusp
I feel your pain, we also had this problem, and resorted to exactly the thing
you've done.

Personally I always felt that this was a shortcoming with Heroku's Buildpacks.
Which seem unwilling to support modern rails.

------
straws
Webpack-based projects are heavy. A Rails app comes with a Webpack-based
Javascript scaffold out-of-the-box.

I wonder if something like Rollup or NCC could be used to create a single-file
Javascript dependency in the general case that is only used for vanilla
Webpacker builds, and relying on the individual packages could be something
left for when you need to customize the Webpacker build.

Judging by some of the Hey developments, it's possible we could see a much
more lightweight JS integration in Rails in the future:

[https://twitter.com/sstephenson/status/1272608117604397063](https://twitter.com/sstephenson/status/1272608117604397063)

~~~
rileytg
I've worked on the whole loop of ¡server side! to ¡client side! and now back
to ¡server side!. Using Hey for a little, I'm really impressed with the "feel"
of it. I really hope they will write about how they do this and even
contribute stuff upstream.

Going back to primarily server side was about long term projects that aren't
very active. I have a rails app I wrote in 2011, I occasionally make fixes,
update gems etc. No stress.

I have a client side app from 2016 that I eventually gave up on trying to get
the build system running again (grunt?). I just edited the minified build and
told the customer this was the last major edit. Only copy changes now.

This isn't an isolated incident. I frequently have to relearn an old project
b/c everything changes so fast in js. First we were on prototype, then we went
to jQuery (2011?). Homegrown UI components lacked the browser support we
needed (IE5/6?) so we added jQuery UI. Our code was a spaghetti mess without
components, so eventually we moved to React (2015?).

Those changes took 10+ years. The rest is all in the last 2.5-3.5 years. And
personally, I find it way more complicated than previous js ecosystem changes.
Ok now we really want to use modern js so lets add a transpiler. Now our
components state is getting out of hand. Lets use reflux. Oh no dont use
reflux, use redux. Oh but now you need redux-forms. Remember that build system
we had? Now everyone uses grunt. Oh no, everyone uses webpack now. Are you
still using redux? Use react hooks. And while I value this ecosystem, I'm
nervous that we wont "solve" our problems and just keep writing new partial
solutions.

It's a bit story-telly (rant?) but the gist is Rails w/ ssr doesnt come with
huge mental cost over time. Client side can. We still love and use the client
side apps, just only for complex "apps" like an interactive document viewer
where we are actively developing year over year.

------
benatkin
That can be answered in one phrase, coined by Django:

> Batteries included.

It's ultimately up to Rails how they want to do things. It's OK to complain
about it but it's a shame when it discourages people from creating and
maintaining open source projects.

~~~
dllthomas
Other use in the Python ecosystem seems to predate Django
([https://github.com/python/peps/commit/53474baa0790912b8a02f1...](https://github.com/python/peps/commit/53474baa0790912b8a02f1468efd782e90dd09eb))
although I'd certainly be interested in more context.

~~~
benatkin
Yeah I wonder if they arrived at the phrase independently. I think Jacob
Kaplan-Moss was inspired by Python's standard library.

It seems the owner of that PEP could probably provide more context. He wrote
"Meditations on the Zen of Python" recently which means he's still thinking
about pythonic software design:

[https://orbifold.xyz/zen-of-python.html](https://orbifold.xyz/zen-of-
python.html)

------
cocktailpeanuts
Webpack.

I always appreciated the design philosophy behind rails, how it was always
about "getting things done", and not about being the latest cool tech of the
day, but this introduction of the webpack fad into the "rails convention"
feels just like going backwards.

------
rileytg
FWIW 106mb will fit ~11 million chars.

I use many of the features I get with rails webpack. I also use create-react-
apps (which I think is even bigger). I think they are valuable to the
development process.

I would think we can achieve most if not all of these goals with less code. I
don't know exactly how we would get there or if there are any projects
underway. This seems like a problem that can't be fixed in one place but must
be addressed holistically. This being a decentralized ecosystem of packages,
the only way I can see that happening is by publishing stats and creating a
culture of reduction. For example: When contributing to the one of the
presumably hundreds if not thousands of packages that roll up into CRA or
rails, you should see if that overall increases or decreases size. You may
have introduced a huge tree of new dependencies thinking your adding one
package to your tiny package.

Just a rough idea, does someone more experienced in the community have ideas
on what can be or is being done?

~~~
schneidmaster
I think there are two primary reasons why JavaScript projects tend to have
much larger dependency trees than in ruby or other ecosystems.

1\. The JavaScript standard library is much more sparse than what you get out-
of-the-box with ruby or many other languages. The existence of lodash is
evidence of this in itself. The infamous left-pad fiasco would never have
happened if JavaScript's native String had an equivalent to ruby's
String#rjust. This part of the problem is significantly improving with time --
for example, node.js 8.0 and up do have a native string pad function,
padStart. However, especially for tooling libraries, using the latest native
JS features means dropping support for semi-recent versions of node, and
there's also generally a lot of inertia.

2\. npm and yarn allow you to install multiple versions of the same package.
If I have dependencies A and B, which both require different major versions of
dependency C, npm and yarn will both happily install multiple copies of C.
With ruby/bundler this is an error and you have to work out how to tweak your
dependencies so they're all happy with the same version range. I suspect this
architectural choice is largely a consequence of #1 -- especially in the
earlier days of npm, getting all your dependencies on the same version of a
commonly used helper library would have been nightmarish. And there are
benefits to this approach; I have certainly spent long hours trying to upgrade
ruby gems with version conflicts. But it results in a lot more bloat almost by
default.

To answer your original question -- I don't know that anything in particular
is intentionally being done. I do think we're heading in the right direction
and things will be much better in a few years as packages slowly move towards
new native language features rather than helper libraries. I also just don't
think this is of critical importance at the end of the day. If you add up the
brain cycles that I've ever spent worrying about 100MB of hard drive space,
it's almost certainly less than what I've spent typing this comment :)

------
ankurpatel
This should also be done for create-react-app

------
mwkaufma
"It's not frontend" isn't an answer. Why should a text-packager clock in at
100mb?

~~~
isochronous
Because modern web development is a friggin' minefield of browser support
matrices, front-end development languages with experimental features, and
aging specifications that require the use of preprocessors for simple things
like variables and functions. Almost everything in that 106 MB is a
preprocessor of some type that's designed to either make developer's lives
easier, normalize code to a widely-supported standard, and/or optimize said
code to deliver the smallest possible bundle to end users.

~~~
mwkaufma
Why can't it be 50mb? Why shouldn't it be 200mb?

~~~
isochronous
I don't understand the point you're trying to make. That's just how much disk
space those tools and their dependencies take. You might as well ask why iron
is as dense as it is.

~~~
mwkaufma
Web-pack is not an elementary particle. It's a man-made construction.
Describing its disk-size as an intrinsic aspect is a categorical mistake.

Webpack is no-doubt a big program with a lot of features. But I have several
even-bigger programs on my PC that clock in at just a few mb - so saying it
has a lot of features doesn't answer OP's original question, it simply
substitutes an easier question and then answers with a "the size is what the
size is" tautology.

When a native app links a static library, it performs dead-code elimination.
It might have hundreds of megabytes of dependencies, but the overwhelming
majority is not copied to the application. When I make a new rails project I
don't download the text-source of Firefox or MySQL and all their downstream
dependencies into every project root. There's no reason webpack has to do
this, except because of arbitrary workflows imposed by NPM.

If no user-complaint about NPM is ever quantified or interrogated, then NPM
will fade into irrelevancy.

------
onychomys
I just tried it for a new Angular app. I chose to include Angular Routing
(because everybody will need that) and then to use CSS as my style sheet
because hey why not. End result was a folder that was 273MB in size, with the
node modules being basically all of it.

------
Scarbutt
I guess he hasn't see the size of a create-react-app project.

------
thatguyagain
Can't you just pass "\--no-webpacker" to the "rails new" command to skip the
npm packages? I haven't tried it myself.

~~~
rileytg
Yes. Makes a huge difference. I try to stick with defaults as much as I can. I
find that helps with long term maintainability. In this case, I don't know if
I will (as I chose in the past to always use pg and rspec).

------
mattmcknight
node_modules is insane. JavaScript needs an STL. Then we need an AI to rewrite
all of these modules to use the STL.

------
inglor
Basically several compilers are bundled (node sass, webpack and Babel) so that
you don't have to download them as a user and their version is stable for the
app. You can use a tool like pnpm to only ever have one copy of them on your
system but these aren't deps that negativity impact the app.

------
mihaifm
Great fan of Rails but man, that’s a lot of bloat. Last time I used Rails it
only came with jQuery. Live-reloading was an exotic thing, quite difficult to
setup. Now it seems to be enabled by default, explaining some of the stuff in
there.

~~~
isochronous
It's also 99% server-side preprocessors. Only a single package in that list
(lodash) is something that could be actually loaded via browser, and even that
would only be a small percentage of the full package size. It's also not
required for a new rails project, it's simply a convenience workflow that
integrates many of the tools used in modern best practices.

~~~
rileytg
Your point is valid that a large portion is server side only (which is still a
concern for my use case). But note a lot is omitted from my gist, there are
768 packages.

