
Speedy Web Compiler – A Rust port of Babel and Closure Compiler - kn8
https://github.com/swc-project/swc
======
losvedir
This is really cool. I understand why all the front end tooling is written in
JS but it really is a pain to deal with in my experience, especially when it
comes to building and deploying. Since they're essentially just scripts you
have to also install the right versions of node and npm and use npm to install
the dependencies (and watch out for native extensions!). It's fairly tedious
to get it all right, whenever I try it, since I'm not a pure frontend dev, and
things have always changed since the last time I did it. But the promise of
this is just dropping in a binary and having it all work!

I actually like JS as a programming language for the web, I just don't like
scripting languages in general for command line tools.

~~~
chrisseaton
> Since they're essentially just scripts

What do you think is the essential difference between a 'script' and a
'program'?

~~~
8note
I kinda think of libraries vs scripts:

the library implements how to do the lower level -- which things you can do,
and takes care of doing it

and the script deals with which things you want to do and when

some examples:

I made some opencv bindings for JS, and the library changes did the heavy
image processing algorthims, and the JS script chose which ones to use with
what parameters

for some video game or another, there was a library for what an NPC can do,
and scripts for how they act within the world, including things that are more
like movie scripts detailing whow says what adn where they walk

~~~
pvorb
So if I wrote a Java program that uses OpenCV that program would be a Java-
Script?

------
alangpierce
I've been working on a very similar tool, interesting to see some competition.
:-)

[https://github.com/alangpierce/sucrase](https://github.com/alangpierce/sucrase)

In my case, it's still running as JS, but rearchitected to solve the more
straightforward problem where you don't need to compile to ES5. I've thought
about rewriting it in Rust to see how much faster it gets, though currently
I'm trying to get it running in WebAssembly via AssemblyScript (
[https://github.com/AssemblyScript/assemblyscript](https://github.com/AssemblyScript/assemblyscript)
), which has been promising.

I'm curious about the about the Babel performance comparison. Benchmarking JS
is tricky because, from my observations, the performance improves by a factor
of ~20 if you give it 5-10 seconds for the JIT to fully optimize everything.
[https://github.com/alangpierce/sucrase/issues/216](https://github.com/alangpierce/sucrase/issues/216)
. Rust and WebAssembly both have a significant advantage in that sense when
running on small datasets.

~~~
natural219
Hey man, I just started using sucrase in production at work. Great work with
it; just the right features to let me kill Babel, simplify build docs, and
still get use some modern niceties (I use it alongside gulp/rollup)

~~~
alangpierce
Great to hear. :-)

Feedback/questions always welcome, of course! Feel free to file an issue or
chat in gitter. Fortunately it's a pretty well-scoped project that's mostly
done at this point.

------
rattray
Where are the test cases?

Babel's parser alone has quite a formidable suite of fixtures (a few thousand
as I recall): [https://github.com/babel/babel/tree/master/packages/babel-
pa...](https://github.com/babel/babel/tree/master/packages/babel-
parser/test/fixtures)

The nice thing about implementing a JS-JS compiler is that there is already a
huge number of tests out there, like these, that you can use to TDD your
compiler's edge-cases.

SWC's CONTRIBUTING mentions this:

> Include tests that cover all non-trivial code. The existing tests in test/
> provide templates on how to test swc's behavior in a sandbox-environment.
> The internal crate testing provides a vast amount of helpers to minimize
> boilerplate. See [testing/lib.rs] for an introduction to writing tests.

But I cannot find a `/test` directory, and it does not appear in the
.gitignore either.

EDIT: Ah, the tests are in `/tests.rs` with JS embedded inside rust code. This
makes it a bit harder to compare directly with babel's suite, but claims to be
lifted from test262, which babel also based many of their tests on (albeit
recategorized). hzoo's comment here has details:
[https://news.ycombinator.com/item?id=18746905](https://news.ycombinator.com/item?id=18746905)

~~~
steveklabnik
[https://news.ycombinator.com/item?id=18746754](https://news.ycombinator.com/item?id=18746754)

[https://news.ycombinator.com/item?id=18746905](https://news.ycombinator.com/item?id=18746905)

~~~
rattray
Thanks Steve! Missed those; race condition perhaps. Updated.

------
ajross
Seems like transcompilation is an area where you really DON'T want be
overfocusing on performance. Correctness is king here. I mean, compiler bugs
are hard. Really hard. Deep voodoo hard. No one who's ever dealt with a
significant code generation bug on a large project ever wants to repeat that
experience.

That's not to say that babel isn't too slow or that this project isn't
correct, just that it would scare me to try to work with it. Does babel have a
regression suite? Does this pass it?

~~~
empath75
If you want bug free code, rust is as good a choice as any to write it in.

~~~
nkcmr
What part of rust prevents the writing of bugs? I understand it helps you
avoid unsafe memory operations, but to say it gives you “bug free” code seems
like an falsehood.

~~~
eiurafhlfie
Tagged union types (called `enum` in Rust), recursion, and pattern matching
are great for tree data structures, such as ASTs, and help catch many bugs.
This is the reason why so many compilers, typecheckers, and interpreters are
written in languages like OCaml, F#, Haskell, and Rust.

~~~
n-gate
How many non-toy non-self compilers and interpreters are written in Rust?

~~~
steveklabnik
There’s at least a few; cranelift is powering stuff in production at Fastly.
Facebook is writing a MIR interpreter to do static analysis, don’t think
that’s production yet though.

------
kmlx
for me these kinds of tools are only as good as the time spent debugging when
something goes awry. as such, the biggest risk i see is that it isn't built in
javascript. i simply don't see myself debugging rust while trying to compile
javascript. i really don't.

this being said, i do wish this project (and any others like it) great
success. we definitely do need something faster when compiling js.

(this tool reminds me of a similar project for a small client. they required a
fast js compiler. the project was a failure because no-one wanted to debug
non-js code when the build failed)

~~~
sam0x17
This is why I'd really like to see some projects that compile to readable js
when not in production mode. Throw in some auto-generated comments above every
method that tell you the file and line number in the Rust/Go/Crystal/Whatever
code that this comes from, and debugging will be fairly easy.

------
rattray
Perhaps the best thing about Babel is its plugin-based architecture. As of
Babel 7, even its parser accepts plugins.

This not only makes it easy to keep up to the latest of ECMA's standards and
extensions like JSX, Flow, TypeScript – it also makes it surprisingly easy to
implement your own syntax extensions to JS (eg;
[https://wcjohnson.github.io/lightscript/](https://wcjohnson.github.io/lightscript/),
which I have worked on).

Curious how this project is thinking about plugins and staying up-to-date
sustainably.

~~~
mintplant
> As of Babel 7, even its parser accepts plugins.

Does it really? Last time I looked into this, the "parser plugins" were all
fake: single-line modules which simply switched on hidden config flags in
Babel to enable functionality already hard-coded into the core parser. Left a
bad taste in my mouth that the developers weren't up-front about the true
(non-)extensibility of the system.

------
PascalW
There's also Pax [1], also written in Rust. Pax however is only a bundler, I
don't think it actually transpiles code.

[1] [https://github.com/nathan/pax](https://github.com/nathan/pax)

------
calineczka
How is it so much faster exactly? Is it due to parallelization or being
written in a compiled language or something entirely different?

~~~
steveklabnik
I don’t know much about how Babel is implemented, but this uses the excellent
rayon crate, which makes data parallelism awesome and easy. Rust is also
generally fast, especially when you put effort into performance, which (from
the creator talking about this on reddit) was a big goal.

This is the kind of thing Rust should be excellent at.

~~~
purple_ducks
> which makes data parallelism awesome and easy

There's still going to be a finite number of cores on a machine. I can't see
gaining parallelism being that big a boost (as opposed to gaining
concurrency(if not already there))

~~~
steveklabnik
I would expect this task to be CPU bound, not IO bound. Maybe that’s
incorrect.

------
JeremyBanks
Closure Compiler is a poorly specified pile of edge cases and bugs.
(Embarassingly poor quality for something Google relies on so much.) Is this
attempting to support its type system, minification moded, or what? I'd be
impressed if it managed a significant subset.

------
rattray
Those interested in fast JS parsers may also be interested in cherow –
[https://github.com/cherow/cherow](https://github.com/cherow/cherow) – though
it does not provide compilation.

------
LinaLauneBaer
I never had any problems with Babel's performance to be honest...

Also a quick check with google trends does seem to support that feeling.

[https://trends.google.com/trends/explore?cat=31&date=all&q=S...](https://trends.google.com/trends/explore?cat=31&date=all&q=Swift%20slow,Babel%20slow)

Swift is probably used a lot less than Babel but it yields a lot more
spikes...

------
pookeh
Honestly I don't think Babel is the performance bottleneck in most dev setups.
Most slow frontend dev setups are because of webpack and it's core
architecture. Even the ts compiler is quite fast.

~~~
alangpierce
I've looked into dev tooling performance quite a bit and I think your comment
is half-true. Babel is just part of the performance story, and in some
situations, speeding up Babel doesn't help at all.

One complication is the fact that Babel results are often cached, so with a
reliable cache, Babel performance barely matters at all. However, every
caching approach I've seen has reliability issues in practice. Certainly a
tool is more pleasant to use without a cache, since "maybe I have a bad cache"
is never a worry when running into strange behavior.

It also depends a lot on your scale, how you compile your code, and what
operations you want to be fast. When switching to Sucrase (my own Babel
alternative project, mentioned elsewhere) on a large codebase, webpack startup
went from 40 seconds to 20 seconds, though incremental builds weren't any
different. Running tests (in node with a require hook) became much faster,
from minutes to seconds when uncached and about a 2x speedup when Babel pulled
from cache. In both of these cases, the remaining time is now in other things:
webpack processing the files, node running the imports, etc. So there's a lot
more besides Babel, but Babel still is a non-trivial component in the cases
I've run into.

------
rattray
Does this work with JSX and TypeScript/Flow? These language extensions, when
enabled, have nontrivial performance impact.

On that note, they don't say what plugins/presets were used in the Babel
comparison. Was it the same set of features that are supported in swc?

~~~
hzoo
Yeah I've had a difficult time figuring out how this was tested:

I would recommend using the benchmark at [https://github.com/v8/web-tooling-
benchmark](https://github.com/v8/web-tooling-benchmark) in the future if it's
currently not possible since currently it only seems to be testing ES5
(jquery.min.js) for the parser and a short ES6 snippet for the transforms.

~~~
rattray
(I think when you say "tested" you mean "benchmarked" – your comment at
[https://news.ycombinator.com/item?id=18746905](https://news.ycombinator.com/item?id=18746905)
gave a nice link to how it is tested).

------
Solar19
How is it Rust project if it's installed via Node?

~~~
steveklabnik
You can put anything you want in an npm package.

