
ESbuild – A fast JavaScript bundler and minifier in Go - todotask
https://github.com/evanw/esbuild/
======
constexpr
It's cool to wake up in the morning and see my project on HN! For the record,
I'm not trying to compete with the entire JavaScript ecosystem and create an
extremely flexible build system that can build anything.

I'm trying to create a build tool that a) works well for a given sweet spot of
use cases (bundling JavaScript, TypeScript, and maybe CSS) and b) resets the
expectations of the community for what it means for a JavaScript build tool to
be fast. Our current tools are way to slow in my opinion. Let's build faster
tools!

~~~
abetlen
Very cool project, do you know how this compares with Hugo's javascript
minifier / bundler?

~~~
earthboundkid
Hugo uses
[https://github.com/tdewolff/minify](https://github.com/tdewolff/minify)
internally, so that’s the relevant comparison.

------
alipang
I believe _something_ like this project is inevitable. You're probably
thinking "Where's the Rust version of this?". I'll save you a roundtrip to
Google that'll find you SWC. [1]

SWC is already more mature, at least more so than this project. Using rust
does also seem to have some advantages compared to Go.

Still, nothing wrong with some competition, just think it'll be pretty hard to
replace the _entire_ ecosystem around Typescript/Babel/Webpack in one go. Not
going to work for existing project - probably way too high of a risk for new
ones, sadly.

[1] [https://github.com/swc-project/swc](https://github.com/swc-project/swc)

~~~
constexpr
Author here. I think the Rust vs. Go question is interesting. I actually
originally wrote esbuild in Rust and Go, and Go was the clear winner.

The parser written in Go was both faster to compile and faster to execute than
the parser in Rust. The Go version compiled something like 100x faster than
Rust and ran at something around 10% faster (I forget the exact numbers,
sorry). Based on a profile, it looked like the Go version was faster because
GC happened on another thread while Rust had to run destructors on the same
thread.

The Rust version also had other problems. Many places in my code had switch
statements that branched over all AST nodes and in Rust that compiles to code
which uses stack space proportional to the total stack space used by all
branches instead of just the maximum stack space used by any one branch:
[https://github.com/rust-lang/rust/issues/34283](https://github.com/rust-
lang/rust/issues/34283). I believe the issue still isn't fixed. That meant
that the Rust version quickly overflowed the stack if you had many nested
JavaScript syntax constructs, which was easy to hit in large JavaScript files.
There were also random other issues such as Rust's floating-point number
parser not actually working in all cases: [https://github.com/rust-
lang/rust/issues/31407](https://github.com/rust-lang/rust/issues/31407). I
also had to spend a lot of time getting multi-threading to work in Rust with
all of the lifetime stuff. Go had none of these issues.

The Rust version probably could be made to work at an equivalent speed with
enough effort. But at a high-level, Go was much more enjoyable to work with.
This is a side project and it has to be fun for me to work on it. The Rust
version was actively un-fun for me, both because of all of the workarounds
that got in the way and because of the extremely slow compile times. Obviously
you can tell from the nature of this project that I value fast build times :)

~~~
tejinderss
Would you say go language is more fun to code in than rust ? How does it
compare with nodejs and Typescript in terms of developer productivity ?

~~~
jchw
Not them but my personal opinion:

\- Go is more fun if you are trying to be productive and push stuff out. The
programming experience feels fluid and there’s not much agonizing over small
details; the language is simple and you don’t need to think as much about
things.

\- Rust is more fun if you have a focus on perfection. It offers a lot of
tools for abstraction and meta programming. These tools can be challenging at
times. I do think even with NLL you will find yourself fighting the compiler,
trying for example to resolve how you can avoid overlapping a borrow with a
mutable borrow in some complicated bit of code, but you definitely get a lot
of nice guarantees in exchange. I also do find it frustrating when something
as simple as passing the result up can end up being really tricky.

~~~
littlestymaar
There's truth in what you say, but it only describes the learning phase of
Rust. Rust is quite challenging to learn and you spend a pretty long time in
the uncomfortable place you describe. But one day, you end up internalizing
the borrow checker's rules and you just don't think about it anymore and don't
have any productivity penalty at all.

~~~
jchw
I don’t know how long it takes to fully get past stumbling through borrow
checking and learning the intricacies of Result but it’s long enough to be a
detriment. Obviously learning curve on its own is a downside, but also this
complexity does not disappear when you understand it. It’s similar to, but
less severe than, C++, in this regard.

I also think it depends heavily on the type of program you are writing and
how. I’ve certainly hit cases where I still don’t know the optimal way of
structuring things. In other cases people have managed to help me figure out
what I need to do.

Concurrent memory safety is a huge plus, without a doubt, though there are
applications where its not enough and applications where its too much. I think
that puts Rust in a spot where it has use cases where it is clearly the best
option but many use cases where it is overkill. As an example, Go shines
particularly well for servers thanks to Goroutines and the fact that many
servers have a shared-nothing architecture these days.

~~~
littlestymaar
> I don’t know how long it takes to fully get past stumbling through borrow
> checking and learning the intricacies of Result but it’s long enough to be a
> detriment.

That's right, but no one expect to learn quantum physics in a few weeks
either. Rust is indeed way longer to learn than Python, Javascript & others,
but it's also much more powerful. And with the same level of power, both C and
C++ are way harder to master than Rust (and arguably, nobody really master
them in practice since even the most brilliant programmers shoot themselves in
the foot from time to time. Yes, even DJB [1].

> Rust in a spot where it has use cases where it is clearly the best option
> but many use cases where it is overkill.

Indeed. It would make no sense to switch to Rust if Python is good enough for
the task. I was just arguing that once you've learned Rust, you can do pretty
much anything you want with it without friction, and I personally wouldn't
bother writing anything in Python nowadays, because I can write Rust as fast
and get the static typing guarantees and sane error handling that comes with
it.

> Go shines particularly well for servers thanks to Goroutines and the fact
> that many servers have a shared-nothing architecture these days.

Having done quite a bit of Go, I don't agree with you. It's way too easy to
accidentally share data between goroutines, and then cause data race or
deadlocks . The day they introduced the race detector, we found 6 data races
in our code (a few thousand loc) and a few others in our dependencies, and in
the next year we found two not caught by the race detector (because it's a
dynamic thing, it can't catch all races). More than generics (which are being
introduced if they don't change their mind like they did for error handling)
Go really need something akin to _Send_ and _Sync_ in Rust. M,.or maybe like
Pony's capabilities system, but Go definitely needs improvement on that front.
Multithreading is a hard thing, and Go makes it too easy to use, to people
without the necessary experience (because Go is so easy it attracts a lot of
junior or self-taught devs) without safety net and this generally doesn't end
well.

[1]: [https://www.cvedetails.com/product/16058/D.j.bernstein-
Djbdn...](https://www.cvedetails.com/product/16058/D.j.bernstein-
Djbdns.html?vendor_id=9069)

~~~
jchw
\- A trade off is a trade off; no need to justify it. Increased cognitive load
is a con.

\- Sorry your Go experience was bad. I can only say my anecdotal experience
was the opposite. Mostly for shared nothing architectures, but I also worked
on an MQ-style software in Go and had a relatively good time. I think things
that are well-suited to CSP concurrency fare pretty OK. Rust could’ve
prevented things like accidental concurrent map accesses, but it _still_ can’t
guarantee you are implementing concurrency correctly on the application level
(from perspective of say, coherency or atomicity.) So for many apps I’ve
written, even somewhat complicated ones, I don’t feel like Rust would always
be the best option. To me Rust makes _most_ sense when you really can’t ever
afford a memory error. Web browsers seem like an obvious winning case.

------
_bxg1
Alas, there's such a massive ecosystem at this point around Webpack and Babel
and family (extra features like tree-shaking, loaders support for different
things, project configurations, etc.), that I think you'd be hard-pressed to
make an outright replacement take off.

What you _could_ do (and I'm surprised this hasn't been done yet) is lower
some of the hot paths of Webpack itself into a lower-level language. Possibly
even moreso than Webpack, Babel seems like a natural target since it has much
simpler input/output.

Although one barrier I can see with that (unless it's C/C++) is requiring
people to install a whole other build system just for it to be used in an npm
install. C/C++ compilers generally come with the system, but Go/Rust ones do
not.

Edit: further info [https://stackoverflow.com/questions/20728255/could-one-
write...](https://stackoverflow.com/questions/20728255/could-one-write-a-
native-node-js-extension-in-go-as-opposed-to-c#39262638)

~~~
phlakaton
[https://what-problem-does-it-
solve.com/webpack/interoperabil...](https://what-problem-does-it-
solve.com/webpack/interoperability.html) nailed head-on what I detest about
Webpack and many Javascript build solutions for that matter: the burning
desire to shove the kitchen sink of operations into a single tool with its
own, incompatible-with-everyone-else, plugin framework and an ungodly
configuration language. Where all I really wanted was a somewhat more
ergonomic Make and a bunch of well-written, decoupled tools to use with it.

This is on my back burner of ideas to run with one day, but as a back-end
engineer working in other langs, it hasn't quite bubbled up to the surface
yet.

~~~
zdragnar
Several of these exist and predate webpack. Webpack became popular precisely
because it _wasn 't_ grunt, gulp or npm run scripts.

~~~
phlakaton
And yet, look what it has become.

------
sunflowerdeath
Actually, webpack does a lot of additional stuff that might be needed for some
complex configurations, so it can't be easily replaced with this tool. But
also almost 75% of the time (in webpack) is spent inside the minimizing
plugin, and that is very simple task. So it would be great to have just the
minimize plugin based on this tool. Then maybe investigate what other
performance critical parts of the webpack can be replaced without requiring to
change the ecosystem.

------
q3k
Finally. The extreme slugishness of webpack&co has always bugged me any time I
ventured into JS development, to the point of me getting immediately
discouraged.

~~~
tmpz22
It’s a natural cycle though. New bundler created that is fast and simple to
use -> New features are requested and added to the bundler which makes it more
sluggish -> New bundler created that is fast and simple.

I wonder how this compares to ParcelJS.

I’m a big fan of Go but there are usually fringe benefits in having the
bundler written in the language that it is bundling I.e. JavaScript.

~~~
mtmail
Benchmark against parcel is in the documentation. 200x faster.

~~~
tmpz22
That’s exceptional, if the benchmarks test identical workloads and features I
would definitely consider switching to this.

FWIW it may sound like I’m hostile to this framework when I’m just trying to
be critical in a constructive way. I use golang as my main driver at work and
find most projects in Go to be better then its common counterparts, however I
have been through so many JS bundlers that I scratch my head when anyone
claims superior results on (Self-declared) hobby ones.

OP should know this seems really solid and like great work. I just don’t want
junior devs running head first putting this in production.

------
mariopt
If these numbers are true, it would be nice to have more documentation and
tutorials. This would help the javascript community a lot.

I built a gaming computer to work with javascript codebases, true story.

------
Gehinnn
I think if somebody replaces webpack with something that is 100x faster has
80% of webpacks most important features, many companys would pay real money
for it.

------
tyfon
I made my last webpage in 1999 using HTML tables and cut-up "design" jpgs so I
am a bit out of date in this area.

I see all these preprocessors and crap for "web apps" and I wonder why nobody
just writes HTML/CSS/Javascript instead of all of these other "languages" and
abstractions or is the base web language now so bad that nobody touches it
manually?

~~~
wruza
Desktop-to-web deserter here, I’ll try to sum it all up.

\- Browsers still have bad and ugly environments. You don’t have all features
everywhere, even in recent versions. Things like “class Foo { static x = new
this }”-level are missing, not some rare magic (like proper modules). Node.js
is a different beast, despite a [vaguely stated] promise to be
indistinguishable. Lots of env-neutral code written in node doesn’t work in a
browser as is. Think msvc vs gcc/c99.

\- CSS is just too low-level, people do like sass, less, etc very much. These
have “includes”, functions, variables, flow control, non-css inheritance, bem-
related isolation, common shorthands for a syntax. Modular apps cannot live
without that.

\- DOM is so flipping slow that it requires precise differential updates to it
if you don’t want to charge your battery four times a day. It is also much
less “cool” than desktop ui data-controls, and it requires a framework, which
usually comes from node_modules for reasons above (prebundled versions exist
for syntaxless fws). Also, too few people like to write a big single html
(again, no “includes” etc) and bind it to the code directly, so there is vue,
jsx, pug, etc. The more you abstract it away (dom, not html), the better it is
for everyone.

In terms of programming, if you want an analogy, pure browser feels like a
single qbasic file on steroids, while bundlers make it feel _almost_ like a
visual studio project (put here any decent ide if you don’t like vs).

~~~
tyfon
Thanks for this it makes it a bit more clear, but I'm now never going to touch
web again lol.

It sounds like hell to me.

However, what you describe above sounds like what you need to write a full
blown application. Why does one need this to display simple information like a
blog or news? Tasks that I used the HTML tables stuff for. Nowadays I can
enter a personal blog with maybe 100 lines of actual text but it still pulls
several megabytes of javascript.

I just can't wrap my head around why all this is necessary when all you want
is to display information that could be equally served by a text or html file.

~~~
grayed-down
I feel your disdain! I'm an old-timer and remember complaining about using
those new-fangled cascading style sheets instead of just putting everything in
html tag attributes.

And don't get me started about having been forced to use div's for layouts
instead of tables. In fact, I still don't understand why the html table
construct couldn't have been evolved to accommodate a layout role.

~~~
wruza
What is funny, both grid and flex still fail at some use cases. For one, grid
is unaware of “media print” and cannot paginate itself (that was a big
facepalm after an hour of preparing). It is also 10k rows/cols max by
definition, 1k max by a common implementation. Can’t style gaps too. Otoh,
flex is non-tabular, so you cannot simulate both with just one. I also met
some trouble with inner flex scrolling (“overflow-xy” hierarchy forwarding
issues), but these details I forget as fast as I meet them.

The entire css thing is overly situational. In a generalized model, any
problem is solved by a finite set of building blocks. In css, any problem
except few premodelled ones is a pile of hacks and worms that never get back
into a can.

------
vijaybritto
I was thinking that a parser and tokeniser distributed as wasm could increase
the build speed while maintaining the js part for customisation. But this is a
full fledged native bundler. Though less flexible it's incredibly fast!

------
shrumm
This is really cool - our dashboard uses Vue for the frontend and a Go
backend. Everytime I deploy, I think about how insanely slow the frontend
'compilation' \+ unit tests step is compared to testing and building the Go
code. It's one of the things that makes Go a joy to code in for me.

On a similar note, we have some services written in Python and I was doing
some research for some static type checking tools. At least in my testing,
Pyright (written in Node) outperforms the pants out of similar tools written
in Python. Using Go to bundle + minify Javascript sounds perfectly reasonable
to me.

------
tzury
reading the go code is a true pleasure and great resource to study the
language further.

------
globular-toast
I've been using parcel because it's the only thing I know and I always thought
it was slow but it claims to be fast and I'm new to js so I took it's word for
it. According to this it's the slowest of the lot, though. Why does it claim
to be "blazing fast"?

~~~
constexpr
Yes this was a surprise for me too. The benchmark I used for esbuild
deliberately tests full build time without a cache, since that is an important
metric to me. I assume Parcel is talking about incremental build times when
the cache has already been populated. In that case I'm sure Parcel is not as
slow.

------
truth_seeker
> Parsing, printing, and source map generation are all fully parallelized

I think this the major cause of speedup. Now that NodeJS supports
WorkerThreads, it can be done in parallel. Along with this if right choice of
data structures used there is no need to depend upon writing a build manager
in another language instead of host language.

Also, as of today JS(V8) performance is getting closer to Go runtime.

[https://benchmarksgame-
team.pages.debian.net/benchmarksgame/...](https://benchmarksgame-
team.pages.debian.net/benchmarksgame/fastest/go-node.html)

~~~
mappu
_> I think this the major cause of speedup._

This simply can't be true. On a 4 core laptop you might expect parallelism to
provide up to 4x performance. But this has 75x / 200x performance, that must
come partly from other reasons.

~~~
truth_seeker
First of all you didn't even care to read I also mentioned that choosing the
right data structures also. ;)

You are deluded by the benchmark posted too. I terms of different build
process steps that library isn't doing as much as other js build frameworks.

~~~
mappu
I understood your phrase "the major cause" to mean majority reason, that
couldn't be true since the remaining difference is greater. If you meant
something different than that, then I misunderstood you. We both agree that
the remainder of the performance (the bulk of it) must come from other reasons
than parallelism.

As well as your suggestions (doing less work, choosing the right data
structures) - I would also propose (A) no V8 JIT warmup time (B) no need for
fstat on many node_modules files (C) fewer AST passes (D) possibility of value
struct types => less boxing => lower GC pressure (E) less dynamism/no plugin
architecture.

------
saddam96
Superb project. Did you try comparing against Packem. We leverage Rust as a
backend for the bundling and resolving phases:

[https://packem.github.io](https://packem.github.io)

------
dcre
Nice work! Building from scratch without any caching is not the build time
devs will be seeing 99% of the time, so it’s probably not the best benchmark.

~~~
boulos
How fast is an incremental rebuild with webpack though? That esbuild finishes
from scratch in .5s (though probably with the code previously loaded from
disk) means it’ll still likely be the fastest.

Posters up thread claim that minifying is the slowest part of webpack, so
presumably that’s pretty consistently slow relative to input size.

~~~
dcre
You don’t need to minify during development, so like building from scratch I’m
not sure how much it matters for a practical benchmark.

I don’t at all doubt esbuild is faster than the others, just in practice I
doubt it’s as much faster as the benchmark suggests.

------
throwawaygo
Yass. This is a really well written go codebase. Go read it and see what life
was like before the framework kids showed up.

~~~
christophilus
Eh. I’ve been working in the industry since before the framework kids showed
up. I’ve seen plenty of truly awful C++, C#, and COBOL codebases.

------
shirshak55
how does this compare with SWC made with rust?

------
rienbdj
Go is such a strange choice of language for this kind of thing compared to
something in the ML family.

~~~
earthboundkid
Yes, think of all the great tools written in Haskell, like pandoc and uh… a
second example someone will reply to me with that no one has ever fucking
heard of.

~~~
throwaway8941
Nah, the second one is also pretty popular.

[https://www.shellcheck.net](https://www.shellcheck.net)

~~~
earthboundkid
Well, I can say I have heard of it. It still strikes me as a pretty much
trivial tool compared to the bazillion things written in Rust and Go in the
last five years, but it’s better than nothing.

------
drej
Can we drop the "in Go" in links like these? I don't really see how that adds
any value to the end user.

~~~
tylerchr
The thing that caught my attention about this post was specifically that it‘s
a JavaScript bundler _not_ written in JavaScript. The “in Go” in this case
made the difference between my noticing and skipping a very interesting
submission, so I think it’s relevant.

------
wadkar
X in Go seems to be the new Y in Rust.

I am curious to know what is the benefit of doing this in Go?

I thought that the reason why webpack is complex is because the whole
JavaScript ecosystem is complex with respect to the ES20XX standards. I fail
to understand how rewriting the logic in another language is going to reduce
the complexity.

Speed is important, sure, but certainly not at the cost of correctness!

~~~
tmpz22
I think X in Go preceded Y in Rust by several years.

In short Go is a simple language with minimal syntax and explicit design
patterns that make it very easy to read and understand once you’ve learned the
ropes. It’s a typed and compiled language with all the benefits that entails,
with great concurrency support.

It’s standard library is fantastic, compile times are great (one of the
original goals for Go was to reduce c++ compile times at Google), with out of
the box support for networked/web applications on the standard library.

Finally the community for Go is very strong with things like the gophers slack
channel which is insanely popular and friendly.

~~~
wadkar
Haha, yes.

Thanks, I’ll certainly check out the gophers community.

As the sibling points out babel/webpack are like compilers of JavaScript world
so reducing compile times have a huge impact on developer experience.

I am just wondering where does the complexity of juggling various ES*
standards go? Also, the webpack plugins and loads of config examples might
require this project to support additional complexity. Will the compiled/typed
nature of Golang make it difficult to correctly maintain parity with
babel/webpack?

I am really looking forward to learning golang this year. I am almost done
~~suffering~~ learning JavaScript/React + CSS3/media-queries this winter.

~~~
tmpz22
Browsers lag so hard behind the ES standard that most developers are using
shims provided within Babel or Typescript to use the features early. Normally
you specify whatever ESNext features your using in your tsconfig,
webpackconfig, babelrc, etc.

I'm not sure if the implementations are always formalized such as the shim you
use is identical to whatever implementation that major browsers end up going
with.

Ultimately you just end up bloating your bundle because you need added code in
your shims for backwards compatibility of these advanced features.

