Esbuild looks amazing. It seems to have been written almost exclusively by one guy (evanw) during 2020. Just the main JS parser file [0] is 12k lines. According to the Github stats he has committed 280k lines, that's almost 1000 lines per day every day since he started. Amazingly productive.
While that is impressive, I think this is also an indication of a problem: the grammar is becoming unwieldy. For example, even for someone as prolific as Evan, he must decide between feature depth (e.g. bundle splitting) and feature breadth (e.g. implementing the equivalent to @babel/preset-flow, which is used by both flow and hegel[0] type systems)[1]
Esbuild supporting Typescript and JSX is undoubtedly a byproduct of these grammar extensions having become popular. But supporting extra grammar extensions does add to complexity, sometimes in non-trivial ways. In Typescript, for example, you can import types using runtime import syntax, making it ambiguous whether running side-effects from a library is intentional or not. This gets problematic once you consider treeshaking and not-really-standard things like conditional resolution via package.json's `browser` field.
It gets even more fun when you realize that module resolution isn't even specified, meaning that as far as a bundler is concerned, something like `import 'lodash'` means completely different things for a browser vs a project installed via npm vs one installed via yarn v2...
> It gets even more fun when you realize that module resolution isn't even specified, meaning that as far as a bundler is concerned, something like `import 'lodash'` means completely different things for a browser vs a project installed via npm vs one installed via yarn v2...
Ugh, why does it have to be different between npm and yarn v2?
I'm an occasional JavaScripter and yetserday wrote an automation with Puppeteer.
The first line I wrote was
import puppeteer from 'puppeteer'
only for Node to fail on that. I thought that syntaxt was supported, but I had to change it to
const puppeteer = require('puppeteer');
I've no idea if it's a problem with my Node version (12) or what, but it was a real "Huh?" moment.
FWIW, that's not an `npm` vs `yarn` thing. That's a "Node has only recently added support for ES module syntax and it's still mostly experimental" thing:
It's an odd thing. The way people talk, you'd think NodeJS and JavaScript are synonymous. Turns out, NodeJS is this generation's JScript, including shoddy support for the language proper, and it includes vendor-specific extensions that you're expected to use instead. It has somehow though managed to escape any of the criticism that gets thrown at other projects pilloried for stuff like this.
Node came out way before any of these features were invented, and the ES committee knowingly broke compatibility when designing for the browser. Modules are supported since v14 but you have to opt in.
They didn't break compatibility. NodeJS never attempted to be compatible with the Web. The design work that TC-39 and the browser makers did is also much better than what the NodeJS community puts out. And it's a design that was in ECMAScript 6. Six years later, and you still have to opt in. The TypeScript team got it right, working in a way that NodeJS should have been taken lessons from.
Yeah, technically better, at the cost of huge complexity.
CommonJS was easy to implement anywhere and dead simple. We only need static analyzers and tree-shaking optimizations because the JS ecosystem went wild on dependencies. I honestly think we were better off 10 years ago.
> NodeJS never attempted to be compatible with the Web
It goes the other way around? Node caved in and support modules now, what has been done in tc-39 for web modules to be compatible with node (considering it predates all of it)?
Modules weren't production-grade until Node version 14. Before that, it used CommonJS modules ("require" instead of "import").
It's because Node was invented before EcmaScript modules existed. CommonJS worked well and supporting both was awkward, so the transition took a while.
> that's almost 1000 lines per day every day since he started. Amazingly productive.
I know what you mean, but I also hate using lines of code per day to measure productiveness, and I hate it when managers in corporations look at number of lines added in PRs to measure how productive a dev is.
I don't know about anybody else but for me a good approximation is the time interval between the first line of code and the critical line of code. The better my solution the shorter it often is. In particular, 'shorter' in the sense that future functionality doesn't have to constantly acknowledge that code, creating a tax on new features that builds until the project dies.
I was going to say "start of coding" but some tasks are just so boring that I may procrastinate on starting it. That also indicates an element of difficulty (in giving a shit), but not what we're trying to measure here.
I discovered the Mikado Method quite on my own as a consequence of trying to tackle increasingly complex cases of 'large-scale' or 'top-down' refactoring before accepting that as the oxyest of morons. I was sold when I realized that when you finally discover the crux of the problem, the 'Rome' that all roads lead to, between half and 80% of the code you just wrote isn't strictly necessary. Throwing it all out wholesale is how you avoid code hoarding. Yes, once in a while you go out to the proverbial trash can to pull something back out that you actually did need, but the tabula rasa aspect is profoundly useful psychologically, especially for that 'tax' bit I mentioned above.
There is always some function in the middle of the call tree, that if you add or change the meaning of an argument then the feature or bug fix becomes a logical conclusion of that new semantic. Either the code above it or below it hardly needs to change, decoupling the feature from all but a handful of functions/concerns.
A bad programmer will happily plow through adding a new argument to fifteen method calls and then propagating that change to a hundred call sites. And that code will be buggy as hell because their coworkers have already written them off. If it's too much code for others to review, you can be sure there are bugs in it. And if you didn't listen the last ten times people told you to stop writing so goddamned much code, you aren't going to listen this time, either. So have at it, sport. We'll just make sure you get the blame for the bugs, until someone in management wakes up. If they don't, then they see that the sections of code people "won't touch" but you will are getting bigger, and mistake your ownership of this code for a sign of prowess instead of a sign that you are inspiring apathy in others.
Just to put the 280k lines into perspective, deletion rate seems close to addition rate[0]; measuring the number of changes made to code is also a poor proxy for productivity, but is at least not quite as bad as the pure "lines added" metric.
Not to be the one who is the one, but 1k lines per day sounds like the opposite of productive. If you still have to add/remove such large swaths of code, you're surely doing something weird.
Still a great achievement to produce a bundler and minifier as a one-person team, but not sure we should use the amount of code line changes as a measure for productivity.
Productivity might be the wrong word. Discipline, grit, perseverance?
I've written parsers/lexers/code generators by hand for much smaller toy languages with no thought about performance at all and those were huge undertakings for me. I was just trying to convey my impressedness with some numbers that we can relate to somewhat.
But yeah, he has a "net" loc contribution of 284-148 = 136k lines of code. Assuming the code base is resonable (which I don't doubt), it's a pretty big project which he has built, which I'm impressed by.
If you read the linked page, it was a very intentional decision:
> Everything in esbuild is written from scratch.
> There are a lot of performance benefits with writing everything yourself instead of using 3rd-party libraries. You can have performance in mind from the beginning, you can make sure everything uses consistent data structures to avoid expensive conversions, and you can make wide architectural changes whenever necessary. The drawback is of course that it's a lot of work.
> For example, many bundlers use the official TypeScript compiler as a parser. But it was built to serve the goals of the TypeScript compiler team and they do not have performance as a top priority. Their code makes pretty heavy use of megamorphic object shapes and unnecessary dynamic property accesses (both well-known JavaScript speed bumps). And the TypeScript parser appears to still run the type checker even when type checking is disabled. None of these are an issue with esbuild's custom TypeScript parser.
1k lines per day sounds pretty reasonable when writing your own TypeScript parser, among other components
> 1k lines per day sounds like the opposite of productive
Having built a compiler before, this seems normal, especially since the parser is written manually (as opposed to generated). Parsers are always a lot of lines of code.
Not all of them are the most complex, and in fact a lot will just be defining all the different nodes of your AST and branching on them, which, as blocks do, tend to inflate code a lot. I wouldn't expect a JavaScript parser to be any more compact though, or the development (especially in a codebase with few contributors) to be any less "thrashy".
I don't think that's even up for debate. As far as I'm concerned they have definitely met their goal.
I find it similar to the fact that, while I am thankful that the operator's manual for my toaster was written at a 4th grade reading level, I find it unlikely that anyone has that task in mind when they say they want to be a writer when they grow up.
I think Go is a genuinely fantastic experiment testing the idea that it's actually easier to write and modify long, boiler-platy, but painfully explicit code.
Yup, that's the one! He's also the author of http://madebyevan.com/. I remember seeing that WebGL water experiment many years ago and being blown away. That FSM Designer came in handy for some classes where I had to draw FSMs too.
SWC, another competitor project written in rust is also handled exclusively by a single person. Half a million lines of code and they have even built a type checker for typescript which is not included.
[0]: https://github.com/evanw/esbuild/blob/master/internal/js_par...