
ECMAScript modules in Node.js: the new plan - ingve
http://2ality.com/2018/12/nodejs-esm-phases.html
======
apo
Meanwhile, Deno (alternative server-side JS built from scratch by the creator
of Node) keeps chugging along:

[https://github.com/denoland/deno](https://github.com/denoland/deno)

It will be interesting to see which happens first: developers migrate to Deno
or Node finalizes its ES module implementation.

I think both groups are doing excellent work, but the Node team seems to be at
a disadvantage having to continue support for some decisions made early on in
Node:

[https://www.youtube.com/watch?v=M3BM9TB-8yA&vl=en](https://www.youtube.com/watch?v=M3BM9TB-8yA&vl=en)

~~~
settler4
> import { test } from
> "[https://unpkg.com/deno_testing@0.0.5/testing.ts";](https://unpkg.com/deno_testing@0.0.5/testing.ts";)

This feels both very pragmatic and frightening at the same time.

~~~
Ataraxy
I'm not an "expert" but that feels just as insane as the npm argument people
make. I'd love to hear from someone more in the know as to why they aren't the
same.

~~~
mmmeff
They really aren't if you think about it. Going straight to a URL for a
version of a dependency is the same as pulling it from a registry, except it's
decentralized from a single source (NPM) and removes the extra hops in between
the package vendor and the package consumer.

On the flip side, that extra hop adds a ton of convenience in the form of
name-resolution, security and governance. It's the age old double-edged sword
of centralization.

~~~
oscargrouch
Maybe if you turn that url into a hash, than just use the hash to check if the
package has a local copy already, it wont be so bad. But you will need to add
the package version in the URL so that you know you will always have the
package you really want in your local cache.

------
russellbeattie
Let me state some things which I think are true:

1) A vast majority of developers hate '.mjs'.

2) In 5 years, most developers will have moved to import statements and
stopped using CommonJS' require().

3) Importing from a directory using index.js is both common in popular
projects, and a useful way to organize code like components.

4) We've already been waiting years for a standard solution and many of us are
getting annoyed at the delays.

5) With transpilers the norm nowadays, it seems NodeJS could make a clean
break from the past and simply provide tools to help convert older projects.

Am I assuming too much?

~~~
BillinghamJ
To be honest I don't mind `.mjs` - we renamed everything in our system to that
about a year ago, but I do think it's important for it to be consistent with
browser standards.

The index.js thing is nice, but I think consistency with browsers is more
important and I'd be willing to lose it.

~~~
JimDabell
> I do think it's important for it to be consistent with browser standards.

There's no browser standard about .mjs. In fact the web doesn't really notice
file extensions; it goes by media types and attributes. A browser knows that
something is JavaScript because it is served with the application/javascript
media type (or similar), and it knows it is a module because the <script>
element has a type=module attribute.

A lot of the talk about "browser equivalence" is extremely loosely worded and
leads people to think that .mjs is a browser thing, but it's not. It's a Node
thing. What they want is for code to work the same way whether it's running on
Node or running in a browser. The problem they face is that Node hasn't got
the type attribute to decide whether something is a module or not, so they
can't use the same mechanism the browser has.

The solution they have devised is to assume that .mjs is a JavaScript module
and .js is not. The consequence is that if you want the same code to work in
Node and on the web, you've got to change what happens _on the web_ to follow
the _Node_ idea of naming things .mjs. Otherwise when you import a module
named .js – which works in the browser – it will fail in Node. So this isn't
about making Node match what happens in the browser, it's about making what
happens in the browser match the Node approach, which is not a standard in any
sense.

~~~
BillinghamJ
Yes that's my point. As a Node dev, I have no problem changing my extensions.
However the browser environment should not be pushed to change and what works
on the browser should largely work with Node.

------
s_tec
I wonder why they kept the ugly .mjs extension. This is a hold-over from an
earlier plan, and doesn't seem necessary in this iteration.

This new plan seem to be going down the path that `import` is for modules
while `require` is for everything else. If that's true, then they shouldn't
need a special file extension for modules - it should always be clear from the
context which is which.

The only exceptions I can think of are executable scripts (which could use a
command-line option) and packages in node_modules (which can use a special
package.json field).

~~~
FactolSarin
They say that it's required for ecsmascript modules that neither import nor
export (for instance, modules that place things in the global namespace), but
I would argue those are mostly shims for a pre-moudule era and they should
just be `required()`. Or perhaps they could use some special comment at the
top to let Node know it should be treated as a module even though it doesn't
look like one.

Otherwise, just work down the `import` tree from index.js and anything that is
imported is a escmascript module, everything that is required() isn't. Of
course, this is just an armchair quarterback opinion, but I'm curious to know
why that approach wouldn't work.

~~~
s_tec
Exactly! You have to know whether something is a script or a module before you
can parse it, but with this plan, that's always clear from the context.

This is exactly how browsers do it, by the way. If you `import` something, is
a module, period. The `<script>` tag is the only place that supports both, so
they use a `module="true"` flag to indicate which is which.

~~~
jkrems
> This is exactly how browsers do it, by the way. If you `import` something,
> is a module, period. The `<script>` tag is the only place that supports
> both, so they use a `module="true"` flag to indicate which is which.

This is not entirely true. If you import something and it's served with a
content type associated with JavaScript modules, then it's interpreted as a JS
module. But if they are served with a wasm content-type, they may in the
future interpreted as wasm modules. In node there's the additional content-
type of "CommonJS file" which has to be handled somehow as well.

Non-module script tags aren't really relevant because node itself never
supported scripts (at least not what the browser and the ECMA standard calls
scripts).

------
tokyodude
I'm sure there are valid reasons but I disagree with the extension solution.
That is not in any way meeting the stated goal of

> One of the goals of the Modules Team is “browser equivalence”

There are already tons of browser libraries not using the .mjs extension.

Not clear to me why just using 'import' makes it a module and 'require' not.
I'm sure that's spelled out somewhere

~~~
snek
>There are already tons of browser libraries not using the .mjs extension.

Browsers don't care what extension you serve something as, so we can't really
directly match that in the default behaviour. What you _can_ do is use a
resolve hook to tell node to interpret certain files as esm even when they
don't have a `.mjs` extension.

>Not clear to me why just using 'import' makes it a module and 'require' not.

import doesn't "make something a module". if you try to require something that
resolves as being esm (rn we check that with .mjs) it will throw, because esm
resolution is async and require resolution is sync. You can't require esm but
you can import anything.

~~~
tokyodude
Why does import have to work with cjs? It doesn't in the browser. Why does
require have to work esm?

Why not just have a package up it's major semver and those packages that want
to start using it need to import it? That seems no different than the browser.
AFAIK you can't import a non-module and you can't non-import a module in the
browser or am I wrong?

~~~
snek
>Why does import have to work with cjs? It doesn't in the browser. Why does
require have to work esm?

`import` doesn't _have_ to work with cjs, but we want to do so because there
is so much code in the ecosystem that is cjs.

`require` won't and can't work with esm, because esm resolution is async and
require resolution is sync.

~~~
ricardobeat
Let this be solved in userland through compilers, like everyone already does
today. The stubbornness from current node maintainers on this subject is
astounding. Years of feedback, like you see in this thread, ‘but-what-if’ and
‘you-dont-get-it’ handwaved away.

------
through
The Michael Jackson Script is an absolute joke and had arisen solely from
NodeJS attempting to solve its own require legacy. It’s not the business of
any other party to do such a ridiculous thing.

.mjs is an aberration, a solution to a poor implementation of one web server,
not a solution. NodeJS need to do the hard work of solving their own problem
and stop attempting to manufacture “real world solutions” with .mjs.

~~~
snek
If you feel you can come up with a better solution for module syntax ambiguity
please feel free.

Over the last three years we (including hundreds of devs like you who come in
guns blazing calling us all idiots) haven't been able to come up with
something that works better.

~~~
through
Proposing or fixing a bug in a third party product should not affect a web
standard. If you want my personal help to assist the NodeJS fix, yeah, I would
be good there, but I can’t fix the toxic bikeshed culture that you obviously
enjoy. MJS is NodeJS solution. It is not a web solution.

~~~
through
You could support .es file types which would work just as well and is more
succinct than .mjs, but AFAIK the group won’t because of “branding” issues.
Bull. Use .es. Standard semantics for ECMSScript which is the name of the
language, no games or pretence.

------
bryanrasmussen
Since I've worked on International Standardization before I sort of know how
the sausage is made, but you always hope for a better sausage.

Anyway - browser equivalence? That, in a lot of things, especially where URLS
is concerned, means HTTP equivalence.

A file extension having a specific meaning is not HTTP equivalence or browser
equivalence, it is at best Windows OS equivalence where not having mime types
understanding meant you had to use the file extensions to identify what 'type'
of thing something was.

A hack solution that was maybe necessary (I don't know Windows internals
enough to disagree) and that caused me a lot of problems in the years
1999-2009 approximately.

~~~
snek
Browsers don't care what extension you serve something as, so it doesn't break
the interop we want with browsers if you put your files as `.mjs`.

~~~
bryanrasmussen
my point exactly, browsers care what mime type you use and the extension is
meaningless. I thought I was pretty clear on this matter?

~~~
BillinghamJ
Both old "script"-style JS and the new module kind have the same MIME anyway

------
chmln
Instead of forcing this joke of a file extension, it would be far better to
make a breaking change in a semver major release and do away with commonjs.

There is no usecase for having two module syntaxes at the same time. All the
big projects already use babel or typescript. The community would catch up
eventually.

~~~
through
I completely agree. As I pointed out below, following the path you suggested,
if the filetype has to change to fix the require regression, they could use
.es. Would it be surprising to use .es with ECMAScript? No.

~~~
chmln
Requiring users to adopt a new extension is a massive change and as an end-
user I have nothing to gain from this - I'm already using babel and typescript
along with a huge part of the community.

If this creates regressions like you said, let users deal with their code that
deletes things from frozen objects. Allowing bad behavior to continue and
stall the module evolution is questionable at best.

------
nobleach
I think it can be a bit difficult for users of compile-to-JS setups to
understand why this is better. As of right now, one can just `import foo from
'./bar';` and the compiler just makes it work. This has become fairly tried
and true. What might be confusing is that node's internals, and perhaps their
design goals do not align with this paradigm. While I'm certain that they have
reasons, it's going to be awfully hard to convince developers. (It already has
been, look at the backlash on `.mjs`.) The solution? Continue to use
TypeScript/Webpack/ESMloader forever. I personally like the semantics of
importing the way it currently works. If it never works like that natively?
Fine.

~~~
snek
The issue is that transpilers didn't implement correct ESM (both ts and babel
have opt in flags to enable more-correct-mode, but its still not 100%). they
implemented the syntax but forgot about the actual semantics like dependency
graph verification and such. Node is an actual implementation of js and so we
must perform these tasks, even while transpilers don't.

~~~
efdee
Honest question: Does anybody actually care that it doesn't?

~~~
snek
I don't think its as much a matter of people caring or not as it is that node
is implementing ecma262 esm, not babel esm or typescript esm, etc.

~~~
nobleach
What happens if people seem to prefer the babel-esm semantics? I'm not saying
that babel's semantics ARE better, but what if developers prefer them?

~~~
snek
then they can continue using them, and not use the real esm semantics? i don't
think anything would change for them.

------
snek
This is a pretty good breakdown but to the author: i would like to see a
better explanation of defaults. Node will ship with the defaults needed to
perform tasks logically and unambiguously. This however doesn't mean that a
user's use case is locked away. ESM in node has hooks which allow people to
override resolution behaviour, which means you could put esm in `.esm` files,
or cjs in `.sadness` files, or have all files in a specific directory be esm,
or read package.json and pull metadata out of it, etc.

~~~
styfle
Is this documented anywhere?

------
jagthebeetle
I'm not strongly opinionated or informed about this, so why does Node need
file extensions when Other languages don't?

Is it because of non-JS imports? I know that when Ryan dahl announced Deno, he
mentioned it as an _obviously_ good change, but, I'm not sure what the
problems with extension-less imports are.

(Edit for context: at work, we either use Closure or put everything under a
global root, and only import JS, so maybe I've been shielded from the state of
the world.)

~~~
snek
> why does Node need file extensions when Other languages don't?

Node allows you to require things that aren't commonjs javascript. (json,
native extensions, etc).

> when Ryan dahl announced Deno, he mentioned [removing extensionless
> resolution] as an obviously good change

I think a lot of people disagree with ryan dahl in this case (I certainly do).
The ability to have a dependency expose an interface without knowing what type
of thing that dependency is is very powerful, and i think it is one of the
reasons people like require so much.

~~~
ng12
> The ability to have a dependency expose an interface without knowing what
> type of thing that dependency is is very powerful

Is it? I honestly can't say I've actually used that feature -- although I can
say I have gotten burned by it by accidentally importing the wrong thing.

------
ianwalter
I appreciate the work going into this but the timeline is a bit disappointing.

------
writepub
Even with new features, node chooses to do things differently. Take worker
threads for instance. Node added worker support much later than the browser,
and chose to mysteriously use an API that's significantly different.

I wish interop with the browser becomes more Central for upcoming features.
Where the browser is lacking, node can supplement, but to supplant those APIs
is just inconvenient

------
keithwhor
Is there any reason we can't just have a `'use esm';` declaration at the top
of a file instead of a janky filename extension?

It's just as arbitrary, but there's precedence with `'use strict';`, and
instead of renaming all of my files I just prepend them appropriately.

~~~
jdd
The `esm` loader allows specifying a parse goal pragma as one of several ways
to disambiguate source – [https://npmjs.com/esm](https://npmjs.com/esm).

Node will likely end up needing more than just an extension to disambiguate as
well.

------
keepingscore
Did I read that correctly we are going to lose the ability to require paths
and modules eg import {} from './services' or import {} from 'lodash'?

~~~
jkrems
First: none of the exact rules are written yet. But current discussions seem
to suggest "Yes, this will go away" for "./services" being resolved to
anything but an absolute location ending in "/services". But importing so-
called bare specifiers ("lodash") will almost definitely be supported.

~~~
snek
> current discussions seem to suggest "Yes, this will go away" for
> "./services" being resolved to anything but an absolute location ending in
> "/services"

to be clear, this is still heavily in contention. node made a good choice to
support requiring something without knowing exactly what type of thing it is,
and a lot of people feel we shouldn't throw that away.

~~~
BigJono
Can you give an actual use case for this? Because it seems insane. How can you
use something without knowing what it is?

------
strictnein
from [https://nodejs.org/api/esm.html](https://nodejs.org/api/esm.html)

> For now, only modules using the file: protocol can be loaded.

Is it the plan to allow modules to be loaded from any protocol? Because that
seems dangerous, from a security viewpoint. I've been trying to find mention
of anything that mentions SRI support, or something similar, but I haven't
found it yet.

------
shurley
No one will use this Interoperability hack in light of the fact that Rollup
and Webpack perform this for you seamlessly:

    
    
        import {createRequireFromPath as createRequire} from 'module';
        import {fileURLToPath as fromPath} from 'url';
        const require = createRequire(fromPath(import.meta.url));
        const cjsModule = require('./cjs-module.js');
    

Bundlers will not likely be able to produce efficient static code for that
bizarre interop code.

------
z3t4
can a script using es6 modules start with something besides a import statement
? then it would be easy to know if to use es5 or es6 module syntax. why was
the es6 module system made to break the es5 module system ? why not just use
es5 modules ? why is es5 modules so slow? are es6 modules faster ?

------
pvorb
Can anybody explain to me why ES didn't simply adopt the require syntax from
Node.js?

~~~
yuchi
Import statements must be asynchronous in the latency-controlled environments
(mostly browsers). CJS is by definition synchronous and has different order
semantics when treating execution of an asynchronous loader.

~~~
pvorb
But it could simply be implemented using the same routine in browsers like
`import`. I don't see why `require` can't be asynchronous. There's an implicit
join after the last import. Maybe it would have been enough to restrict calls
to `require` to the top of a file.

------
donateee
babel/register: [https://babeljs.io/docs/en/babel-
register](https://babeljs.io/docs/en/babel-register)

------
revskill
Webpack with css, less, sass assets with React as a SSR library. What's the
current solution now ?

