Hacker News new | past | comments | ask | show | jobs | submit login

First of all: Thank you, yarn, for helping the community see the naked emperor. Deterministic builds by default are such an obvious (in retrospect) core requirement.

Couple questions:

Question 1: Does anyone else who's been around more than a couple years share my view that Yarn : NPM :: IO.JS : Node?

IOW: healthy competition, catalyst for necessary change, ultimately a bridge or stopgap.

Question 2: Any good comprehensive writeups on best practices?Committing lockfiles is a no-brainer. But what about globals? Per-node-version seems logical, but there are also semantic problems with "global" packages vs concept of executable binaries. I'd love to see a strong writeup outlining and defending a standard approach.

When io.js merged back into node.js the following happened:

1. Node.js was abandoned and replaced with io.js.

2. Io.js was relabelled as node.js.

3. The node.js project was put under open governance via the creation of the Node Foundation.

This doesn't really compare to npm:

* The npm-cli's name is using npm Inc's trademark: giving it away would leave the company with no name or create unwanted ambiguity between npm Inc the company and npm-cli the (then) community-owned open source project unaffiliated with the company.

* Yarn does not share any source code directly with npm-cli, it's a complete reimplementation of a similar feature set. It's currently backed by a mirror of the npm registry but that's the extent of the overlap.

* The Node Foundation already exists and Yarn is already owned by the community. I could see Yarn joining the Node Foundation and the Node Foundation replacing its bundled dependency on the commercially owned npm-cli but even then npm Inc has nothing to contribute to the process.

Personally I would much rather see Yarn join the JavaScript Foundation because it speaks to the broader appeal of Yarn outside the traditional "Node community" and the politics involved in the latter and its close ties to npm Inc.

FWIW I consider the close ties between the Node project and npm Inc nothing more than a historical accident that has resulted in a conflict of interest that is overdue to be resolved by dropping npm-cli from the official releases.

Now that yarn is no longer distributed using npm (although it's still possible to install it that way) that seems more realistic than ever.

What replaces the npm registry, though? That's the reason that npm Inc. exists, not just stewardship of the CLI.

A decentralized repository of packages that anyone can help out maintain. Not entirely sure how, but I think that would be the ideal scenario.

Back when the npm registry was really struggling with reliability (I believe because it was secondary to profit for Joyent) I was saying we needed a decentralized registry solution.

But right when that got some upvotes on r/node, within a few days isaacs had npm inc. going. And within a few weeks or months, the npm registry was pretty rock solid. And I have not had a single registry issue in forever.

And npm 4 works well, and I'm sure npm@5 works even better.

I still think it makes sense to have a decentralized repository and I guess its nice to have a non-commercial alternative to npm. But for me its not important anymore because npm is working really great now.

That's great to hear!

Reliability sure was tough in those days. npm wasn't secondary or tertiary or any-ary to Joyent. It was my nights and weekends project the whole time I was there, and IrisCouch generously donated infrastructure, which we pushed to its very limit once Nodejitsu acquired them.

Something like IPFS could perhaps be used for this. I think the biggest issue is probably access control. I.e. just releasing packages addressed by content isn't enough, because people generally depend on @whatever/left-pad and not 5b055a0b42f1c or whatever the hash may be. Real estate in a global name space means someone gets there first, so you'll need to consider how to deal with name squatters and the likes. With a centralized source like npm it's as easy as getting in touch with support (they're very good) but when no one knows the name space everyone owns the name space – makes it hard to make things "nice".

So I guess what I'm saying is that just storage isn't the tricky bit to distribute really, but having a global namespace is I guess.

well, you could depend on left-pad@5b055a0b42f1c, and the name doesn't really matter. Normally you would include that as require('left-pad') but if you have multiple versions, I don't see why require('left-pad@5b055a0b42f1c') couldn't be used. There is a tool for doing this in Golang already[0] and I've also done some experiments to get this to work in JavaScript[1]

- [0] https://github.com/whyrusleeping/gx

- [1] http://everythingstays.com

Go does something like this, by referencing packages through Git url. Of course, 99% of packages you install are through Github, but that's not Go's fault. It has downsides, but its really a great and extensible system.

You can do the same thing with npm and yarn as well by referencing the git+https URL and a tag/branch/commit instead of a version number.

Yarn does not run a mirror of the registry. registry.yarnpkg.com is a pass-through domain to the npm registry. It allows them to collect stats about yarn usage but is not a mirror.

>resolved by dropping npm-cli from the official releases.

But that would break 7 years of documentation and tutorials, likely bad for newcomers and thus the ecosystem

so npm has created a huge vendor lock-in and we should live with it?!

"The only reason God could create the world in six days is, He didn't have to worry about backward compatibility."


As with any vendor lock-in situation, decisions should be made based on cost/benefit

Perhaps you can help come up with a strategy that gracefully handles breakages and eventually results in dropping npm-cli

Not to mention quite a few CI build pipelines.

FWIW it's trivial to install yarn already: https://yarnpkg.com/en/docs/install

NPM currently doesn't provide a standalone distribution as far as I can tell, but presumably they would offer one if they no longer had the luxury of being bundled with Node.

Since changes generally don't happen overnight I would expect a transitional period where Node still bundles npm-cli but npm Inc has the time to prepare a standalone distribution before yarn replaces npm-cli. Additionally downstream channels could decide to provide legacy packages containing both.

They probably need to do something like docker did.

Answer 1: Yes but they're not exactly analogous and I don't see them merging. Yarn is a complete rewrite, and the compatibility "surface area" is much lower than Node itself so there's not much pressure for them to merge. Yarn has been working essentially flawlessly for me, so npm-cli is really going to have to surpass yarn in some way for me to switch back.

Answer 2: I don't use global packages often. For binaries I generally add a "scripts" entry to package.json for common tasks, which automatically adds "./node_modules/.bin" to the PATH. For one-off tasks I just run "./node_modules/.bin/executable"

Git dependencies with semver support seems to pretty much leapfrog yarn. One of the few reasons for a private npm package is removed with this feature.

It's more accurate to say that npm v5 : npm :: io.js : node.

Same team, performing a big refactoring and significant improvement and modernization of some bits that were outdated and difficult to improve without a controlled demolition and rebuild.

The analogy breaks down, of course, because the governance didn't change, the project's relationship to its corporate backer wasn't standing in the way of progress, etc. (Except, I guess, that it might have gone faster if we'd had more money to hire more devs? But some of this was just a slog through a very old codebase with a lot of scar tissue, and it's hard to speed that up by putting more hands on it.)

Everything in npm 5 was literally planned years in advance. When we have this many people depending on a thing, we have to be careful about how we make drastic changes. Yarn was a strong signal from the community that we were on the right track, but it only seems like a "catalyst" when seen from the outside. Correlation is not causation, even when it lands first.

Thanks Isaac! Both for this insight and - more so - for your massive contributions to the community. Slainte! :)

"Deterministic builds by default are such an obvious (in retrospect) core requirement."

I agree... I like the theoretical concept that our dependencies, could be automatically managed/upgraded behind the scenes via a declarative versioning system, like semver. That, in my view, was the paradigm behind not having defaulted to dependency graph snapshot (lock/shrinkwrap) by default. In practice, people don't follow semver correctly and autoupdating patch+minor versions can cause you a lot of pain. It would be cool to make versioning more tied to actual API signature, so that changes to the signature could not actually be published as minor/patch changes... but would be forced to use a major version bump in publish. I don't know how this would work particularly in a language like javacscript w/out a typed interface... but I'm just theorizing...

As @pluma mentioned, a "merge" seems unlikely (for one thing they don't share much, if any, code).

However, npm could certainly compete with more of yarn's features (fast local cache[0], shorter syntax for running scripts like `npm start`, etc)

[0] turns out they did that: https://twitter.com/maybekatz/status/865393382260056064 wow!

Yeah, I think that's a good comparison, except the upsides of yarn seemed much more obvious to the community than was the case with IOJS. There seemed a lot less controversy than with IOJS.

What a time those days were, feels like a million years ago already.

npm would need to add the --flat option for it to work for me

> Thank you, yarn, for helping the community see the naked emperor. Deterministic builds by default are such an obvious (in retrospect) core requirement.

https://docs.npmjs.com/cli/shrinkwrap provides deterministic builds and has been around far longer than yarn. Since Oct 2014 npm v3 started automatically updating shrinkwrap whenever '--save' was used. See https://github.com/npm/npm/pull/4918#issuecomment-61344871

Calling npm 'the naked emperor' for not implementing something it did, infact, implement is uncalled for.

Edit: added links in response to unexplained downmods. The parent is simply, provably wrong.

Even with shrinkwrap, npm install is not deterministic, install order still matters. To get a deterministic install, you need to use shrinkwrap and do an `rm -rf node_modules` before every install. And it's still not completely deterministic if any dependency or sub-dependency has optionalDependencies.


That, and the fact that it's not the default behavior in npm gives yarn a pretty big advantage in my opinion. Especially for devs who are newer to Node.js.

I've read the https://docs.npmjs.com/how-npm-works/npm3-nondet but it make no mention of shrinkwrap. Since shrinkwrap captures versions for the entire tree, install order wouldn't matter.

I personally had invalid dependencies generated with shrinkwrap which did not work with a "npm install" after and had to edit the file manually so I would say it's not yet there.

Shrinkwrap is a partially successful attempt to solve the problem, but still has obvious flaws. For example, instead of the relatively neat version specifications normally used in package.json, with a shrinkwrapped project you wind up with https://registry.npmjs.org/... paths instead, but only sometimes. This gets messy if you want to both install stable versions of all your packages and use a local registry to supply them for reliability, which is not an unlikely combination.

Applications are open for YC Winter 2020

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact