The fact that this is possible with NPM seems really dangerous. The author unpublished (erm, "liberated") over 250 NPM modules, making those global names (e.g. "map", "alert", "iframe", "subscription", etc) available for anyone to register and replace with any code they wish.
Since these libs are now baked into various package.json configuration files (some with 10s of thousands of installs per month, "left-pad" with 2.5M/month), meaning a malicious actor could publish a new patch version bump (for every major and minor version combination) of these libs and ship whatever they want to future npm builds. Because most package.json configs use the "^1.0.1" caret convention (and npm --save defaults to this mode), the vast majority of future installs could grab the malicious version.
@seldo Is there a plan to address this? If I'm understanding this right, it seems pretty scary :|
So we need gpg signed packages :> And... all packages should be namespaced under the author who published them. And... I kind of want to say "once it's published, it's forever".
> And... I kind of want to say "once it's published, it's forever".
This is effectively the norm with more traditional, curated package managers. Say I release a piece of open source software, and some Linux distro adds it to their package manager. Under a typical open source license, I have no legal right to ask them to stop distributing it. They can just say "sorry, you licensed this code to us under X license and we're distributing it under those terms. Removing it would break our users' systems, so we won't do it."
The difference is that NPM is self-service - publishers add packages themselves, and NPM has chosen to also provide a self-service option to remove packages. I honestly wouldn't have a problem with them removing that option, and only allowing packages to be removed by contacting support with a good reason. (Accidental private info disclosure, copyright violation, severe security bug, etc.)
I honestly wouldn't have a problem with them removing that option, and only
allowing packages to be removed by contacting support with a good reason.
(Accidental private info disclosure, copyright violation, severe security
bug, etc.)
Even Rust's Cargo won't allow you to revoke secrets [1]. I think this is the correct policy.
Aside from secrets there is also sensitive data. If someone accidentally uploads some personal information, they need a way to remove it if, say, they receive a court order ordering them to remove it.
If they receive a court order, and there is no technical way to do that, then the court is out of luck. "A court might order it in the future" is not a design constraint on your decisions today.
This combined with the cost of hosting (I remember the ruby community freaking out over rubygems costs a couple years ago) makes me think maybe we're evolving towards decentralized dependency hosting. Something like Storj where users offset hosting fees with blockchain payments when dependencies are fetched.
There's nothing preventing decentralization with npm now; it's a matter of configuration. Tying the namespace to a host seems more like instant excessive coupling.
Tying namespaces to a hostname isn't really that controversial -- it's no different than email.
If you want to be your own provider then host your packages on your server(s) and tell your users to add npm.cooldev.me/packagename to their configuration.
If you don't want to host your own then you can choose from a few public providers like npmjs but then have to be subject to their guidelines, policies, and fees.
Throw in some automatic bittorrent support in the client to help offload costs and you've got something great.
npm already supports all of that except the bittorrent bit, with the proper configuration, and without requiring that idiosyncratic namespace convention. [0] I don't think bittorrent is actually relevant to most use cases. Most people complaining here just don't want their site to go down, so they should vendor or fork all their deps and run their own registry to support that. Downstream users of public modules can either go through npmjs or perform the same vendoring and forking work themselves.
There have been links to child porn in the Bitcoin blockchain. To date, this has not resulted in any courts preventing full nodes from running in the US.
"So what you're saying is, your computers cannot possibly not continue damaging the plaintiff's interests." "That's correct." "You're being honest with me." "Yes, your Honor." "Will the computers continue harming the plaintiff's interests if shut off?" "... That would be dreadfully inconvenient, your Honor." "Do you have a more convenient solution?" "No, your Honor." "You are hereby ordered to turn off your computers in 48 hours." "... You can't do that." "I can do a lot of things, including jailing you if you disobey my lawful authority. 48 hours."
Engineers often think that they are the first people in history to have thought "Hey, wouldn't it be easy to pull one over on the legal system?" This is, in fact, quite routine. The legal system interprets attempts to route around it as damage and responds to damage with overwhelming force.
What Patrick says is technically true. But before granting the "extraordinary remedy" of an injunction, U.S. courts would apply the traditional four-factor test, which includes assessing:
+ the balance of hardships between allowing the conduct in question to continue vs. issuing the injunction;
+ whether the damage being caused by the conduct in question could be satisfactorily remedied by a payment of money as opposed to a mandate or a prohibition; and
It's not about whether the removal is logistically possible, it's about whether a court can punish someone for failing to carry out the removal.
Even when the former is actually impossible, a court could still punish for the latter. "Ha ha ha I use technology to cleverly show how futile your orders are" is not the kind of thing you want to say to a court with broad contempt powers.
The court can't punish you for not being able to do the impossible. That's ludicrous. "We have shut down all of our servers, yes. We can't stop people from downloading this, no"
It only doesn't work like that in the context of safe harbor laws.
If the safe harbor law protection doesn't apply, and the defendant is responsible for the illegal behavior, the defendant can absolutely be held legally liable and pay the legally-appropriate punishment.
IFPS is cool, however pretty far away from being usable as a package management system... Some package management system could use it as a backend, though.
That's why I'm looking into IPFS(https://ipfs.io) as part of my infrastructure. How that would look then, with IPFS...
> "So what you're saying is, your computers cannot possibly not continue damaging the plaintiff's interests." "That's correct."
> "You're being honest with me." "Yes, your Honor."
> "Will the computers continue harming the plaintiff's interests if shut off?" "No it wouldn't, your Honor.".....
And suddenly things like NPM can transfer the data to other machines, and those machines themselves can also provide to others. Deletions are impossible if people still want the content.
And IPFS guarantees that if a single node has the data, then any node can download it and also be part of the cloud that provides the data. Once it's out, it's impossible to retract.
I'm surprised all package managers don't use an IPFS-like system that uses immutable state with mutable labels and namespaces. Now that IPFS exists, and provides distributed hosting, it's even easier.
gx is a generic package manager on top of IPFS that uses git-style hooks for adding per-language support. It's already being used to manage dependencies on the go-ipfs project: https://github.com/whyrusleeping/gx
Yes the IPFS implementation might change but not the content multihash addressing. Linking to data with those addresses is the generic 'package management' that solves all these problems (references to mutable data at mutable origins, circular dependencies, data caching, namespace conflicts). The specifics of resolving links will hopefully be something we don't think about much.
I've played around with ipfs.js for resolving links into eval'd js at runtime and imagine a npm replacement would be pretty trivial. The IPFS peer to peer swarm seems stable to me but you could also dump all your hash-named files into a s3 bucket or something as a fallback repo.
What you mean by that? It used to be possible to republish a version (it broke our build when a dep was republished with a breaking change, that's how I learnt about it) but this was fixed some 2-3 years ago IIRC
Somewhat related, I just coincidentally stumbled upon https://github.com/alexanderGugel/ied . "Upcomming Nix-inspired features", to paraphrase their README, could well prevent this debacle.
(And btw, We Nix users very much do hope to start using IPFS :).)
Looks like the npm team will not be removing the ability to unpublish packages - see reply by core committer "othiym23" on https://github.com/npm/npm/pull/12017
There are a lot of problems with NuGet, but they got this right. I do wish there was a way to mark a package as deprecated, though. For ages, there was an unofficial JQuery package that was years out of date.
Read the whole thread. Rather concerned by the final one. "Locking temporarily" to get away from the discussion?
That feels sort of like the online discussion equivalent of sticking your fingers in your ears and going "la la la I'm not listening".
I don't expect someone in their position to be unable to ignore a conversation and "take a break" but I would expect them to be capable of doing so without resorting to "suppressing" the ongoing group discussion.
Or it's a "we're discussing internally, and would rather not deal with the shit-show that Github issues becomes once the issue becomes politicized and rampant misinformation and misguided activism take over."[1] There will be plenty of time for people to froth at the mouth and complain that they chose one way or the other once they've made a clear decision, which as of the locking the thread to collaborators, they have not (the current thinking has been outlined, but they said they are thinking about it).
I suspect your right, but honestly... His choice of language sounded much less like 'were thinking as a team', and much more like 'your all talking too loudly, you've given me a headache, so i'm going to shut you all up for a while'.
You mean the response that says, verbatim: "I'm thinking about the points that have been made, and I'm sure that we as a team will consider them going forward" ? Sure, he also says for now the behavior won't change, but that's the sane thing to do with the errors are rare, as changing something too quickly may introduce new bugs or unforeseen problems. Honestly, your interpretation of that comment is the exact reason why it's good to shut it down for a little while. The conversation gets so charged that even a "we need time to think about it" response is viewed negatively.
GPG isn't strictly necessary if you trust NPM's authentication system (of course, that's a big "if" for many folks).
Publish forever (barring requests to remove for legal compliance or whatever) is a good idea. Or at the very least, it should be a default option. And if you install a dependency that isn't "publish forever", you should get a warning.
This is what happens with Clojars. It is possible to remove packages, but it requires a manual email to the admins, along with an explanation, e.g. published internal lib accidentally. This prevents scenarios like this, but also cases where people want to 'clean up' things they no longer need, even though others are depending on them.
I think I'd just want to add that namepsacing by author doesn't entirely fix the problem. For the fewer instances where there is a collision, we still have this issue with lawyers asserting trademarks.
"Would the real Slim Shady please stand up?"
We want multiple 'kik's and multiple Shady's simultaneously. So record the gpg sig of the author in package.json, and filter the name + semver against just their published modules when updating.
Depending on how unique you need to be:
npm install <module> --save
npm install <author>/<module> --save
npm install <gpg>/<module> --save
On a side note, npm-search really sucks. It lacks a lot of fine-grained filtering. I'd love to be able to search by tags, or exclude results with an incompatible license, or even prioritize results by specified dependencies. npm-search needs love.
That's a good idea, but what if Kik lawyers come knocking on your door saying that you're breaking the law and this package cannot stay there forever, or for any moment longer?
Do they seriously not use any sort of public key cryptography to sign packages? Frankly, for a project that's been around as long as NPM, that's downright irresponsible if that's the case. That's like package management 101.
I really doubt anyone checks them. It's not integrated or enabled by default, there's no way to pin keys in the build files, etc. GPG isn't the solution to such problems, unfortunately.
In one of my old projects (bitcoinj) we did write a Maven plugin that let you put the hashes of the dependencies into your build file.
However it's rare to see Maven/Gradle builds that accept version ranges. And once downloaded it's cached.
For example, RubyGems it's possible to sign them, but it's not used as much as it should be because option security never gets used. Waxseal is a gem to sign other gems.
Generally though, it's a worse vulnerability than bitsquatting because it gives quarter to silent, advanced, persistent threats by definition. (Dynamic, untrusted code modification either in-flight or at-rest in difficult to prevent/audit ways.)
The primary way for change to happen is for some company to get hacked, but then it will only change for that one platform because most people are reactive not proactive. Changing this proactively seems painful, but it's less onerous than the risks of the alternatives. The key point is to make end-to-end verification and public key management mandatory and simple.
User A: Publish package X1.0 (with 2 factor auth)
User B: Download with npm package X1.0
User A: Unpublish package.
User C: Publish package X1.0, malware code (with a different 2 factor auth)
User B: Somewhere else, download and install package X1.0
For that to work, every package should be signed in package.js so that when you download a different version, you know about it. Also, I don't think it should be possible to alter previous versions. Package X1.0 should always be X1.0.
EDIT:
Seems like it's impossible to re-upload X1.0, which fix this issue. I thought once a package was unpublished, it was possible to republish the exact same version.
I'm left thinking of how Go does it, whereby repositories are downloaded via git or whatever version control software's URL. Making it impossible for the admins of "NPM" to take down a package. Add in IPFS and you've got one heck of a "package manager" I guess Go's approach is not really a "package manager" but nobody can truly stop you from downloading from a git repository, and if they do, you can go to the forks etc.
1. However, it would deal with GP's problem - that is, that somebody else could upload a package with the same name as one of the ones removed and cause ... surprises the next time someone runs npm install.
Github repos are namespaced to their owners, at least, radically reducing the potential for this kind of thing.
2. /s/github/whatever . As long as there's a public URL from which something can be cloned, the idea still works, as long as that URL doesn't someday point to something completely different. Again, not impossible, but less likely than with NPM.
3. There are, like it or not, real advantages to centralised archives - discoverability, optimisation of dependency resolution, etc. 'Privately controlled' is an elastic idea - it seems to me that there's a difference between a non-profit foundation, say, and a for-profit company like NPM or GH, but both are 'privately controlled'. The question is whether these advantages outweigh the disadvantages. In my opinion, they do.
About your third point, discoverability and optimization of dependency resolution can be solved by a proxy, in a decentralized system, like the one used in Go.
Except Go isn't GitHub specific, I could use BitBucket, Gitlab, or any service I setup myself. Even locally, just like git itself I suppose. Go supports other tools like Mercurial as well as others.
The vendor experiment provides a nice solution to that problem. Check in the vendor directory into your own repository and you always have the required source code available, even after the original author removes his repository on github.
It's just as bad. Commits can be re-written and/or deleted. I've always been surprised that anyone thought release management based on SCM commits was a good idea. It's not.
Not only is there a solution, but it's pretty well known computer science. Use public key cryptography to create a digital signature. It's not even a novel use of this -- .NET has been using this to sign assemblies and packages from day one.
I think he means a solution to the current problem of the packages being replaced with malicious ones, not the problem with npm not supporting package signatures.
Package signing by the package's author /is/ the solution, right? This way the only entity that can publish a valid update is the the package author. This still has the problem of bootstrapping the trust relationship, and leaves the door open to the author publishing a malicious but signed package or the signing key being stolen and used to do the same. However if you don't trust the package author to be responsible or take precautions against the keys being stolen then you're essentially taking on the burden of implementing of your required functionality.
We also need some way to link packages to authors. In the Java world packages are supposed to be in a namespace that's the reverse domain name, so you could (potentially, theoretically) connect that up with something like DNSSEC and enforce that people can only publish packages for domains that they control. (Though even that is really just punting the problem up to the DNS registries).
There is no need to sign. Just keep a cryptographic hash (SHA256 is a good bet) of the package in the dependencies manifest, and check it after download.
Attacker changes package, then changes hash in manifest to match. Checks pass, users are compromised.
Attacker clones git repo, creates new commit with trojan, pushes to repo with compromised credentials. New users clone compromised repo, others pull and fast-forward. All hashes are valid.
I don't get it. If I keep a hash of every dependency in my project, and an attacker change a dependency, I will detect the attack by computing the dependency hash and comparing it with the one stored in my project (for example in the dependencies lock file), which cannot be changed by the attacker.
That's fine on your personal system, where you manually update your package's dependencies and manually update the hashes of your package's dependencies.
Now, what happens if an attacker compromises the repo where people download your package from? Or what if he executes a MitM attack on the repo or a user who downloads from it? He can change the entire package, including all manifests, all hashes, etc. Users who download it will be none the wiser. By the time someone notices and corrects it, people will have downloaded the compromised versions and be infected.
The only thing that protects against this is strong crypto signatures. For example, if a Debian mirror were compromised and an attacker uploaded compromised packages, users would be safe, because apt would refuse to install the packages, because they would fail signature verification.
Please read the link I gave. It explains everything in detail.
SHA1 collisions are affordable for large actors now and getting cheaper all the time, so git, with its SHA1 assumptions hard-coded, unfortunately doesn't protect you anymore. But the local hash approach is a good one, proven in pip 8 (Python) and npm-lockdown (JS).
You don't get any prepublish hook when pushing to a git repository though, so you can't do things like preprocess with babel, typescript, etc, unless you check in the compiled source.
They're placed in the repo itself, under .git/hooks, rather than in the working tree. If you want to version them, you can always place a symlink in .git/hooks and have it point to a file inside the working tree. Note that this won't work for bare repositories (which have no working tree), but usually in that case you want a separate deploy process where a sysadmin (or build script) manually copies over files, to prevent a random dev from borking the repository, which a bad update hook can easily do.
(This doesn't help with the GitHub case. But then, as we saw from the recent CocoaPods story, using GitHub as your CDN is probably not a great idea either.)
Well, sure, but that's complicated, and ".scripts.prepublish" in package.json is not. I'd love to have the git knowledge to do what you've described here without googling and frowning, but I don't have that yet, and I've been using git for years. Whereas I could use package.json to accomplish the same task on basically the first day I used npm.
Legally isn't it totally NPM as a company to publish existing versions of left-pad because those versions were already released under a license that allows redistribution, whether the author wants them to or not? Or can the author effectively veto this?
Good question. Under something like GPL or Apache, npm (or anyone, really) would have a clear legal reason to continue using and redistributing the code; there's nothing the copyright holder can do about that (unless you actually violate the license terms).
Under WTFPL, that's certainly the intent of the license, but I haven't heard any court declaring a case based on WTFPL, so you don't actually know what the courts would make of it. But in general, yes, I'd bet a court would let people do what the f* they wanted once they got some code under WTFPL.
The separate question is: ethically or business-wise, do they want to continue publishing code from an author who explicitly tells them not to? Legally they (almost certainly) could, but as a FOSS business, would they want to?
Software Licenses are pretty easy for the basics - after you read the whole thing.
From what I can tell[0], left-pad was released under the "Do what the fuck you want public license"[1], so I suppose NPM can do whatever the fuck they want, including redistribution.
[0] https://www.npmjs.com/package/left-pad (although I suppose its possible the library was added after the package was re-added -- I don't see a license on the GitHub page)
Yes, but I can publish an evil left-pad@0.0.10, and if you're not shrinkwrapping or any sub-dependency has left-pad: "^0.0.3", it will pull in the evil 0.0.10 version.
EDIT: I stand corrected. See below, looks that's not the case specifically for "0.0.x" versions, but gets progressively more relaxed if there's a non-zero minor version specified. However, many of the unpublished packages had varying major and minor versions, which would have the more loose caret range behavior.
Wouldn't (shouldn't) npm block same named modules from being uploaded with a different username?
Proposal: go with a java based naming system: com.yc.mymodule.
All installations would require the user to use com.yc.* in package.json and all users would be required to register as a com.yc style organization name. Thus only one user can remove/add that exact module.
Hmm but then is the scope/namespace reserved for each user that reserves/uses it? If I publish @andy9775/left-pad can someone publish under @andy9775/other-module or can only I publish under @andy9775/...?
Yes, namespaces are specific to your username. Nobody else can publish packages under your username namespace.
I think it's less common because it was released along with private modules and is often conflated with them. Basically all packages are still namespace-less. TBH I never thought about the benefit they'd have until now.
So left-pad is too important to be removed but Kik isn't? What if the author had originally called left-pad Kik? Why is it ok for the Kik trademark holder to break thousands of builds but not the module author?
This is a very good question. I was going to make the argument that the author merely continued what NPM Inc. started, and that if you fault him, you should also fault NPM Inc., but then I noticed that NPM Inc. didn't unpublish his module, but transfered the name to another account, which is much worse, if you think about it.
The ultimate conclusion is that if it's anyone's fault, it is the fault of the person who relies on NPM Inc. when building his software.
I'm a known downer of npm and node in general and this illustrates how bad npm is about this... You can't just change owners to kik on npm without the previous maintainer's permission. NPM has violated the trust of the community and will continue to do so until someone forks npm and makes an honest community out of it.
What's next? If I have foo.bar domain, can kik come and say I can't have foo.bar/kik or kik.foo.bar? To the people who think npm is right, what if you owned foo.bar and Google decided they didn't want to deal with kik lawyers and redirected kik.foo.bar to kik.com?
I feel like the best way to avoid a disaster like this is for developers to avoid using registered brand names when they write their packages.
It's not difficult, since there are all sorts of rights brand owners can't get you on.
1. You don't really need a catchy name for an open source project, since you're not in competition for funds. Call it something descriptive. Descriptive words can't usually be protected, so you should be fine.
2. In most countries, using your personal name is fine irrespective of any IP rights.
3. If you want to use a catchy name anyway, check on the USPTO TESS database for registered rights. If any are live, choose another name.
Remember when Groupon tried to register Gnome for software applications[0], and the open source community (rightly) came out in force supporting the Gnome foundation? But when it's the other way round, it makes no difference.
Notice that I didn't talk about the node package manager, but about the author and the company NPM Inc. (and their registry service).
It's not that your "build got broken", it's that you had a broken build process. You are the one at fault. You chose (perhaps unconciously) to rely on various entities, their services, their whims, and they proved to be unreliable.
The simplest solution for those who write an application is to commit the dependencies into the repository as well. This significantly lowers the amount of entities relied upon when building it. (Alternatively, have your own registry, whatever.)
Then you can discuss issues like the ideal module granularity, ethics of this or that actor, names and trademarks, etc. without worrying about your "broken build".
ooh gosh, they should really do something like ban new projects from using those names until they figure out a better solution, hopefully with the module author.
Seems that we should now prefix any package name with a random string like 'skdjdihsawqy'. This way you could maybe avoid to be the target for some of those 'has to proof itself' lawyers.
This would solve classes of issues with the npm ecosystem (many of which remind me of trying to register a domain name) like name-squatting, trying to think up a marketable-but-untaken name, and ownership transfer.
OP claimed a bunch of sweet dictionary words (concat, iframe, door, bud, alert, map) and new owners are now claiming them and it's a security disaster. But it'd be a lot less interesting if they unpublished "azer/map", "azer/alert", "azer/iframe", etc. and new owners republished under their own names.
(Edit: this was written under the assumption that the lawyers in question are working on behalf of kik, the cheap clothes company, not kik, the messenger company. The original article is unclear about that)
This really has to be attacked at the root: let's all stop pretending that a sequence of characters can be owned. Before the web came along, people were completely sane about the protection of brands. Nobody had delusions about string ownership, but deceptive abuse of brand names was suppressed just as well. Enter the web, and suddenly corporations start thinking they somehow deserve exclusivity for their stupid little three-letter-acronym, at first on the DNS, now, apparently, also in search engine hit lists ("oh noes, someone might google themselves onto github instead of our site, people will start wearing NPMs instead of our clothes!").
That's pretty interesting. I wonder if it's within npm's legal right to distribute someone else's IP in such a way that they do not desire. Granted the license is WTFPL but does that include taking over the IP itself?
> Granted the license is WTFPL but does that include taking over the IP itself?
In some jurisdictions you have "moral rights" in addition to your copyright - but even in those, I'd expect the WTFPL constitutes a license to falsely claim authorship of the covered code. I mean, the text pretty clearly authorizes you to do so on its face, and courts lean pretty strongly towards reading words under their plain, normal meanings. IANAL.
Yeah I get the "do whatever the fuck you want" but in a legal sense I wasn't sure that could (or did) include actual ownership. I thought that had more to do with copyright and any other IP transferring. Then again I also said I had no idea :)
The more I think about this license, the more I wonder if it's even legally enforceable. How do you enforce a license whose only terms are that it has no terms?
> Granted the license is WTFPL but does that include taking over the IP itself
Well, I'm certainly not a lawyer, but I personally think thats in the spirit of the "Do What the Fuck You Want to Public License"... And at the end of the day the IP is still owned by the original author; the distribution of it has changed.
Post Berne, I believe copyright is the default. It may be WTFPL but it appears to be surprisingly hard to actually put something into public domain based on the brief reading I've been doing.
Uh... which part of "DO WHAT THE FUCK YOU WANT TO" is actually unclear here? Is there a word or phrase that you're not grasping?
"DO WHAT THE FUCK YOU WANT TO" clearly includes not only taking over the IP but also RE-LICENSING IT under whatever terms you like. That's kind of what "DO WHAT THE FUCK YOU WANT TO". Do. What ever the fuck. You want to.
Licensing doesn't really have anything to do with copyright / trademark / IP in general. It just grants you the ability to use something in the way specified.
I'm not aware of case law where a license has been able to move the original IP from one party to another; I've only heard of that happening through standard legal documents.
But I also pointed out that I wasn't sure how it would shake out anyway and was seeking feedback.
But why are you saying the copyright was transferred? I don't see anything that implies that. Someone got a copy of the code under an irrevocable license that grants them to right to re-publish it, and that's what they are doing.
> But why are you saying the copyright was transferred?
I don't think I did?
> I don't see anything that implies that. Someone got a copy of the code under an irrevocable license that grants them to right to re-publish it, and that's what they are doing.
Yeah mostly curious if the original author could, say, use the DCMA or something similar to force npm to take it down if he really wanted to.
I mean, you mentioned "take over the IP" and "move the original IP from one party to another", but I don't see why you're asking that, as NPM hasn't taken over any IP (nor has the new user), they simply redistributed the code.
npm didn't take it over, another user did and npm helped re-publish the unpublished version. I already answered the why I asked that..."mostly curious if the original author could, say, use the DCMA or something similar to force npm to take it down if he really wanted to."
> But the user didn't take over the IP, it only took over the name on NPM's registry.
That's almost the same thing. Like if someone dropped their domain name and you grabbed it up. The customer / user isn't going to notice a difference but it's now being represented by someone else.
I don't think anyone cares enough to really do anything about this though. Mostly curious if it was possible to do something about it but I'm guessing the waters are pretty untested.
This is extremely severe. Any package i install might after x levels of sub-dependencies pull in one of these names which are potentially pwned. React and Babel pulled in a few of them to take some well known examples.
I would say the whole npm is pwned until these packages are either restored or that the package name is blacklisted/reserved. Is it possible to blacklist packages locally so I get warned if they are pulled in as sub dependencies? I don't trust every package maintainer to be quick enough to remove these dependencies.
Because NPM knows a module has been unpublished, it can try to be smart about it and change existing package.json so there will be no more updates to that module.
I also think dependencies should be non-flat/tree/static. So that you only have to trust one dependency, and not all of it's child dependencies and their dependencies too. You should only need to manage your own dependencies, not other peoples dependencies.
There should also be a way to contact everyone who use your module, so you can tell them about critical bugs etc.
One interesting thing to me, is that it is pretty clear that the kik lawyers pretty dramatically over enforced their trademark.
For those who don't know, the purpose of trademarks is to prevent customer confusion; essentially we don't want people to be able to sell cheap knock-offs of someone else's thing without the general public being able to easily distinguish between them. In practical terms, trademarks are "scoped" by their "goods and services" declarations.
For example, Apple the device manufacture[1] and Apple the record label[2] could both be trademarked because they had non-overlapping goods and services declarations... until iTunes started selling music[3].
If you look at kik's trademark application[4], you can clearly see that the trademark is limited to chat/media consumer applications, a pretty obvious over enforcement.
Most Germans wouldn't know that kik is also a chat app ;-)
"KiK is the largest textile discounter chain in Germany and operates about 3,200 stores in Germany, Austria (since 1998), Slovenia and Czech Republic (since 2007), Hungary and Slovakia (since 2008), Croatia (since 2011) and Poland (since March 2012)."
Being from Germany, I had initially assumed that was in fact the company in question. I was wondering why a textile discounter would care about some NPM module...
The Apple thing doesn't work though, for a specific reason: it's a very common word/name.
Smith is a very common surname. If one person starts the Smith Automobile Company and another person starts the Smith Farm, obviously there's no issue there. That's because none of us invented "Smith", it's understood to be a common name, etc.
On the other hand, if I start "The Google Paper Company", I'm pretty damn sure I would quickly and easily lose that case, despite the fact that google does not sell paper. And nobody would think that is weird, because Google is very obvious an invention (Yeah, I know, "googol", but the spelling makes it unique).
"Kik" is a lot closer to the "google" situation than the "apple" situation.
That is not how trademarks work. Feel free to start the Google Paper Co. Uniqueness is not a merit for trademark infringement. To infringe you have to be a competitor.
That is how some trademarks work. Google is by now a famous mark and, as such, gets additional protection beyond what mere mortal companies do: http://itlaw.wikia.com/wiki/Famous_marks
I guarantee you won't last long if you start a Coca-Cola School of Hairdressing, even though the beverage company doesn't compete for that business.
Famous and "Unique" aren't the same thing. I was responding to the theory that because "Google" is a made-up word, they'll receive extra protections. That's false. That said my example was a bad one, since Google is a famous mark Google Paper Co. would likely be infringement.
That is incorrect. You are also infringing if people could reasonably assume an affiliation due to your use of the trademark, which would definitely be the case with Google Paper Co. You're basically piggy-backing on (and diluting) the brand recognition they've built.
You are correct in that my Google example would likely be trademark infringement because Google is a famous mark. Kik is not a famous mark, so for this case the famous/well-known aspect of trademark infringement is entirely moot.
And keep in mind that the OP I was responding to was arguing that if the name is unique it gets extra protection. That's false. Fame offers extra protection, uniqueness does not.
"According to Kik Interactive, as of December 2015, Kik Messenger had approximately 240 million registered users, and was used by approximately 40 percent of United States teenagers." https://en.wikipedia.org/wiki/Kik_Messenger
Are you sure a court wouldn't consider that qualifying as a famous mark? I'm not going to dig into case law, but my guess is it very well could be.
"Evidence relevant to the fame of a trademark may include sales, advertising and revenue figures; geographical scope of use; channels of trade; registrations in home and other countries; past enforcement efforts; and the results of consumer recognition surveys (provided the survey methods are approved by the courts in that jurisdiction)." http://www.inta.org/TrademarkBasics/FactSheets/Pages/FamousW...
> Are you sure a court wouldn't consider that qualifying as a famous mark? I'm not going to dig into case law, but my guess is it very well could be.
Teenagers represent 9.5% of the US population meaning that 3.8% of the US population (using their 40% figure) have "used" their app. Alternatively the 240 million registered users would imply about 3.5% of the total global population has used it.
I can say that I honestly have ZERO doubt that "kik" would never be ruled famous by any possible measure.
I don't know whether you're right or not, but I do know it isn't prudent to have ZERO doubt about anything - but especially legal matters - without first doing the relevant research. And even then, courts disagree all the time, so I can't imagine ever being 100% certain that a ruling would never go a certain way.
To be even reasonably certain in this case, I would need case law of similarly well-known brands being challenged. Do you have some? Capturing 40% of their target market seems pretty well known to me, but again, that's meaningless without the case law.
Check out the case law about Mcdonalds v Quality Inns (re: McSleep Inns)
Mcdonald's was able to show that there would be consumer confusion DESPITE Mcdonald's not doing hotels nor Mcsleep inns not doing food.
As a result, Mcdonalds basically has an open trademark enforcement on "Mc-" whatever.
While I'm not fond of the judgement here, it was nonetheless decided thusly, and contradicts your otherwise accurate (so far as I know) statement. (And to weaken my own point, I believe Apple lost a similar case about i<whatever>, so nothing here is clear and reliable.)
That's not necessarily true if Kik were planning to open up some kind of API. Also, the way the law works in the US encourages corporations to overreach. If you don't try to protect your trademark from every possible angle, it can lead to your loss of it in a court battle. A lot of times, lawyers will send out letters without caring how the recipient responds -- they only need to have sent the letter at all to have covered their bases.
It doesn't have anything to do with the API. Kik (the company) registered the name and have rights to the name.
Technically, Kik (the company) registered their trademark in the class "Computer Software" [0]. That means that no-one else can use the word Kik (and the logo) for this class of activity. The key issue is when the registration happened. Since they've been going since 2009, and Kik (the software project) only started in 2015, Kik the company was there first.
The reason their lawyers are asking for "Kik the project" to change name is because trademarks can become generic if you're not seen to protect your mark [1]. As a private company (particularly VC funded), IP has a lot of financial value: the business will be valuing their trademark. If they don't protect it then they'll be "throwing money away".
The fact that the business world (and VC's generally) highly value IP is why most people over-register. If they didn't go for wide classes, and then in a few years decided to work in a particular area they might not have rights to their "own name" in that class.
It's a case of the Open Source and Business world-views clashing.
[0] Someone else in the thread found the specifics but the link doesn't work for me.
[1] Everyone knows the example of Spam.
Please show an example of someone other than Hormel Foods Corporation marketing a meat product using any derivative of the name 'Spam' and getting away with it.
Spam was mentioned because it's a brand name that became generic because it wasn't protected aggressively enough. Coke, Xerox, and Kleenex are other classic examples of brands that are commonly used generically, and the company owning their trademarks must aggressively send threats (that they rarely follow up on) just out of legal necessity.
Spam very aggressively protects their trademark. So aggressively that it lost a trademark lawsuit against Spam Arrest. (They lost cause people don't get food and junk mail confused)
Indeed, Hormel once sent nastygrams about their trademark to the ASF over our Apache SpamAssassin software. Trademark owners can ask everyone to use or not use their own names. Actually going to court - or actually winning - is far far rarer.
See also: YellowPages.com, Salesforce.com, etc. Just because something exists in real life (and used to describe a non-digital version of the same thing) doesn't mean it can't then be trademarked.
> A lot of times, lawyers will send out letters without caring how the recipient responds -- they only need to have sent the letter at all to have covered their bases.
Any chance someone at Kik the company wanted to create an npm module and decided the escalation to lawyers was the right way to resolve things? Seems like the lawyer was more interested in getting possession of the name on npm then actually fighting for a trademark. It doesn't even sound like they bothered with a formal cease and desist.
Guess that answers that. Bob may have been going beyond what he should have or what they intended but in the end he represented Kik and they have to take responsibility for that.
Yep, I was going to say the same thing. As you say we can look up the US trademarks [1, 2] and see that this mark covers:
Computer software for use with mobile phones and portable
computing devices to:
- download audio, video, digital photos and programs;
- electronic payment systems, namely, a computer application
software used for processing electronic payments to and
from others;
- computer software for use with mobile phones and portable
computing devices to create video and digital photos to share
with other users; computer software for use with mobile phones
to launch other applications and connect to other software
services.
"Providing an interactive website featuring online non-downloadable software" indicates to me that there could indeed be confusion in a repository of software code to be used by other applications.
I'd suggest it's certainly not obvious over-enforcement.
But TFA's "Kik" seems to be sort of a "project generator" like yeoman or something. There is no overlap between that and even the generously expanded description of the litigious "Kik" that you cite.
So, in your opinion, if I built a software application that let you design and order plate glass in decorative form to be placed in the walls of your house and named this app "Windows" would Microsoft's inevitable attempts at protecting their trademark be overreaching? The windows I named my app for are physical things and Microsoft's are virtual, but I'm releasing a software product called "Windows."
Both kik names exist in the realm of software. There's an argument to be made for confusion.
Yes, even this faintly ridiculous fake example of painstaking mental construction would be overreach. No one would ever download your hypothetical architectural software expecting to get an operating system.
As a public service, some toilet or compost bin manufacturer ought to start "violating" trademarks by naming toilets etc. after litigious companies. We'd all get a kik out of hearing them argue that database consumers are likely to confuse the Oracle database with the Oracle water closet. [EDIT: "Sure it's a CRUD app, but I'm more interested in Deletion than Retrieval!"]
Of course there are arguments to be made; lawyers are involved.
People with experience in software wouldn't likely confuse the two pieces of software. But most people think "computers" and stop there. Most people visiting an app store would see "Windows" for 99 cents, buy it, then complain when it wasn't Microsoft Windows. They wouldn't read descriptions, and they don't know what an operating system is.
Confusion. Microsoft is very likely to prevail in the infringement lawsuit.
Those products would not conflict there isn't a giant category called computers and further you can't just take a generic word out of the English language and take possession. Apple is a generic word for example but not in computer technology.
This matter has even already been litigated. Microsoft Windows is a valid trademark, Windows is not defensible.
Microsoft's claims were rejected by the court, which asserted that Microsoft had used the term windows to describe graphical user interfaces before the Windows product was ever released, and that the windowing technique had already been implemented by Xerox and Apple Computer many years before. Microsoft sought a retrial and after this was postponed in February 2004, offered to settle the case. As part of the licensing settlement, Microsoft paid an estimated $20 million, and Lindows, Inc. transferred the Lindows trademark to Microsoft and changed its name to Linspire, Inc. [0]
Oh yeah, it seems like M$ were on the verge of winning that appeal, and only discontinued (and paid Lindows a multiple of its annual profit) out of pity.
Just a general comment about the whole thread: trademark law is far more complicated and particular than you know (unless you're an IP lawyer; then you might appreciate it). Not only are developers used to problems that can have provable or obvious answers, trademark law really hasn't caught up with the speed of the internet, especially the instant and global reach that a domain name has.
- The only thing you can say about a trademark dispute like this, given the information that's public, is that "it depends on the specific situation as well as the perception of a large enough user base who might get confused".
- Trademark owners can ask (or C&D demand) just about whatever they want. What they could actually force you to do in a court is far, far less than what they typically ask.
They have the responsibility to defend their trademark, otherwise it could be abused by a rival claiming they did not adequately defend it. That said, NPM could have just said, "No, stop bothering us" and the lawyers might have backed down, satisfied their attempt to defend the trademark fulfilled their duty.
>They have the responsibility to defend their trademark
This is not limited to only issuing cease and desists. Kik Interactive can offer a zero cost license for the trademark if they want to assert their ownership, but let the project continue to use their name.
You do not have to defend your trademark against people who are not infringing it. You can only infringe a trademark when you use the mark in the same context it's registered in.
I applaud this action and while I'd like to point the finger at NPM, there's no real other method to fix historical package versions that depend on this.
It is worth pointing to the silly state of NPM packages: Who decided that an external dependency was necessary for a module that is 17 lines of code?
module.exports = leftpad;
function leftpad (str, len, ch) {
str = String(str);
var i = -1;
if (!ch && ch !== 0) ch = ' ';
len = len - str.length;
while (++i < len) {
str = ch + str;
}
return str;
}
Developers: less dependencies is better, especially when they're so simple!
You know what's also awesome? The caret semver specifier[1]. You could install a new, broken version of a dependency doing that-- especially when other packages using peerDependencies rely on specific versions and you've used a caret semver specifier.
The caret character doesn't appear anywhere in the semver spec, so whatever that does, it's non-standard: http://semver.org/
If your modules are small and well-defined, they probably won't need many versions anyways - they might just stay on 1.0.x forever. If you want to do something different, it might make more sense to just write another module.
Less dependencies are better. This job is to communicate everything you do in a software project: its reasons and its purposes for every piece. For example, application code that will never be in a public API does not need anything as complex and thoroughly considered than a generic library. When it comes to writing a 10 line function that is represented in an existing module, the most likely reason is that I am writing it for my own purposes and without having to explain why I brought in a package to answer a very small problem.
I implemented KeyMirror from NPM once. It's a simple array->object transformation. It's been in production for months without issue. But, I initially got guff from my boss over it for not using the package. If anything, the package is just an example proof-of-concept of an extremely simple idea. But, carrying the bloat of another package next to more relevant packages seems to be more important here than just merely owning a simple piece of code like this.
The caret character is a specification in NPM, not semver. It's designed to work within the semantic versioning rules to ensure you get the latest version which includes bug fixes, but also won't include breaking changes.
For example, ^1.3.2 will allow anything greater than 1.3.2 but not 2.0.0. It also has special behaviour that makes it more strict for projects with a major version of 0. If your dependencies follow semver then you'll get bug fixes and security updates without having to do anything or worry about breaking changes.
This dichotomy is silly. You should write as little code as possible to do the job required, and should use only the dependencies required. This might be none, or nearly everything, depending on what your app or library is supposed to do.
...Only that, in Java, it's not exactly the nuclear reactor you want, which forces you to use someone else's solar power plant, and then they deprecate the original nuclear reactor (because it's known to leak uranium) in favor of a second nuclear reactor, which still doesn't work that well because it's the wrong polarity, so now you have three energy sources available in your project and if you plug your lamp into the wrong outlet everything explodes (see the Date/Calendar mess).
Having a multitude of small utilities like this is a great thing with many advantages.
It may seem simple to write leftpad, but if 1000 projects that need it all write their own version, there will be at least 2000 more software bugs out there in the wild because of it. If you think that's rediculous, you're not being realistic about the huge disparity in skill levels of industry programmers as well as the considerable rush under which many projects are done.
Also important is that every time I can use a public ally available utility instead of writing it myself, it's one less thing I have to think about. The benefit of making less decisions really adds up, saving a ton of mental capacity to focus on the more important parts of my project. Even the simplest methods require some thought to design.
I know there are disadvantages (such as what happened as the topic of this post), but there are also ways to mitigate them. As far as having many versions that all do the same thing, there is usually winners and losers over time. Because of this I believe that eventually the dependency graph shrinks overall.
Note that I wouldn't advocate creating utilities for things that are not very generalizable.
It doesn't have any dependencies yet -- what if leftpad decides it should depend on "pad", which depends on "stringutil" which depends on "unicodeutil" and "mathutil", etc.
I've never used npm, but doesn't it take at least as long to find, evaluate, and install a package like left-pad as it would to just write the function yourself when you find you need it?
No. I could find, evaluate and install that package quicker than I could write the code that carefully. And the second time I need it, it's just "remember, install". Also, keeps my code small and focused.
Do you not read the code of packages you're including in your projects? I usually do at least browse through the "beef" of the code for anything I consider adding as a dependency. And try to figure out what's the level of maintenance I can expect, history of bugs, sometimes try to contact the authors, etc.
In short: it would take me a whole lot more time to evaluate whether or not to depend on something as trivial as leftpad than to write it myself. I'm pretty confident I can do that in a minute or two and I trust my collaborators to not break it.
The possibility of having bugs in code you don't control (that usually has a clause for no warranties) is an argument for implementing it yourself, not against it. Don't forget how hard it is to get a maintainer even agree on whether something is 1. a bug 2. that needs to be fixed.
The reality, however, is that if you took this point of view, you will spend your time reinventing the wheel, introducing bugs and wasting resources. That's how it works in real life.
If someone already wrote the base code, we can always fork it and fix a bug or add a feature ourselves if it runs contrary to what the original authors desires.
Even getting a response just so you can know what the original author desires can take a long time and there is no warranties or guarantees that you will even get any response. To me, all the downsides that come with dependencies are not even close to worth it for saving 15 seconds.
If you fixed the bad behavior you're experiencing, and the original author's effort saved you hours or days of coding, what's the downside?
Perhaps I'm not arguing for 15 second long code changes. But other than typing a single if statement, what takes literally less than one minute to securely change in any partially-complex project?
Fair point. One does run the risk of having bugs in code out of their control by using a package manager such as NPM, but one gains a swath of other programmers inspecting the modules for bugs. And in module repositories for interpreted languages, its very much in your control to fix any bugs you might find, regardless of what the maintainer might say about it.
No - then you won't get updates easily and everyone reading your project would have to make sure that your copy of the module hasn't diverged from the original module before working with it, especially if it's a larger module that has full documentation and a community of people who know how to work on it.
That's what I thought, but what concerns me with 100's or even 1000's of dependencies is managing them. Things like figuring out which ones have security issues without reading the issue tracker for each.
I'm curious since it strikes me as a hard problem to solve: How do you resolve having to deal with security issues with tens or hundreds of dependencies (and their dependencies)? How do you even know whether they have a security issue or a version bump is just a bug fix without digging into each one on a regular basis?
That's a fair point. But what would concern me, as a lone developer, is liability if you get hacked due to a known vulnerability in an npm module. If the company is looking for a head to roll and someone points out it was a known and resolved issue in later versions that could be a problem for me.
Does npm let you tag releases as security fixes? That would make automation to discover it possible.
NPM itself is clearly faulty, but I don't think the concept of outsourcing logic to dependencies is. If something is complex enough to have a legit security vulnerability, it's probably the sort of thing I don't really want to write myself. And yeah, that comes with the responsibility to stay up-to-date. But pretty sure my head would rightfully roll anyway if I wrote my own bootleg SSH protocol and got my company exploited.
> As far as having many versions that all do the same thing, there is usually winners and losers over time. Because of this I believe that eventually the dependency graph shrinks overall.
That's... very naive. No one goes back and rewrites perfectly working code just to change a library. If it works don't touch it. Computers don't care, and if you rewrite it, you're introducing a bug. Also, there's plenty of new code to write! And oh yeah you have a billion little libraries, all used by an engineering culture constantly distracted by the new shinny, so you're going to be stuck with libraries that that haven't updated.
Personally i'm going to use an installable module for something even that small, because i can, and it works.
The benefits from an install registry don't go away just because the module is very tiny...
Why would i spend my time re-inventing the wheel for every little thing i do? And if i'm not reinventing, then i'd be copy/pasting which is much worse. At best that's a waste of time and effort to properly document the source, and at worst it's stealing or license violations.
I don't care if a module is a single line, if it does what i need it to and is well tested, then i'll use it. That might seem silly, but the fact is that it's pretty much no overhead, and no software is immune from bugs (even a 16 line function), so updates might be useful in the future.
Yeah, there is a chance that stuff like this can happen, but within an hour there were several alternatives to solve issues with installs, i'd say the system is working pretty well. Plus with proper software development techniques (like vendoring your dependencies) this wouldn't even be a problem at all.
The overhead is in your management of your dependencies. The size of the module isn't the problem, it's the fact that you end up using so many of them (especially recursively).
Consider this specific case. This author moved all their modules from one hosted location to another. Now, if you want to use these modules from that author, you need to update the scripts and configs that install them (some package.json files in this case). In a better world, like the C or Python world, you might need to update one or two urls which point to a couple of this author's popular libraries (maybe one you use directly, and one used by one of your handful of direct dependencies).
In this crazy npm world, this author has 272 modules. Maybe 20 are in widespread use ... it's already a lot of work to figure that out. Maybe you use a couple directly, and your dependencies have private recursive sub-dependencies on additional copies or versions of these or other of this author's modules! Maybe you have to cut your own versions of some of your dependencies just to change their package.json to refer to the new URLs! Anyway, you probably have to sift through hundreds of your dependencies and sub-dependencies to see if any of them are included in these 272 moved modules.
I've seen npm dependency trees with over 2000 modules (not all unique of course). That's totally unmanageable. I think that's why privately versioned sub-dependencies is a big feature in nodejs: so you can try to ignore the problem of an unmanageable dependency tree. But if you need to make reliable software, at some point you need to manage your dependencies.
I agree that NPM needs to push namespacing much harder, as that would make the whole process much easier.
Also a "provides" field could go a long way into stopping issues like this. Allow packages to say that they provide a package in them that is compatible with another in these version ranges.
That would let "API compatible" packages be dropped in to replace even deeply nested packages easily, and would allow easy "bundling" in big libraries while still allowing easy creation and access to "micro libs".
I really believe that composing tons of small libraries is the way to go, but there needs to be better tooling to make it work. In my (admittedly not extremely expirenced) opinion, bundling many small libs into one big package to make it manageable is a symptom of a problem, not its resolution.
This will become easier with rollup, webpack@2, and so on, which can effectively reduce the penalty of including large modules like lodash by tree-shaking out all of the parts you don't use. I would expect many more utility libraries to then be aggregated into a single package/repo and for users to simply pick and choose functions at will.
By this logic, every Stack Overflow snippet should be a module. I'm almost hesitant to suggest this since many people who read this will be capable of building such a thing.
"echo" is not versioned and delivered on its own. It's part of gnu coreutils (which contains ~ 100 utilities), or part of various BSD core distributions (more than 100 utilities, plus the kernel and libc), and also built-in to shells.
The fact that in JS land it would be it's a standalone module means you get more choice in what you need (no need to pull down 100 programs if you only need 1 or 2).
You have the same amount of choice. There's no reason that you have to use the other hundred pieces of the package. In the Unix world, there's nothing precluding you from deciding to use the FreeBSD version of tar but keeping the rest of the GNU utilities there.
But to be fair this would have the same outcome if left-pad were part of a library that included another 50+ libs (that he also wrote and published, and subsequently un-published today).
More choice, but now you need 50 different modules from 50 different authors to duplicate what would be in one good standard library, any of which could have bugs or be pulled out from under you for a multitude of reasons that are beyond your control.
Choice can be a bad thing too - when there are 10 different modules for doing a moderately complex thing, you have to figure out which one is best for your project, and whether it's still actively maintained, bugs are fixed, how do they feel about making breaking changes, etc.
>now you need 50 different modules from 50 different authors
Not necessarily, take a look at lodash and friends. There is nothing stopping bundling of tiny modules into big "libraries" to be used.
As for the rest, you need to do that validation anyway. But if it were bundled in a large library there is MUCH more code that you need to review.
with something like the module "left-pad", it's a no brainer to use the library. I know that it's under an EXTREMELY open license, the code is really small, and by vendoring your dependencies (you are vendoring your dependencies right?) this whole issue would have been a 5-10 minute fix that only needed to be done when you want to upgrade your dependencies next time.
But also, if you're shipping things to users browsers, please cut out all the stuff you don't use. I don't want to download 100 extra module's worth of JS code because it was bundled.
Because in JavaScript there are many different implementations of engines, and they run on all kinds of stuff. Adding something to the standard is not a small task, and it means that it's now extra code that needs to be installed on practically every single PC.
And that doesn't remove the need for a library like this (or your own implementation) for a long time because you can't rely on a brand new release to be available for everyone.
Feel free to show a smaller implementation that's more efficient.
I've seen several "one liners" in this thread already, and most of them either blow up when something that's not a string is passed in (regardless of how you view strict typing, js doesn't have it and this shouldn't happen), or are extremely slow comparatively (most of them creating and destroying an array every time they are called).
Plus this has 100% test coverage (even as trivial as it is, it still counts), and is "battle tested" (something like 2.5 million installs per month counts for something).
Sorry, but i'll stick to left-pad vs 20-seconds of thought one-liner.
No local variables, less manipulation of the input string, the string grows at the tail which is more efficient, and the code is much shorter.
(With a bit of work you can use the longer ch string that is built to reduce the number of string appends even more by appending multiple characters at once. Although probably not worth it for this function.)
No offense, but that code is much more difficult to understand. If your goal is to minimize the amount of lines, then you succeeded. If the goal is to produce both correct and readable code, then there's room for improvement.
> No offense, but that code is much more difficult to understand.
I strongly disagree. My code has no magic initializers (the -1 in the original) and a simple linear code path, with no branching. It's very easy to read and understand.
The ternary operator at the top is simply read from left to right, it's not complicated.
> If your goal is to minimize the amount of lines, then you succeeded.
My goal was to maximize efficiency. Often that means less lines, but that was not the overt goal. And in fact this version runs faster, and uses less memory.
> If the goal is to produce both correct and readable code, then there's room for improvement.
You think so?
Then now it's your turn - rewrite this (or the original) to make it as readable as possible. I think you will find that a: mine is more readable than the original, and b: you won't be able to (need to) change much except to lift the len initializer out of the ternary operator in the first line onto its own line.
If you are talking about a module that everyone just includes and expects to work, then I'd imagine the goal would be a combination of correct and efficient, not readable.
Someone with more JS experience can chime in, but isn't this really inefficient in JavaScript? Wouldn't appending rather than pre-pending be better due to the way strings and memory are handled? Or at the bare minimum create the left padding in the loop and tack on str after? Can you use ch.repeat(len) + str; yet in node or if not just do the same idea of doubling in size ch until len is satisfied?
In this case the npm ecosystem is providing more of a surrogate standard library. Imagine if there were no libc, for example, and so people had to reimplement all those functions; would you really want one package per function because of how "Unix philosophy" it would be?
This is where the JavaScript ecosystem is right now -- JS doesn't have the kind of robust standard library other languages take for granted, so you end up with a lot of these things that look like they should be stdlib functions, but are instead third-party packages the entire world has to depend on for a usable programming environment.
I don't understand the standard library argument at all. Standard library is always limited, there is always something that's not included - then what?
The argument is that things like formatting a string -- including padding it -- should not be add-ons, they should be built into the core distribution you get as part of the language.
I didn't say it shouldn't be part of standard library, I am saying that there is always something that isn't part of the standard library. My point is: if this module was something that shouldn't be part of standard library, what argument would you then use?
Which is useful for binaries. Libraries, there tends to be a lot more aggregation. If this wasn't the case, you'd see libc spread over dozens and dozens of libraries like libmalloc librandom libstring libarithmetic libsocket libprocess libfilesystem libafilesystem (hey, async ops have a different api) libtime, etc.
Because a lot of little libraries makes namespaces more complicated, makes security auditing more difficult, makes troubleshooting more difficult as you have to start digging through compatibility of a ton more libraries, makes loading slower, because you have to fopen() a ton more files and parse their contents, etc. Add on top of that those little libraries needing other, probably redundant little libraries, and you can start to see how this can turn the structure of your code into mush really quickly.
At this point, optimizing runtimes to only include necessary functions, and optimizing files to only include necessary functions are both things that are pretty much a solved problem. For example, azer has a random-color library, and an rng library. Having both of those as azer-random or something means that someone automatically gets all the dependencies, without having to make many requests to the server. This makes build times shorter and a lot easier.
Sometimes, in order to best optimize for small, you have to have some parts that are big. Libraries tend to be one of those things where a few good, big libraries lead to a smaller footprint than many tiny libraries.
Not to mention maintenance. Low-hanging fruit here: left-pad, depended on by fricking everyone, <em>is owned by one guy</em>. That is not how you build a serious utility library that people should actually be using in real products. (You also probably shouldn't license it under the WTFPL, but that's another thing). When you aggregate under a common banner, you get maintainers for free and your project might have a bus factor better than 1.
What? If you go all the way, you just review all dependencies too. And if they have a good API, it's actually much easier. For example if your only source of filesystem access is libfilesystem, you can quickly list all modules which have any permanent local state.
Splitting huge libraries into well designed categories would make a lot of reviews easier.
> Having both of those as azer-random or something means that someone automatically gets all the dependencies, without having to make many requests to the server.
Also disagree. One-off builds shouldn't make a real difference. Continuous builds should have both local mirror and local caches.
Yeah but that's not the world of NPM. It's a clusterfuck of a maze of near duplicate dependencies with no hierarchy or anything. There's no organization or thought. It's just a bunch of crap tossed together to encourage cargo cults programming.
Erm, that's not a very good example. You're pointing out some ancient source file from back when unix had no package management. These days echo.c is part of coreutils, a large package which is economic to manage dependencies for at scale.
It's interesting to think about the distinction between promiscuous dependencies (as pioneered by Gemfile) and the Unix way. I like the latter and loathe the former, but maybe I'm wrong. Can someone think of a better example? Is there a modern dpkg/rpm/yum package with <50 LoC of payload?
Edit: Incidentally, I just checked and echo.c in coreutils 8.25 weighs in at 194 LoC (excluding comments, empty lines and just curly braces). And that doesn't even include other files that are needed to build echo. Back in 2013 I did a similar analysis for cat, and found that it required 36 thousand lines of code (just .c and .h files). It's my favorite example of the rot at Linux's core. There's many complaints you can make about Unix package management, but 'too many tiny dependencies' seems unlikely to be one of them.
>"When I looked at the gnu implementation of the "cat" command and found out its source file was 833 lines of C code (just to implement _cat_), I decided the FSF sucked at this whole "software" thing. (Ok, I discovered that reading the gcc source at Rutgers back in 1993, but at the time I thought only GCC was a horrible bloated mass of conflicting #ifdefs, not everything the FSF had ever touched. Back then I didn't know that the "Cathedral" in the original Cathedral and the Bazaar paper was specifically referring to the GNU project.)"
which are needed for things like memory allocation, filesystem access, io, and so on.
One can imagine alternative implementations of cat that are full of #ifdefs to handle all the glorious varieties of unix that have ever been out there.
The main point in support of your argument is the fact that Unix utils are bundled in larger packages instead of shipping in single-command packages. Think of Fileutils, Shellutils, and Textutils... which got combined to form Coreutils! Ha! That leaves only Diffutils, for the time being anyway.
But none of Fileutils, Shellutils and Textutils was ever as tiny as many npm or gem modules.
I thought how commands are bundled into packages was the entirety of what we were discussing. That was my interpretation of larkinrichards's comment (way) up above. Packages are the unit of installation, not use. Packages are all we're arguing about.
My position: don't put words in God's mouth :) The unix way is commands that do one thing and do it well. But the unix way is silent on the right way to disseminate said commands. There you're on your own.
Underscore/lodash is a great bundle of functions that do simple things well. And it's a tiny enough library that there is really no need to split it into 270 modules.
I support packages of utility functions. Distributing them individually is a waste of resources when you have tree shaking.
I trust a dependency on lodash. I don't trust a dependency on a single 17 line function.
While I agree with you, tree shaking is relatively new in the JavaScript world thanks to Webpack 2 and Rollup.js before that, if you had a dependency you brought it's whole lib into your project whether you used one method or all of them. So just including Lodash wasn't an option for people who cared about loading times for their users. A 17 line module was.
This. I'm on Windows. Npm never worked for me, like not at all. Npm has cost me lots of wasted time, I know I should be thankful for this free product, but, but, Grrrrr...
On at least one OS I worked with, it was 0 bytes long because /bin/sh on an empty file returns true. (I think that was IRIX 4.x.) OTOH, that's slower than the executable from a 3 LoC do-nothing program.
Let it suffice to say that Linux distributions (and some closed OSes) try very hard to prevent package fragmentation.
Experience has shown many times that excessive fragmentation leads to proliferation of unmaintained libraries.
The solution (IMO) is between those: Use lodash's padding functions. It's modular by default so you don't bring in the whole library, JDD is a performance junkie, and it's lodash so it's not going to get unpublished.
If not, writing it yourself works too. Or advocate for a better string stdlib.
Less dependencies is definitely better, but that doesn't mean "write the code because we can't install it via npm.
A developer could come along, find this library, and hard-copy it into their source repo. The tested source code is there, and they don't have a dependency on npm.
This wouldn't work quite so well for large packages (chance of bugs is high and so patches are important), but for something like this? Just ditch npm.
> Developers: less dependencies is better, especially when they're so simple!
I tend to agree, but this is conflating the issue of having dependencies with delivery.
It's perfectly ok to build small and compose large, with some of the smaller constituents being external dependencies, but the problem here is that the delivery of the packages isn't immutable and static. When you publish a package to npm, you don't publish it along with a copy of all your dependencies (by default, there are mechanisms to do this however.) The external dependencies will be pulled in separately when people install your package. What you're suggesting could still be done with an external dependency, just by making sure you it's only external at development time, but at publish it is truly bundled along with your package. This obviously comes with other costs, like the inability to dedupe packages.
Reading some of the comments reminds me old tale about a young man, that every morning on his way to work passed by a beggar and gave him a coin (that was back when coins actually had some value). One morning though the beggar notices the coin is smaller than usual, and he asks:
- Why you gave me a different coin today?
and the young man says:
- I got married and now I'm starting a family, I need more money so I can not give you as much anymore.
And the beggar cries out:
- People, look at this putz, he got married, and now I have to feed his family?!
I think the fact that we get so many awesome things for free is unbelievably lucky. I mean, not only we work in the one of the more generously paid jobs, we also get a lot of the tools we need for free! How cool is that? But some people think that if they are given those awesome things for free, they must deserve it and whoever gives them owes them forever. That's not the case. Yes, it is annoying to find somebody who contributed before does not want to do it anymore. It is mildly inconvenient and it can be improved. But let's not lose the perspective - the author does not owe us or npm continued support. It is sad he does not want to do it anymore, but that's what open source is about - people can take it over, and it happened within a single day. Such resilience is something to be proud of, not something to complain about.
A more accurate story would be: A young man is giving a piece of wood for free to a beggar. Then one day, when the beggar has constructed a house using the wood, the young man asks it back :
- But you gave it to me!
- It was for free, so don't dare you to complain
If it was given with the (maybe implicit) clause that there was no limit of time, I don't see how the thing being free give you the right to take it back, especially from people who have used it to build something. Sure the first act was charitable, but in the end you cause more harm than good.
It's more like the young man telling the beggar that he can come cut the wood from his land for free. Then one day the young man stops letting the beggar cut his wood for free. He's not taking back the wood that's already been cut. The problem is if the beggar wasted the wood he did take, and took for granted the assumption that he would always be able to get more.
Relying on something continuing to exist on the internet, without making your own backups of it, and you're going to have a bad time.
Your builds are broken because your build process made the false assumption that you could always get another piece of wood. You could have used the first piece of wood you were given for your build over and over, but you chose to essentially toss out the piece of wood every time your build finished.
> But let's not lose the perspective - the author does not owe us or npm continued support
On the other hand, he wanted his work published in the community registry where they got exposure and were made into dependencies in lots of projects.
When the author offered their modules and then suddenly walked away with them, lots of innocent devs who built their projects with his modules got hurt. He did more bad than good overall, especially that he unpublished hundreds of modules at once without warning, not just one. He should carry some responsibility for that.
The author can publish and unpublish all he wants from his personal site where there are zero expectations that it will continue to exist, but when he's doing it from a public repository where he received a lot of confidence from the community, he should at least make sure his users don't suddenly fall flat. Now people will start wondering if a module with millions of installs in the last month is still going to exist tomorrow.
> Now people will start wondering if a module with millions of installs in the last month is still going to exist tomorrow.
That's a smart thing for people to wonder when there is a very real possibility that it won't. I'm not going to applaud the author's action here, which I consider reckless, but it did bring attention to how fragile this "essential" infrastructure really is.
It is absolutely reasonable to be skeptical of every package. You probably shouldn't be on the bleeding edge of packages and likely ought to have locally stored packages.
You of course need to audit and improve your local store, but you need to do that with your dependencies anyway
> When the author offered their modules and then suddenly walked away with them, lots of innocent devs who built their projects with his modules got hurt. He did more bad than good overall, especially that he unpublished hundreds of modules at once without warning, not just one. He should carry some responsibility for that.
Blame NPM. They were the ones shutting down his module and giving it to someone else. That's the first of the crazy parts.
> Now people will start wondering if a module with millions of installs in the last month is still going to exist tomorrow.
That's a good thing. People should be aware of it - and be aware of the fact that all awesomeness of the open source world is maintained by continuous - and often unseen and unpraised - effort of thousands of people. Who are mostly known when something breaks but not much otherwise. It's ok not to think about it every minute, but it's also useful not to forget about it completely - and that we shouldn't take it for granted as it is in no way law of the universe, it is a consequence of continued effort and continued good will.
Doesn't this go both ways, though? NPM developed and maintains the software and servers used to publish and serve this author's packages, bringing the author fame, glory, and as the author is demonstrating, a serious megaphone, all at no cost to the author. If the author doesn't owe NPM anything, NPM shouldn't owe the author anything even for publishing these evidently valuable packages, right?
Taking it a bit further: npm is worthless without useful modules being published. So it stands to reason that module authors should be cautious because the company that runs it clearly won't push back.
I would say that sans a cease and desist, if not a court order, they shouldn't have done anything at all. Certainly shouldn't have turned over the name to them. Better missing and breaking builds then being controlled by someone with completely different intentions.
FYI I'm the one who republished left-pad after it was unpublished.
I think of it similar to letting a domain name expire. The original author removed the code and I forked it and published a new version with the same package name.
The main issue was there were so many hard coded dependencies to 0.0.3 so I asked npm support if they could allow me to re-publish that version and they complied since I was now the maintainer of that package.
* npm allows a module to be deleted
so all fetches of published name-oldversion fail
* npm allows you to take-over the name of a deleted module
and publish name-newversion
* npm does not allow you to "re-publish" name-oldversion
which had existed before the deletion
but npm ops forced this publish for your special case
That's a strange combination of constraints. If you're going to reserve the name-oldversion permanently, you might as well leave the content there too. (Except, I guess, for license compliance...)
Yes you got it right. The third point was the special exception / grey area.
For the record they made sure the exact same code was published to 0.0.3 so that I didn't maliciously inject anything. I control subsequent versions though.
Now that you own it what's to stop you from pushing out a new version with a slightly reworked string pad function under a commercial license (say a $100 per use fee)? Could make quite a pretty penny. Kind of crazy that this is possible at all.
That's not the point, of course no one would pay $100 for a string pad function. The point is they own the source of code that thousands of people will potentially pull from without realizing it's a different owner who might have different intentions. Not trying to say what they did is wrong or bad (it saved a lot of people's butts I'm sure)--it's more just highlighting the massive issue with NPM package publish & unpublish.
v0.0.3 is open source, though there is no license in the Github repo, so murky, but NPM says its WTFPL
Since old code is under a very permissive license, then the new owner could create v0.0.4 add code and make the new version closed with a restrictive license.
This is where a license like GPL would benefit overall, since all future code requires to be under the same license.
Either way, it seems like a dangerous policy to allow someone to re-own a previous owned and published module. Licensing is not the real threat but malicious code that potentially could be deployed.
Now I'm seriously wondering if npm is a goldmine for GPL infringement. If someone has an insane dependency tree with 1000 entries then there is bound to be at least some GPL in there forcing everything else to be GPL too.
Exactly--there's a great bug thread on the Atom text editor where they laboriously dug through hundreds and perhaps even thousands of dependencies to see if any had licenses that were incompatible with theirs. It looked like a complete nightmare and you can see the end result if you go to the View License menu in the editor--it's hundreds of little licenses.
With 1000 of dependencies, I would be much more worried that one of them are proprietary licensed (or was ripped from a place with no copyright license attached and then had a new license added the original authors permission). Imagine distributing a "$100 per use" software for a few years without paying, and how the court system would react.
I guess this is why debian takes licenses so serious.
Sadly, there's nothing insane about 1000 entries when it comes to npm. That's the reality of pretty much any non-trivial project, especially with React and Babel.
Except this very same string of events makes it quite obvious that versions aren't systematically immutable.
And vast numbers of people are suddenly shocked (shocked!) to realize there's no mechanical verification of stuff like this. It can all happen whimsically, and the result isn't just a loss of service, it's service with different results and no verification nor notice.
> Except this very same string of events makes it quite obvious that versions aren't systematically immutable.
From this thread it sounds like: a) npm specifically overrode their normal process to allow him to publish new code at the same version number b) npm did verify that the code he was publishing was the same as the old code at that version number as part of this.
So sounds like immutability of specific versions is systematically enforced by npm?
"npm" is two things in this picture: the server and centralized service, and the software tool on everyone's build/dev machines.
There's a difference between trusting a server to do something, and having the software you're using check that the server hasn't changed its behaviors in ways that may... fuck with the continuity of your zen.
If someone can decide to do a special thing that happens to break all promises, and my side of the tooling doesn't let me know, much less let me confirm acceptance of the changes, then my side of the tooling is seriously deficient.
And in a situation where this tooling then starts executing new code from a new author on my host as my user without any authentication except "trust the server"... the same behavior describes $script_kiddie_virus_of_the_week! "download code from C&C server; run it, thanks:D" What's the difference between this and outright RCE exploit? "good intentions"?
I'd rather have something more than "good intentions" controlling what code ends up running on my computer, thanks.
While I can appreciate the predicament this presents, do you not feel like you are going against the wishes of the original author by essentially overtaking him and publishing his code against his will?
The author's explicit wishes, in no uncertain terms, are that anyone can "do what the fuck they want to with it" [0][1]. I think when he did this, he gave up (willingly, and with a bit of profanity) the right to have any say at all about whether, how or by whom it was published.
Which is a nice gesture, but since the wtfpl seems to give up any pretense of ownership on the part of the author, it's also irrelevant. Anyone can do what they want with it. If he wanted to keep tighter control of the code he should have published under a more restrictive license.
And, if he wanted people to take the time to be polite and contact him through his github account, maybe he shouldn't have wrecked so many people's builds.
The "ownership" I assume is referring to ownership on npm. I was just saying that, based on that excerpt from the blog post, it sounds like the author didn't actually want to keep tighter control of the code, and that re-publishing the code doesn't really go against the author's wishes.
Someone was nice enough to write some software, that is clearly indispensable. They were nice enough to not charge money for it. They were nice enough to support it, again free of cost. They were also nice enough to open source it, such that if it ever became more convenient for you to fork/change/do whatever you want with, that you would be able to.
And when that same person is being bullied (regardless of legalities) and instead of building them up, helping them fight against their bullies, you decided to shrug your shoulders, go against their wishes, exploit the fact that its open source, and when asked if you felt bad about it, responded: (paraphrased) "Well... technically..."
On the one hand: Really? You don't even feel bad about it? Do you not see that it is machiavellian? You don't even care enough to use the age old "better of two evils"/"end justifies the means"/etc arguments? Bold.
On the other hand: Kudos! I don't think I would be able to pull it off like you do.
Are you serious? Nice enough to yank it without giving anyone advanced warning, (likely) knowing that it would break numerous projects? My guess based on the timing of his article and his petty argument with kik, is that he intended to cause this commotion to draw attention to his complaint against kik. He could have give the community warning so that major projects could have switched over to another package. So no, I don't feel bad for the guy, I think he's being childish.
You really publishing his code again is "going against his wishes"?
He made his point and got a lot of attention to the situation, he isn't responsible for the packages anymore. I would not assume that he wants further disruption or objects to people using his code.
Keep in mind that `String.prototype.repeat` exists in far fewer contexts than a simple loop. The whole point of using a module is that you don't have to worry about _how_ it works when implementing it in your project.
Sure. And it's easy to copy and paste into a few different projects that you maintain. What happens, however, when you decide to use multiple characters instead of a single character (to create a pattern for example)? Now your padding is two or more times as long as needed. So you write a little extra logic but now you have to update multiple projects to implement that same change.
I think what's great about external modules is:
* you only need to update the package file to get the benefits of new versions of dependencies
* you can call `require('left-pad')` instead of what can end up like `require('../../utils/string').leftPad`
* your code only describes _what_ it's doing, not _how_ (declarative programming)
> What happens, however, when you decide to use multiple characters instead of a single character (to create a pattern for example)?
Then you can’t use left-pad, because it doesn’t support that and you don’t maintain it. So you make your own module to suit your needs, which is a very good thing and returns somewhat to the actual point, which is that anyone can make their own left-pad that matches the existing module, so the author’s wishes wouldn’t have a huge impact even if they were that the module stayed down, which they weren’t. Phew!
If you want the flexibility of pushing minor updates the flipside is you need to provide a definition of what constitutes minor - at which point you should be on 1.x.
This is a surprisingly effective protest action. It got the attention of an incredible number of people very quickly, and the damage is mostly limited to wasting the time of a bunch of build cops.
I don't have much of an opinion on his actual reasons for protesting, but I do think it was a pretty cool protest.
The "attention" he's going to get is people annoyed at him for breaking their shit carelessly over a petty dispute with a third party. I have some stuff at work that uses NPM, and I don't know if he broke it, but my thoughts right now are "fuck that guy" and "I would never use NPM again if I knew how catastrophically badly designed it is" and more broadly "if the JS community is this amateur and petty I'm writing my next server in something else"
only in the most narrow sense. His cause, whatever it is, isn't furthered, but now everyone involved (including him) looks bad, and a lot of innocent people have had their time wasted fixing something that shouldn't have broken. It's a loss for everyone involved. He looks petty and untrustworthy, npm looks amateurish, kik gets bad publicity, and everyone using this junk is stuck with their face in their palms.
Don't blame the guy who pressed a button that allows you to unpublish a core module. Sure, it was done on purpose this time, but what if it happened by accident. Your "package manager" shouldn't be this fragile.
EDIT: For those with a short attention span, the first paragraph is a drastic metaphor, the second is making my point.
If a kid does not get cookies at home it can shoot all its classmates. Lots of media coverage, the kid will get "the attention of an incredible number of people very quickly".
NO.
For me, publishing code under a FOSS-license means giving back to the community. Anyone who then decides to cause collateral damage in that community was never in for the community in first place. Sorry, that sucks. It causes extra work to developers who rely on your code. They have to spend time to fix a non-issue. Time they could spend with friends and families. If I was a js-coder, the author would be blacklisted for life. If I was an employer, the author would not get permission to publish code created during work hours under a FOSS-license without the company having a private repo. Such a reaction actually costs real money to lots of people. And it is a great disservice to the FOSS-community because it sets a precedent.
Really, comparing this to mass murder, well done. It's more like he had a bunch of toys, someone stamped on one of them and he took the rest home with him whilst the other kids are still playing with them.
These are his repositories which he created, he is welcome to remove the code, and others are more than welcome to rehost (as they require).
If you are depending on npm (or any other build tool such as composer, whatever) or on Github, you really need to have a backup plan when the shit hits the fan.
There wasn't any equivocation in the parent between this event and mass murder.
The mass murder example is a counter point to the (implied) argument in the grandparent that "[getting] the attention of an incredible number of people very quickly" is the same thing as an effective protest.
It is possible to get lots of attention for your action without that attention translating to support for you or your cause.
The parent continues to discuss how they believe there was damage above and beyond "wasting the time of a bunch of build cops" and explicitly states what they think the damage would be.
Perhaps choosing a different example would have been more tasteful, and may have avoided this side discussion and being flagged to oblivion, but it did not ever claim the two events were equivalent.
It's possible that the mass murder example, while not directly compared with the original action, is implied equivalent by the mere juxtaposition of the two but I don't think that was the parent's intent.
Hyperbolic analogies are a perfectly valid tool for refuting absolute statements, in particular arguments of the form 'the end justifies the means'.
If someone argues that some $noble_end is served by $means_under_debate, as a justification for the means, that argument can be refuted by giving a circumstance where the same $noble_end is served by some $extreme_means where both parties can agree that $extreme_means is never justifiable.
Then once we've agreed that the form of that argument doesn't work, it is reduced to 'the ends justify sufficiently noble means', we can move on to discuss whether the $means_under_debate are sufficiently noble, setting aside the fact they they lead to $noble_end.
> publishing code under a FOSS-license means giving back to the community
As far as I can tell his code is still up on github. He removed it from a poorly implemented distribution channel on which many people happen to unwisely depend. I'm amused at how much people are concerned about this breaking apparently production code. It will serve as a good lesson to many inexperienced and lazy engineers, I suspect.
Azer has contributed awesome modules to the community, but such a move _obviously_ messes with a bunch of people who previously didn't trust npm, but Azer. Npm works fine. There might be issues with it, but the reason builds are failing right now is that he decided to unpublish all of them - in a move that feels very kneejerky, despite him claiming that it's the opposite.
If this had been actually in the interest of the community (because he thinks that npm isn't acting in our interest), he'd give people a fair warning. I could have lived with a "Hey, this was my experience, it sucked, I'll unpublish things in 30 days. Please update your dependencies." We know how to deprecate things gracefully.
Ironically, this demonstrates the same realization that he had, that when you depend on someone's modules, it is their "private land" too, and when they want to do something rash, they can do it and you will suffer consequences, unexpected and possibly undeserved.
It's the same issue we all experience, trusting people and institutions and finding out that that sometimes that trust can be violated.
> Ironically, this demonstrates the same realization that he had, that when you depend on someone's modules, it is their "private land" too, and when they want to do something rash, they can do it and you will suffer consequences, unexpected and possibly undeserved.
> but the reason builds are failing right now is that he decided to unpublish all of them
If builds are failing, it's because people did not set things up correctly to account for the fact that npm can and will go down or not be reachable. There are well known, well established practices for handling this. Frankly, if this affects your builds, you probably have bigger issues to address.
How does a 30-day notice work unless you have a way of reaching out to people using your modules? It just seemed so unrealistic that even 1% of people would actually see such a notice before things start to fall apart.
You bake deprecation notices into npm, to be displayed during install. He has more than 3m installs a month, if he really wanted to, he could have easily display a giant "npm sucks" banner during every single install. Same message, but it wouldn't immediately hurt people who trusted his modules.
Our build system eats deprecation notices with eggs for its breakfast. While I'm sure that they exist, I've never worked with a developer who paid much attention to deprecation notices unless they were looking to actively update a module.
I highly doubt if most continuous build/delivery systems actually have a person looking at the build output to take some action. That’s why, centralized build systems should not just let the packages disappear but unfortunately that’s not the reality. This is also the reason why there are languages like Go out there that do not rely on centralized package management.
I am obviously a old fossilized ancient developer. This situation seems like insanity.
not the unpublishing part. the part where the thing that you require to sell/publish/do your job isn't under control or isn't stored within your organization.
Am i wrong in thinking that you should just have a local copy of all of your source code dependencies. would it really take that much longer?
It's pretty common, to name only the ones I'm more used to Ruby has gems, Java maven, Perl CPAN, Node NPM, Python pip. All languages have package managers nowadays and has been like that for a long time. They give you the way to make local repositories but I don't know anybody doing it. Maybe developers working inside some companies have some process to go through the company repo. Setting up the repo and maintaining it is quite an overhead and somebody has to pay for it. But I agree that it's the sane way to work, maybe not the most profitable.
Everywhere I've worked has had a local install of Nexus or Artifactory which acts as a caching proxy in front of Maven Central, as well as being a location to store internal builds. Artifactory's pro version can sit in front of pretty much any package repository known to man (NPM, Docker, RPM, APT, Maven, PIP, gem, you name it it's probably in there).
NPM makes it quite awkward to use a non-standard repository, and often builds will download stuff from GitHub on-the-fly rather than hosting all their sources in NPM. It's not my favourite package manager :(. But you're going to need to put your own packages somewhere, and there's little reason not to make that somewhere not also cache your external dependencies.
Pulling packages from npm on every build in relying on it seems like fine for a little startup that no one would actually care if it went down for a day or two. That's fine.
But there are plenty of applications where it would be a catastrophic fuck up to break things because someone somewhere decided to delete something.
99% of developers will use npm directly, anyone claiming otherwise and also claiming that everyone else doesn't have a QA department because they do this is completely OOT and from a different decade.
This thread has massive upvotes and a mass of comments precisely because this is what everyone does. It's utterly ridiculous to claim otherwise.
Yes, I am from a different decade. I am still running things inside plain simple virtual machines, not containers, and using MySQL at day job. I am totally not a cool kid.
I'm using npm / browserify etc in anger for the first time today. This is a horrible issue to have run into and it's left a pretty sour taste.
The fact that it's possible for someone to unpublish 17 lines of js and break the install of major bits of infrastructure for everybody is pretty insane. It seems like at a minimum the dependency tree should be traversed to see what the flow on effect will be. Should it even be possible to revoke an old version that's depended on by other libs?
These few lines of js cost me a couple of hours tonight! :-/
I don't want to sound like a old grumpy man but here goes anyhow. I was looking into using node.js, react, etc after many years of writing web apps using Python and Quixote (obscure web framework like Flask). The whole Javascript technology stack looks pretty insane of me. Getting a working React environment requires a huge number of packages to be pulled down by npm. Browserify requires a bunch more. Recursive dependency resolution is nice and all but isn't this going to create a massive technological debt that needs to be maintained? Semver is not a magic bullet.
Also, seems like a security disaster waiting to happen (I don't use CDN because I like to make sure I review code before putting it on my important websites). Linux distributions like Debian have put in huge effort into making a packaging system that is secure and doesn't lead to dependency hell. You can argue how successful they have been but, and maybe I'm ignorant, I don't see the same effort and level of maturity in the Javascript/npm ecosystem.
As an alternative to React, I'm looking at Mithril. It's relatively small and has no dependencies. I'm quite a bit more comfortable building on that foundation as opposed to a giant house of cards.
> Recursive dependency resolution is nice and all but isn't this going to create a massive technological debt that needs to be maintained?
Spot on. Imagine deploying an application, in 2018, that pulls down 1000 libraries, 300 of which are 6 years old versions and contain vulnerabilities (or just bugs) involving data on transit.
Who is going to do all the work to backport fixes in every affected version of each library?
If things continue on the current path, I would be surprised if a typical 2018 node/NPM-based application only pulled in 1000 libraries (transitively). I just checked an application under development here:
$ ls node_modules | wc -l
517
Then I checked the current Angular 2 repository:
$ ls node_modules | wc -l
804
(Kudos to NPM 3 for finally flattening node_modules; in addition to reducing duplication and making life less miserable on Windows, it is now much more obvious how the transitive dependencies explode.)
Given the ongoing trend toward each dependency having more dependencies of its own, 1000 doesn't sound like much of a stretch. How many of those 1000+ will be up to date and lacking in critical security or functionality bugs? It sure won't be 100%.
It makes me look longingly at languages which ship with a reasonable standard library.
What dependencies are required? I would have thought that for basic, no-frills use, you could just load react.js and use it with no other dependencies. Are you referring to the tooling setup required for JSX? I've never used react, just curious.
react_demo$ du -sh node_modules/
25M node_modules/
react_demo$ ls node_modules/|wc -l
79
react_demo$ cat $(find node_modules/ -type f ) | wc -l
287767
I think I installed react, react-dom, browserify (globally), babel. The exact details are not really important, being inexperienced with react I probably installed stuff that is not strictly necessary. However, the ease of pulling in a huge amount of dependent code makes me concerned. For a long lived stable application you will have to maintain those dependencies (at a minimum, have 3rd party maintainers who you can trust to maintain it, not break your app, not inject security issues into your app, etc).
Have had the same experience tonight - browserify etc 12 modules, 75MB, circa 12,000 files. I guess it's just the npm way but it's definitely a bit shocking for those of us who are unfamiliar with it.
But are those strictly react dependencies? Seems more like a demo that was doing a lot more than just using react. Eg, browserify is mentioned, but that's a totally separate project from react, completely orthogonal. So inexperience seems like a big factor here..
It's the ecosystem that's being complained about here it seems, rather than react specifically.
Javascript has an almost non-existent standard lib. Obviously Python ships with most of the kitchen sink in core. You're not grumpy, but you're misguided.
A lot of people like to think that software fragmentation comes without any drawback.
Having many tiny libraries and/or multiple versions hurts in the long term.
"I don't want to sound like a old grumpy man". Hmm you failed :). On a serious note you might have noticed that there are 2 major camps of node users those who came from web frontend devs & ones adapting node at large scale enterprises/projects
(paypal, walmart, netflix, joyent) so as you might guess there are solutions to address situations like this (e.g. run your own registry etc).
> Recursive dependency resolution is nice and all but isn't this going to create a massive technological debt that needs to be maintained?
Keeping each package small and self-contained reduces the technical debt. If you depend on a monolithic library like Django, an update can be months of work with impact all over your code. If you depend on the same amount of code, but split into 100 smaller libraries, everything becomes much easier to maintain.
> Semver is not a magic bullet
It's pretty close. Having an ecosystem where everyone follows it religiously because all the tooling is built around it is really nice.
> Also, seems like a security disaster waiting to happen (I don't use CDN because I like to make sure I review code before putting it on my important websites).
It doesn't have to be. Reviewing all the code you depend on is a fool's errand IMO, but if you want to do it then NPM makes it very easy.
> Linux distributions like Debian have put in huge effort into making a packaging system that is secure and doesn't lead to dependency hell. You can argue how successful they have been but, and maybe I'm ignorant, I don't see the same effort and level of maturity in the Javascript/npm ecosystem.
The node ecosystem has been very good at avoiding dependency hell. It's well designed. They could stand to learn from Debian on the security front though.
Should one not be able to unpublish? What if there's a bug in a released version, and it's going to take a while to fix? Or what if you simply no longer wish to be associated with the organization anymore?
I don't think it's relevant - if there are other libraries depending on it, the code probably needs to stay. Maybe there's a bug but it's been worked around. The library has been up long enough for others to depend on it and publish their own code using it.
You could have a "soft" unpublish so the module is still available for any libraries that depend on that specific version but it won't be picked up in the "greater than" version case.
In terms of the owner wanting to have it taken down, there should probably be a licence flag that hands control over to the host so at the very least an audit can be carried out before removal.
EDIT some discussion rather than downvotes would be nice. In this case we have a library with a totally permissive license, used by screeds of other libraries, with 2.4M downloads a month. The pragmatist in me says that there should be some mechanism by which pushing a button shouldn't screw up everyone's day. Let's discuss potential solutions.
Rust/Cargo solves this with a `yank` option. Cargo will no longer automatically switch to that version, but if you already use it, nothing changes (though there might be a warning or something)
> if you simply no longer wish to be associated
> with the organization anymore?
Given that it's open source, (specifically WTFPL in this case), that's not something you can actually. Or rather, it _is_ okay for npm to republish stuff: that's part of the license.
> These few lines of js cost me a couple of hours tonight! :-/
Every dependency has some costs and some risks stemming from the need to trust in another person. If you're only using 17 lines of code, those costs dwarf the cost of code maintenance.
Though, there are probably ways to improve the package system such that less trust is necessary. Or even just ways to use the existing system better.
This feels very wrong to me. I know, open source, etc., and it's likely that the source license allows it provided the license remain intact, but still... For better or worse (worse, IMHO), the author decided to un-publish his nam modules. He asserted his authority over that package of his code. For npm to usurp his authority, even if the licensing allows it, feels wrong.
By choosing to publish his work as open source, he made it possible for npm to do this. One of the great things about open source licenses if that the original author decides to remove their work (regardless of whether on valid grounds or not), the community can still access it.
If he wanted the ability to prevent who is able to reuse and distributed his code, a proprietary license would have been a more suitable choice.
Nope, nothing wrong. The code he wrote is his, but he doesn't own the npm namespace, which is all anyone cares about here at this point.
But you know what also _feels_ wrong? people who had no involvement in this at all having their day get fucked up because of this one dude who _suddenly_, just now realized that npm isn't going to really help him out and did the internet equivalent of taking your ball and going home.
Agreed, that feels wrong too. But is npm supposed to be the recess supervisor in this metaphor, taking the ball back and giving it back to the other kids?
The particular license chosen for this module doesn't put any limitations on the code - the 'do whatever the fuck you want' license is as permissive as it gets. If the author doesn't like that, he might want to choose a less permissive license next time around.
Did they? Most open source licenses aren't revokable - you don't have to stop distributing the code just because the author asks you to stop. That seems especially true in this case, considering the author chose[1] to license left-pad under the WTFPL[2]. I'd say re-publishing the package counts as NPM doing "whatever the f* they want" with the code.
You can't retroactively change license. Sure, the author could push a new version with a new license, but the old versions would still be governed by the old license.
Yeah, I appreciate that, but if this becomes a general concern that npm can grant ownership to whoever it deems acceptable (even to the point of republishing existing versions), then could we start seeing e.g. WTFPL with npm exceptions style licence fragmentation.
The source could still be readily available to anyone to republish as they see fit, but only as a different name / version.
Not condoning it, just thinking that the original author surely has the right to do this if they plan ahead (judging by the npm backlash that has been building over a single entity holding all the keys some may be starting to think this way).
All open source licenses allow the licensee to republish your code - that's part of the definition. It's not going to trigger license fragmentation because any exceptions will make it not open source.
If you want to enforce that other people use different names for their forks, the usual way to do this is with trademarks - this is what e.g. mozilla and redhat do. Npm should respect your trademarks, and if someone else publishes a project under your trademarked name you can make npm take it down... which is exactly what originally happened here.
If you delete a module, the name becomes free. Someone else grabbed it and publishing code he has a license to redistribute under it. NPM is "only" fudging with the version number, the author gave the module away.
I can't agree. The author licensed the code under WTFPL, meaning he directly authorized anyone to do whatever with it in pretty much the most explicit and unambiguous terms possible.
Writing a 15 line function shouldn't grant someone the right to break an entire language ecosystem
I think the real issue though here is technical - does npm really allow mutation of published assets like this? That's a shitty situation that can only lead to unreliable builds
Pray tell, what is the minimum size that should be required before permitting a dependency? Constant-time string compare is only 5 lines or so, should we cutpaste security critical functions too? "We ran your code on $new_arch and discovered a timing attack." "It's not my fault!@# The professionally maintained and reviewed implementation was too small to depend on!@#!"
> First, you should never have a dependency for 15 lines.
If those 15 lines are called throughout your codebase, you should do what? Re-write the 15 lines and stuff them into a lib that you maintain? And everyone else should do this, too?
If a library is widely useful, it's widely useful regardless of how many LOC it contains.
Yes. With dependency you are giving up control and in return you save time in not having to implement the dependency. If the LoC is small you get practically nothing in return and so the only reason to do it is ideological.
Many times it is even faster to write a trivial "module" yourself than even discover it on npm, let alone reading and verifying it. Hell, NPM 3.x is so slow that it's literally faster to write any 1 liner "module" than it is just to install it.
"Reuse" is just a means to an end, it doesn't make sense to do it only for its own sake.
This is why you should vendor it. What is "it"? All of it, whatever it may be. You should be able to build your systems without an internet connection to the outside world.
I say this with no reference to particulars of your language or runtime or environment or anything else. This is merely a specific example of something that could happen to a lot of people, in a lot of languages. It's just a basic rule of professional software development.
I agree. I use and love NPM and others like it, but when it comes down to it, i check-in my dependencies when my applications get "released".
Committing the updates is only one more step, and in my experience it's not even another step since we already have a rule that new installs or dep updates need their own commit.
It becomes even more important in an "enterprise" environment. I have machines here which are blocked from executing node.exe (at the time, the binary I had wasn't signed).
I have machines here that have to authenticate via SSPI/NTLMv2 to a proxy to get out to the net. And don't even get me started on the SSL MITM we have here.
I can't run your startup's app here. I can't run your new build tool here.
And the endless list of risk reducing stuff like this (and not just in IT either) is exactly why big companies are slow. Big corps need figure out how to create small disconnected divisions instead of consolidating common services for cost savings. If one of your tiny divisions gets hacked it doesn't take down the rest unless your all on the same systems burdened by the same laundry list of security and policies.
I don't think "vendoring" in the sense of copying all your dependencies into your source tree is a good idea. Source and dependencies are different things and well worth using different tools for. You absolutely do want to ensure that your builds are reproducible even if the upstream libraries go away, but there are ways to do that without putting their source in your source tree.
I don't undertand why this isn't the most common reaction, maybe people don't want to take this much responsibility over their builds since they've grown used to it being "taken care of" by these central package repositories for them?
Still relying on internet connection for builds to pass seems rather silly to me.
Or have a decent package management setup? Or just use an existing one, it's been working for 20 years. Package management is a solved problem, just use an existing one and move on.
@nj48 is just some troll who took all the names. Not sure the intention of @nj48. The hijacked modules are not dangerous at this point, but that could easily change and cause serious issues.
I've never felt good any time I have to use node modules and see this gigantic stream of dependencies come flying down. It's even more painful when you need to assemble license information for your software and crawl through _every single dependency and all of their dependencies_ to find their licenses, etc. to check they are OK to use in your software. Just look at the View License info in the Atom text editor some time for a truly insane wall of text (over 12,000 lines!!). IMHO the entire node / NPM system is seriously flawed with so many tiny dependencies for trivial stuff.
I think that unfortunately this was a foregone conclusion. Copyright law, like most other laws in our society, favor corporate interests.
I support his stand on principal, however. Azer is a talented developer and has an impressive life story, and has certainly contributed more to society than a social network well know for invading children's privacy.
I don't think there's a problem with trademark law here, I doubt that their claim is legally enforceable against the name of an npm module, as it's pretty obviously outside the goods and services which their mark is registered for. The problem is that npm are capitulating to what I suspect is an invalid request. As is often the case, people are scared of lawyers and forget that they have rights.
Copyright law is not at all related to this. It was a trademark dispute. You might've fallen into the trap of grouping several unrelated laws into "Intellectual Property", a misnomer which confuses confusion when discussing such laws.
You start a project with the same name as a company, which owns the registered brand and are surprised when some 3rd party complies with legal suggestions to make an adjustment?
Seems kind of silly to expect that NPM would want to fight for your project name when you didn't seem to do your own due diligence when picking a name. Also, a bit backwards to go remove all your modules as well, therefore breaking builds.
Is that really silly? Coming up with a project name in an unrelated project in the technology space? That's akin to Ajax Detergent (https://en.wikipedia.org/wiki/Ajax_(cleaning_product)) wanting all jQuery wiped off because there is a $.ajax() function in jQuery. Look, it even has a $! Chaching with every occurrence!
It's not an unrelated technology space! Kik is a messaging platform. It's EASILY conceivable that they might want to publish a nodejs API. It's also easily conceivable someone looking to interact with their platform would type npm install kik. They're entirely in their rights to see this as stepping on their turf.
A trademark gives you protection of a mark within a specific industry. "Software" is not a specific industry. They do not get wholesale ownership of the term.
And if you blindly install a module, expecting it to be some official release without looking it up, you're an idiot.
See class 9 of the Nice categories. If you claim a trademark For class 9 it includes "all computer programs and software regardless of recording media or means of dissemination, that is, software recorded on magnetic media or downloaded from a remote computer network."
Also mind the way this practically works: A company will try to register a mark in as many categories as possible.
"Scientific, nautical, surveying, photographic, cinematographic, optical, weighing, measuring, signalling, checking (supervision), life-saving and teaching apparatus and instruments; apparatus and instruments for conducting, switching, transforming, accumulating, regulating or controlling electricity; apparatus for recording, transmission or reproduction of sound or images; magnetic data carriers, recording discs; compact discs, DVDs and other digital recording media; mechanisms for coin-operated apparatus; cash registers, calculating machines, data processing equipment, computers; computer software; fire-extinguishing apparatus."
Computers programs are a part of 090373 which has a breakdown of ~100 subcategories.
There are 45 Nice classes. Being in the same top-level class is not confusingly similar. Unless you think Kik Marine Compasses and Kik Instant Messenger might be confusingly similar.
a) from what I understand, his project was there first.
b) NPM shouldn't have to fight it unless they are requested to, the trademark claim was ridiculous to begin with. Regardless of the claim, enforcing it would have taken years... I'm not saying NPM shouldn't have comply with the request and rename the package but definitely could/should have handled this better.
c) the guy wrote: "NPM is no longer a place that I’ll share my open source work at, so, I’ve just unpublished all my modules." I think it's pretty clear why we pull all of his modules.
Regarding (b), did NPM itself have any legal liability? If not, they could simply say "your beef is with the author of the package, but we'll of course comply with a court order that requires us to transfer ownership".
(I suspect, though, that NPM might have some legal liability, and Kik [the company] could sue them regardless, which would suck for them.)
IANAL but looking at kiks trademark [1] I'm not so sure sadly.
>Computer software for use with mobile devices, namely, computers, personal digital assistants (PDAs) and mobile phones for downloading, displaying, transmitting, receiving, editing, extracting, encoding, decoding, playing, storing and organizing text, sound, images, audio files and video files
Organising text is what Kik (the project) does, so it would infringe the trademark?
> the trademark claim was ridiculous to begin with.
npm's lawyers disagree with you, and they're lawyers.
EDIT: cool HN, -2 for stating a fact. Sure, I don't disagree that this lawsuit is kind of silly, but npm's laywers don't think this suit is _frivolous_, which is what matters.
Given what I know about your politics and philosophy, I find it pretty funny that I have to say this:
1) As "just a fact" it's irrelevant. With some interpretation added, it contributed something. However...
2) As participants in a market economy, the lawyers' primary interest may not be the letter of the law or what is theoretically winnable in court, but what will give npm the least hassle. I'm not sure if that's the case in this particular trademark case, but we all know lawyers sometimes do counsel the path of least resistance. And maybe that's even wise. But it's worth being clear about that ambiguity, rather than just saying "lawyers said it, I believe it."
P.S. I think your comment isn't super helpful but I didn't downvote you.
My point is mostly that often, when it comes to law, lay-people talk about what they _wish_ the law was, rather than what the law actually is. And yeah, lawyers can be wrong too. But sometimes, things that seem common-sense aren't actually legally correct, and this is one of those cases. It does feel silly that a messaging company can threaten to sue over an unrelated software package, but that's just part of how intellectual property law works.
> My point is mostly that often, when it comes to law, lay-people talk about what they _wish_ the law was, rather than what the law actually is.
That is why the law should be formalized such that correctness proofs for argumentations can be given and in doubt even be checked independently by a computer. Exactly because of the possibility of different opinions and wishes, coming up with such a high standard should be considered a maxim.
My background is in 20th century Anglo-American philosophy, which spent a great deal of time seeing how far one can push formalization or quasi-formalization of interesting concepts. I wish I had a good capsule version of why I think this won't work, but it won't. Formalization is a tool, and an important one, and there probably are areas where a more formal approach to law could pay off. However, attempting to remove all ambiguity, vagueness and subjectivity is liable to leave you with paradoxical results.
Perhaps you could start with this: to formalize any set of laws and criminal procedure similar to the actually existing law, you'll have to define knowledge, and that is a quagmire (http://www.unc.edu/~ujanel/Gettier.htm--you don't have to read the entire paper, but at least read the first section for a feel of how nasty the project gets. If you need more background, here is the description of what the Gettier problem is: https://en.wikipedia.org/wiki/Gettier_problem).
I just read about the Gettier problem: In my opinion one should better model knowledge as some kind of estimators for probabilities and being justified on some statistical criterion. This should in my opinion avoid the whole Gettier problem (but perhaps introduce some completely different ones?).
I prefer the definition of knowledge given by Orson Scott Card, if I'm not mistaken: something you believe to be true. It doesn't have to be justified and it doesn't even have to be true - you just have to believe it.
I know that the Moon is about 400,000 km away from the Earth. I have never measured the distance and I haven't even tried to check if it's plausible: I read it somewhere and accepted it as such.
While I can appreciate this sentiment, I'm also not sure that removing any sort of interpretation is a good idea. Look at the horrible impact mandatory minimum sentencing has had, for example. Flexibility can be bad, but it can also be very good.
There are a near-infinite set of complexities in legal cases. Trying to create a formalism for such laws would require solving the entire field of ethics to decide at which point something might be considered "reasonable doubt".
Which means if they say "it's very serious threat" and the code gets deleted everybody would think they are serious lawyers even if the claim looked ridiculous - because you see, lawyers said it's serious and who are you to argue?
On the other hand, if they ever say "it's ridiculous" and then they get sued, they could be fired or at least suffer damage to their reputation for not preventing it.
Thus, there is exactly zero incentive for the lawyers - at least ones acting purely according to near-term incentives - to ever take even a minimal risk and declare anything like that ridiculous. They have all incentives to always react as if it was 100% genuine infringement - because that costs very little and has no risk.
Some lawyers and companies are more civic-minded and reject some ridiculous requests and take risk because that's the right thing to do. But you can not require people to be heroes.
On the other hand, we are not lawyers, so we have all the incentives to proclaim how ridiculous the whole thing is.
There's a huge different between an actual lawsuit and the threat of a lawsuit. Kik's lawyers were being bullies, and NPM caved in, instead of standing up for one of their users who seem to have contributed a lot to the community.
3 letters of a NPM developers package? for real?? I'm pretty sure the owners of Kik wouldn't care that that developer package is named Kik. I can find 1000s of mentions of Kik online which validates the trademark. Starting from @kik on twitter to an online project of a random student.
Pursuing a NPM package names?????? for real?!
If you've ever worked with lawyers you would have known that they would always be on the conservative (safe) side without understand the business implications.
Sorry, but was there actually a lawsuit in this case? It sounds more like KIK emailed some people at NPM and NPM just said, "OK", then replaced a known module with some other thing.
It's only a matter of time until NPM is socially engineered into replacing a module with something more malicious, if it hasn't already happened.
How do you know what they thought? It seems just as likely to me that they didn't see much merit in the complaint, but didn't think it was worth fighting since IP law is such a mess in the US that even clearly baseless complaints can drag on and become expensive.
trademarks do not work this way. in order to infringe on trademark, you actually have to be in the same business. OP did not want to fight a legal fight, which is understandable. And NPM can on their own discretion to do whatever they want, and they prefer to avoid legal battles as well.
I've certainly benefitted from the vast ecosystem of npm. I greatly appreciate the work that goes into making this ecosystem what it is. However, I think we need to be a bit more critical when it comes to acquiring dependencies. Especially authors of very prominent packages.
Fun fact: one of my projects (a web api) depends on over 700 unique name/version modules.
If NPM wants to stay relevant and a serious contender, they need to have more clear policies in case of IP issues. In this case, the companies weren't even in the same space. Republishing someone's package who has chosen to unpublish and leave your platform is akin to Facebook resurrecting a Facebook profile because they had a lot of friends and the social circle ripple effects would be too high for feed quality for other users, so they chose to reactive the account AGAINST the author's wishes. WHAT?!?
We need an open source NPM alternative, yesterday.
> We need an open source NPM alternative, yesterday.
NPM is open source.[1] You don't want an open source NPM, you want a distributed consensus NPM where nobody has absolute power to censor or destroy other peoples work.
It seems like a prime use case for a DHT, possibly with some blockchain functionality identifying ownership of namespaces that people can reserve, like namecoin.
I was referring to NPM as in the npmjs.org project, with is a repository, user management platform, etc.
Very interesting suggestion in creating a blockchain NPM hosted alternative, with namespace ownerships.
As best I can tell, before NPM restored it, Azer had transferred ownership to somebody else, who wanted to republish the old version but couldn't. So they do appear to be respecting the maintainer's wishes.
That's not how the dependencies in npm work, for most who explicitly replied on owner/project-name notation. Even worse, when unpublished, someone (i.e. malicious actor) could have registered a global name for this or one of many hundreds of packages he wrote, and inject malicious code in all projects from babel to react to my side projects. NPM really got caught with their pants down on this one, and veered into the wrong by opting to unbreak tens of thousands of packages vs. doing the right thing.
Wow, very interesting post for me. Earlier today, at work, we ran into an issue where `npm install` was failing because the `shuffle-array` module wasn’t found. Investigation showed that the cause was that it was unpublished today. We found that this was a required dependency of the `match` module and this was in our dependency list in`package.json`.
We investigated and found out that it had been erroneously committed — it’s actually a memory game and has absolutely no place in our webservice project. :) (Mistakes happen… dependency audits can be worthwhile!)
Now, some hours later, I found your post on HackerNews and was really shocked to see, hey, this is exactly why it was unpublished. Quite a chain of events. Never thought I’d figure out why the modules were unpublished, but now I get it! Thanks for the explanation.
funny thing, but assuming that kik is related to kik.com
if you look here http://dev.kik.com/build/, they promote their own server eg. "Our open source web server Zerver can help serve your cache manifest properly, as well as doing other speed boosting stuff like automatic style inlining."
I had my suspicions that this was all motivated by wanting to grab the npm package name "kik" and the fact that they use node and npm almost cements that for me. I have a feeling this was never about protecting trademarks at all.
I think it's amusing to see this from the perspective of the company. Some guy uses your trademark without your permission so you tell him to knock it off. He refuses, so you go around him, and so he protests... by fucking over all of his users. In a dispute that doesn't involve them. And people are celebrating this.
I for one wholeheartedly support him. The more collateral damage he causes, the more people are awareness he raises. While the solution to his problem is unclear at this point, it'll at least get us talking, and find out what all of the stakeholders can do better next time, and we as a community to prevent this from ever happening again. I entirely supports his actions.
"awareness". Awareness of what? His trademark infringement?
Imagine this from the perspective of a hypothetical user. He's cranky and on a tight deadline, and suddenly his project is broken because some dumb dependency three layers away that he's never even heard of broke because some guy decided to throw a tantrum, and now his software is broken and he has to emergency rewrite part of it. All because this guy didn't bother to check if "kik" was a name he was allowed to use.
> "awareness". Awareness of what? His trademark infringement?
Alleged trademark infringement. Since his project was unrelated to Kik's business, it is questionable whether it was actually trademark infringement or over-zealous lawyers.
Awareness of how fragile npm is, both in terms of trusting it to always be available for your deploy but also on the mutability of packages there. What if someone unpublished one single well used package and someone else snapped it up to publish malicious code, it could have taken months before somebody noticed.
You missed part of the story. From the article, it sounds like npm decided to play the "We're a private company" card and was threatening to take control away from him, so he decided to disassociate himself from them. And because of the way npm's system works, this broke other packages and the new maintainers had difficulty fixing the problems.
Bleh, involving brand names or trademarks or whatever on a library that doesn't even seem to be commercial is a pretty dick move to begin with.
And his protest is against npm, who allowed someone to appropriate his work. The users are collateral damage, but really, can you blame the guy? Given the circumstances, urging users to ditch npm isn't a bad idea.
The thing about trademarks is if you fail to defend them, they can be considered "weak" and you can lose your rights to it. If you have a trademark and it's important to you, it's in your best interest to defend it so you don't lose it. It doesn't matter if it's a "dick" move or not, or if the infringement was non-commercial, he was messing with their trademark, and if they want to keep it they have to actively defend it.
Re: the common refrain that always pops up with these stories about how companies are "forced" to defend their trademarks lest they lose control of them
The issue I have with this argument is that, while I accept that it's largely true that once you try to obtain a trademark in a given domain that you have to defend it, nothing is forcing anyone to overbroadly define the reach of their trademark. Kik is a mobile communication app company without a household name, whose name is an obvious web2.0 style construction (so not unlikely to be replicated by chance) who asserts that their trademark applies to the entire software industry[1]. Kik elected to enter that difficult to defend position, and that's a common decision among companies that get negative press for overaggressively asserting trademarks. These companies are culpable for the social impact of their strategic decisions, and so I have no issue with blaming them for the results.
Asserting a not particularly creative trademark like Kik for the entire software industry was an aggressive stance in the first place, so I don't think that the apparent fact that you can't let competitors blatantly use your trademark without defending it absolves them of culpability here.
When you try to defend a trademark against a tiny, not-for-profit non-competitor, you come off as an entitled bully. Companies should take that into consideration when staking out their trademark territory. It works for some (Trump's hard to avoid here, and at one point it looked like Palantir was using the "I don't care if people don't like me" thing to great gain), but if that's not the DNA of your company, then you should consider whether you're biting off more than you can chew in your trademark assertions. If you can handle the heat of that kind of aggression, then more power to you, but don't come crying about how you are obligated to defend your unreasonably aggressive initial position when you get a little negative press.
[1] Which is almost a misnomer. The software industry is more of a meta industry that touches essentially every other industry these days. This underscores just how aggressive a claim it is that a given trademark covers the entire software industry. It's essentially saying that undue market confusion would result if the name were used for a mobile game, a text editor, an aerospace software testing product, an ETF platform, a medical devices operating system, an animations library, or even a platform for programming robotic legs to kick nearby objects [which would obviously be a more suitable home for the trademark of the identifier 'Kik' :D ]
[edit: grammar]
[correction: @overgard: Jeez, I meant to transfer this comment to a top level comment, because there's nothing wrong with your comment here, it's just related to a sentiment that has seemed to be subtly off to me for a while in a way that I couldn't put my finger on. Reading your comment just kicked off that old thought process that has been churning in my mind for a while.
I didn't intend to tee off on this specific comment, because I really don't have any problems with this specific comment, the aspects that I'm complaining about are really outside the scope of what you are saying.
But now that I know that I forgot to transfer it to a new comment window, and instead responded in this thread, the beginning, which was meant to represent a stronger general argument, looks condescending to me (inaccurately generalizing someone's comment in direct response to them? Ugh... that is sooo not what I was going for). This wasn't at all my intention, and I'm really sorry if it gives that impression. I apologize for any confusion or negativity that the error may have caused.]
Hey since you love authority, you realize that NPM decided to change the ownership of the kik module without asking his permission right? Life isn't black and white.
About a year ago I tried to unpublish a version of a library I'd pushed to Elixir's hex.pm package manager but the API rejected it. Turns out they only allow you to revert publishing for an hour after you push.
It was a little inconvenient at the time but in light of this I can very clearly see the wisdom of that decision.
This is a trademark dispute, not a copyright dispute. (And since Kik the company doesn't market a product in the field of command-line programmer tools, whether they have a legitimate case is arguable.)
There are other distribution architectures, both organizational and technical, which would be more resistant to IP claims, especially frivolous ones.
Small modules they say. Small standard lib is ok they say. Just going to point out that in a lot of other languages, string padding is just built into the standard lib.
brouhaha, this is why you should not put node_modules into .gitignore (same for PHP's composer.lock and vendor/ folder).
To be honest, I have waited for something like this to happen so that people finally wake up and realize how deeply and truly compromised the JS ecosystem really is. 11 SLOC not available any more and all over the internet builds are breaking etc.?!
And please, why isn't essential stuff like this in the JS standard string library?
> brouhaha, this is why you should not put node_modules into .gitignore (same for PHP's composer.lock and vendor/ folder).
You should obviously not do that if you are writing a library. You shouldn't publish vendored modules on npm. If everybody did this everybody would end up fetching 100MB packages.
Better, you should run your own NPM repository, which transparently caches packages from the main repository. Unfortunately, that's not nearly as easy with NPM as with (say) maven -- the only turn-key solution I'm aware of is Artifactory, and then only in the pro version, and configuring npm to use it is a bit awkward.
It sums up the biggest issue with npm. Modules shouldn't be a name but a namespace + a name , just like composer. Someone shouldn't be able to have a monopoly on names like "web" or "async". It should be "some-namespace/module-name".
Does this mean that I can no longer safely run `npm update`, or ask anyone to download my Node.js project and tell them to run `npm install`? Because the npm repo has in effect been compromised and is unsafe to use, until further notice?
That's what I'm assuming right now anyway. I'm not going to upgrade any Node.js dependencies or run `npm update` or tell anyone to run `npm install`.
Even if those trademarks would include a tool like kik, it is completely brainwashed to enable trademarks for three letter words and enforcing them on software packages names.
What are we supposed to type for package names in 10 years. 'abshwjais_kik', or will it be hipster to use unicode like in 'κικ'.
In addition to what other people have said, the exact same thing could have happened even if the project was only hosted on Github. Github could receive a DMCA notice, take down the repo and buf.. gone.
Because there's no version control that way. If I depended upon babel/babel (or whatever it's called on Github), when it changed from v5.x to v6.x, it would have broken my build.
Seems odd that a patent lawyer is being involved in a trademark dispute. Also, given the fact that he didn't make any money off it, I severely doubt that it would ever go to court.
Trademark disputes (unlike copyright and patent disputes) are often and need not be about money. US trademark law also has a fun "use it or lose it" provision that requires trademark holders to actively defend their mark.
The package author decided to unpublish the package. A new author has now stepped in and re-published the package (yay open source!) and deps are fixed.
We're sorry for our part in creating the impression that this was anything more than a polite request to use the Kik package name for an upcoming open source project.
My congrats and respect for his decision. Through actions like this companies might understand that the current trademark (and patent) law is only benefiting lawyers.
And wouldn't it be wonderful if as a result of this the build for KIKs Pedo API are broken?
I don't think Kik is the bad guy here. This npm module was rather new (<6 months?), while Kik Messenger has been around for years, and is VERY popular with the young crowd. They are both software. It would be like the author naming his module 'imessage' or 'spotify', except this is with a company that isn't as visible to the HN crowd.
I personally think him not knowing Kik existed was odd, and not googling the name at all even odder. Even still, I think Kik's response and npm's response were perfectly valid.
Looking at the voting of the comments here makes me sad for what has become of the HN community.
The number of coders complaining about an author exercising the basic of intellectual property rights is too high.
1) all coders should understand authors right be the code free or closed;
2) there is no excuse for someone whose value is based on creativity to ignore how IP works (the good and the bad part) because our comfortable incomes come from the protection these rights gives to our work
3) if your code is broken for 11 sloc, maybe you depend too much on others work and you have no value yourselves.
Benevolent persons sharing their code on their free time owes you nothing.
Repay authors whose code you use.
Buy closed source software, and at least respect the free software authors. It costs you nothing already.
> 3) if your code is broken for 11 sloc, maybe you depend too much on others work and you have no value yourselves.
That is very out of touch with the reality of development on the modern JS stack. Babel is a key transpiler that allows people to target multiple browsers with consistent JS, and it was completely broken by this move. Are you suggesting that everyone rewrite their own babel? I guess if they don't they have no real `value`.
I worked in ad industry. We had are own 3k lib with code inspired by jquery to deal with that problem of "fixing the browsers incompatibility", because vanilla JS does ,not exists when DOM/browser functionalities are added to the equation.
I learned to use CSS without framework, and it helps a lot both writing efficient CSS selector and fast loading HTML that is reactive ... and maintainable easy to fix/deploy CSS. Once you learned it.
Yes I suggest that it can be done. But I cannot do it anymore because no modern coders want to learn all that. Thus my CTO, and colleagues always complained that it should not be done this way.
Well ... I used to complain to much dependency is fragile, vulnerable and they say, let us follow the hurd.
I say, your code is expensive to write, maintain, support.
They said no. Else we find no one, and no one wants to learn this ol' shit.
Look where we are today?
The mass can be wrong I guess. Being right makes you jobless but at least when the bubble will explode I will be valuable again. And I will make people pay a lot for my skills of having no skills in babel, angular 2, react ...
The good outcome of this nasty incident (especially with the followed hijacking) is that it was a wake-up and (hopefully!) package managers will take notice!
Assuming it's kik.com that complained, the complaint to take down the kik NPM module seems legitimate. They've clearly been around a lot longer, are known by more people, and are in an overlapping market.
It seems like a lot of people would expect a kik module in NPM to be related to the company in some way, and it wasn't.
Was that lawyer overreaching? I don't know. But for this guy to expect npm to use their resources to defend him (which they may even possibly lose!) and get mad at them is... a bit presumptuous? Github isn't open source either so is he going to get mad when the lawyers send them an email about kik?
>Are you a developer? Kik has open-sourced tools and libraries to help you create great web experiences that can be discovered and instantly shared by Kik's 240 million users.
Rather sounds like a valid use of the law, not necessarily nice but valid. They are in the same space, open source web development.
It's so easy to avoid name collisions that it doesn't even make much sense to think about it. And trademarks aren't even a feature of the capitalism he so seems to hate: OSS uses names for the same marketing purpose as companies do and larger projects register those trademarks.
NPM Inc like to pretend NPM is infrastructure so they should act like infrastructure. Reinstating the unpublished version of the unpublished module was okay. Taking down a package over an unclear trademark dispute without also informing the author and the public about their reasoning was not.
For all we know the claim may have been invalid and npm just doesn't want to risk legal expenses but because they made the decision in private we won't know until they release the prepared statement.
Open source community needs to aggregate a list of lawyers who will consult on these sorts of things (related to the community at large) pro-bono. This way all parties on the open source side can feel a little less pushed around and bullied and a little more protected.
The best part would be to learn that the claim was not valid in the first place. At the very least, having representation would provide for some wiggle room where you can have days if not weeks to resolve the issue, instead of feeling you have to take immediate action.
And what if in the judicial system of the country that the programmer lives in there is a different legal situation than in the country that the Node.js Foundation (the US)? And what if there are npm mirrors in third countries where the legal situation is different in a third way?
Can we talk about how patents own namespaces? If I have a little "kik" soccer tournament that no one knows about, then it's fine. As soon as the namespace collides with the HUGE, vastly connected internet, it's a "problem".
In all honesty, I think npm handled this whole situation poorly. I'm not excusing the dev's behavior but I take issue with npm removing the module as well as arbitrarily restoring left-pad (with the cop-out of same name, different dev) WTFPL notwithstanding.
For those defending Kik's trademark, its like MS trying to trademark the word Windows. Didn't work that well for them either.
Since these libs are now baked into various package.json configuration files (some with 10s of thousands of installs per month, "left-pad" with 2.5M/month), meaning a malicious actor could publish a new patch version bump (for every major and minor version combination) of these libs and ship whatever they want to future npm builds. Because most package.json configs use the "^1.0.1" caret convention (and npm --save defaults to this mode), the vast majority of future installs could grab the malicious version.
@seldo Is there a plan to address this? If I'm understanding this right, it seems pretty scary :|
[1] https://medium.com/@azerbike/i-ve-just-liberated-my-modules-...