
Time for developers to address the chaos of Node.js and NPM - intvocoder
https://www.techrepublic.com/article/why-its-finally-time-for-developers-to-address-the-chaos-of-node-js-and-npm/
======
gringoDan
An additional element adding to this chaos is the security vulnerability of
npm packages. How often do we _actually_ examine the package's code in detail
before including it in a project?

Relevant article: [https://hackernoon.com/im-harvesting-credit-card-numbers-
and...](https://hackernoon.com/im-harvesting-credit-card-numbers-and-
passwords-from-your-site-here-s-how-9a8cb347c5b5)

~~~
deusex_
How is it different from maven deps?

~~~
kkapelon
Usually you don't use maven deps directly. You have a registry in your company
that caches maven packages for you and runs vulnerability scans on the binary.
So the binaries you end up using are "tested" internally.

Is that how people use NPM?

Also maven central does NOT support unpublishing of binaries. So if you have
version 1.2 of something and you have scanned it once for vulnerabilities, it
should always be the correct one. Nobody can publish again 1.2 in the central
repo

~~~
kabes
In every somewhat serious company I've worked they used a local npm cache like
jfrog (usually same one that's used for maven) that can get scanned for
security. So yeah, that's how people use npm.

Unpublishing within those companies is usually not allowed, in the official
npm registry you can only unpublish within the first 24 hours of publishing,
after that you need to request it by email.

~~~
kkapelon
>In every somewhat serious company I've worked they used a local npm

If that was the case for everybody, then left-pad would not have caused a
problem for anybody. People would not notice anything wrong with their builds.
The amount of github comments where people say that their production build
just broke because of a missing package implies that people are _NOT_ using
NPM with a local registry

>in the official npm registry you can only unpublish within the first 24 hours
of publishing

I don't believe this to be the case
[https://news.ycombinator.com/item?id=16087024](https://news.ycombinator.com/item?id=16087024)

~~~
twelve40
Just because it's a shitty common practice doesn't mean there isn't a proper
way to do it (local repo). For example, there are tons of people who check in
their secrets into public and private Github repos (including Uber eng), then
they get jacked and complain. If they were smarter than that, "it would not
have caused a problem for anybody". Doesn't really prove that what a lot of
people do must be automatically good.

~~~
kkapelon
True, but the fact that NPM seems to be the only repository where packages can
be "re-published" further exacerbates the bad practices.

[https://status.npmjs.org/incidents/41zfb8qpvrdj](https://status.npmjs.org/incidents/41zfb8qpvrdj)

The fact that 9 packages could be "published over" _after_ the left-pad
fiasco, shows lack of attention.

Tools should try to foster good practices, instead of worsening bad ones.

------
stephenr
Related: 3m installs in 7 days, of a “module” to check if a number is odd.

[https://www.reddit.com/r/programming/comments/886zji/why_has...](https://www.reddit.com/r/programming/comments/886zji/why_has_there_been_nearly_3_million_installs_of/)

------
arenaninja
It's troubling that PHP gets so much shit because it has an extensive if
clumsy standard library, where node gets a free pass because you can get
whatever doesn't exist from NPM, welcoming any number of security
vulnerabilities into applications.

I recently cleaned my NPM cache at work after 9 months, it was 6GB. ES6 is a
massive improvement, but Node's standard library is still lacking and the
sheer amount of tooling needed to be effective and up to date is too much.

~~~
slowmovintarget
PHP isn't criticized because it has a big and clumsy library. The criticisms
for PHP are very clear: [https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-
design/](https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/)

It is badly designed.

~~~
arenaninja
The fractal blog post will be 6 years old in a couple of weeks. I'm going
through it again, and __a lot of it is about the big clumsy library__. Some
choice excerpts:

> There’s no clear design philosophy. Early PHP was inspired by Perl; the huge
> stdlib with “out” params is from C; the OO parts are designed like C++ and
> Java

There's even complaints about PHP 5.2, but PHP 5.4 had just been released a
month earlier:

> If PHP was compiled with --disable-url-fopen-wrapper, it won’t work. [...]
> Note that this flag was removed in PHP 5.2.5.

> foo()[0] is a syntax error. (Fixed in PHP 5.4.)

> PHP errors and PHP exceptions are completely different beasts. They don’t
> seem to interact at all. (Fixed in PHP 7)

> There is no finally construct (Fixed in PHP 5.5)

> Methods cannot be named e.g. “list”, because list() is special syntax (Fixed
> in PHP 7)

I'm going to stop there. Not saying any of it wasn't valid criticism or that
some of it doesn't apply to this day. Just that too many people, yourself
included, are happy to throw a 6 year old blog post at a language that has
continued to evolve since. It's also strange that you say it isn't criticized
for a big and clumsy library, because in the link you provided about half of
the page is complaints about the library.

~~~
wvenable
> I'm going through it again, and __a lot of it is about the big clumsy
> library__.

And even a lot of that isn't fair because PHP has always integrated C
libraries directly; which was a huge strength when it was new. A function like
mysql_real_escape_string isn't a PHP function, it's a MySQL function (
[https://dev.mysql.com/doc/refman/5.7/en/mysql-real-escape-
st...](https://dev.mysql.com/doc/refman/5.7/en/mysql-real-escape-string.html)
).

------
Zelphyr
When I was checking out Serverless Framework I was floored that a single
Lambda function (that barely did anything) resulted in 15+Mb of data being
uploaded to AWS. I had only installed two NPM modules yet when I looked at the
"node_modules" directory there were nearly 300 entries in there. Things like
Redux and GraphQL, neither of which I ever intended to use in a simple test of
Serverless Framework.

When Node and NPM first came out and I thought, while not perfect, they were a
really promising set of technologies. Now, NPM specifically is something I
will go out of my way to avoid. And by virtue of the fact that NPM is so
closely tied with Node, I tend to avoid Node as well. Which sucks because I
have always enjoyed programming in Javascript.

If anyone has come out with a curated alternative to NPM, please let me know.

~~~
pedalpete
Though I agree with the article, and can appreciate that somebody just
"checking out Serverless" (I'm assuming you were also new to Node), it just
takes a little bit of awareness when installing packages.

It took a while to learn this lesson, but it pm;u tales a few seconds in the
process of deciding if you want to use a package, to check out it's
dependencies, I also check out the git page and license. Is it active, is it
open-source, does it have far too many dependencies, is it in regular use.

[https://www.npmjs.com/package/serverless](https://www.npmjs.com/package/serverless)

That's serverless. I was surprised to see it uses graphql as a default, and
looking at graphql, how many dependencies does it have? Just one, which has no
more. So that chain ends quite quickly.

It doesn't take long looking at the dependencies to see that the same packages
get used again and again throughout the community. So you'll find you won't
keep going down this path once you are comfortable with the packages.

Having said all of that, it is also good practice to look at the source of a
package, and decide if you could do it better and easier yourself. It is
amazing how many packages are only a few lines of code, and fairly
unnecessary.

------
jbboehr
I'm kind of surprised no one has tried to make a standard library for node -
although I guess underscore/lodash come close.

The package situation is a serious problem for distro maintainers, which
AFAICT have basically given up packaging any npm packages the traditional way.
Luckily Nix[1] and Guix[2] can most likely support automatic importing from
npm.

Incidentally, one of the many things I love about typescript is that it has
zero non-dev dependencies [3].

[1]
[https://github.com/svanderburg/node2nix](https://github.com/svanderburg/node2nix)
[2] [https://lists.gnu.org/archive/html/guix-
devel/2017-03/msg008...](https://lists.gnu.org/archive/html/guix-
devel/2017-03/msg00814.html) [3]
[https://www.npmjs.com/package/typescript?activeTab=dependenc...](https://www.npmjs.com/package/typescript?activeTab=dependencies)

------
st3fan
Is this article truncated? The conclusion or end part seems to be missing. Or
is this just oddly written?

------
gr__or
There's two distinct problems I see here, which probably plague most package
managers:

1\. Package naming: All the good names will be gone soon, which'll probably
lead to big players trying to get package names of little players. Which will
then result in broken dependencies a la left-pad. Afaik the latter has been
solved in NPM [1]. The former is probably more of a usability problem, we want
installation to be simple and able to install deps just by remembering their
name.

2\. The power of deps and our reliance on good will: When installing
dependencies, we basically grant them the power to wreak any kind of havoc
they want. The ways we mitigate the worst: Belief in the magical power of open
source, mostly using libs that are popular (hence might've gotten a good read
by an IT-sec person) and at enterprise scale some static analysis tools to get
an idea of how it interacts with your system. I think we're in need of a
technical solution at the package manager level, where a dependency has to
explicitly ask for certain capabilities (similar to how Android apps work).

[1] [http://blog.npmjs.org/post/141905368000/changes-to-npms-
unpu...](http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-
policy)

~~~
krapp
1) seems to plague NPM a lot more than others.

> The former is probably more of a usability problem, we want installation to
> be simple and able to install deps just by remembering their name.

There's a point beyond which the desire for simplicity creates dangerous
abstractions. A developer should expect to put more effort into installing
deps than just remembering a package name.

~~~
gr__or
That it plagues NPM more than others might be due to size, NPM has almost 3x
as many packages as the next large module repository (Maven) [1].

[1] [http://www.modulecounts.com/](http://www.modulecounts.com/)

~~~
krapp
Size does mean the scope of any issues with NPM is greater, but many of those
issues seem unique to NPM's design and the extremely fine-grained nature of
Node packages as a whole. One developer had a temper tantrum and unpublished
one package and the entire Node universe caught on fire... that kind of
catastrophic failure doesn't seem to happen elsewhere, even at a smaller
scale.

~~~
gr__or
True, that WAS a deficiency, though one that seems to be fixed now.

Given that dependencies are also resolved by name in the Ruby & Python I'd
think there's no technical reason for why a similar thing wouldn't happen, is
there?

~~~
krapp
>Given that dependencies are also resolved by name in the Ruby & Python I'd
think there's no technical reason for why a similar thing wouldn't happen, is
there?

I don't know, I'm not familiar enough with those languages, but I would guess
that if anything similar did happen, it would probably wind up posted here,
although absence of evidence isn't evidence of absence.

It may also depend on implementation features besides package names - the
worst that should happen if a remote dependency no longer exists is that
future updates fail, but a local copy should always exist and the current
build should still keep working. To me, vendoring dependencies should be the
default.

------
pan69
One of my least favourite things about the NPM eco-system is that it doesn't
have support for name spacing, i.e. package names are global. I think this is
one of the things that Composer (PHP's equivalent of NPM) does very well [1].

Personally I'm not to worried about the size of my node_modules since it
doesn't reflect what gets transpiled into the target anyway, but yes, if you
have a bunch of projects sitting on your hard drive, it can get a bit busy.

[1] [https://packagist.org/](https://packagist.org/)

~~~
simplify
npm has namespacing [https://docs.npmjs.com/getting-started/scoped-
packages](https://docs.npmjs.com/getting-started/scoped-packages)

------
mabynogy
Only NPM causes troubles. Node.js is stable and is improved quickly since the
io.js merge.

You don't need to use NPM to run node programs. The binaries provided by
node.js website are standalone (no dependencies):
[http://www.nodejs.org/dist/](http://www.nodejs.org/dist/)

------
TheAceOfHearts
Clickbait. At least propose what it is that you think needs to be done to
address this so-called chaos.

Worth noting that ES2017 added String.prototype.padStart and
String.prototype.padEnd. The whole left-pad thing is blown out of proportion.

Second, you shouldn't be relying on npm for deployments. You dig your own
grave. If package deletion is a concern, then host a mirror with something
like Nexus [0], or pay for a service [1] [2] to do it for you. Over the years
there have been a few local npm mirror tools [3], but I don't think any of em
have really caught much traction. There's also the option of bundling each
release. For that you can use a tool such as Webpack [4], or take it one step
further and generate a single binary using something like pkg [5].

Developers also have the option of forking their dependency graph and updating
references in package.json. Sooner or later you'll start to find bugs in your
dependencies and have to fix them. It's your responsibility, you own all that
code. Don't be afraid of maintaining your own fork. If you find a critical
issue you'll probably need to be able to deploy your update without depending
on a third-party anyway. This is fairly common in other ecosystems. Heck, you
can also just check in your node_modules folder into version control.

Don't get me wrong, npm certainly has its fair share of problems, and
historically they haven't had the best defaults... But there's usually ways to
work around the issues, and they're gradually working towards improving
things. It's the developer's responsibility to handle their ecosystem's
problems, which is also nothing new in the world of software development.

[0] [https://www.sonatype.com/download-oss-
sonatype](https://www.sonatype.com/download-oss-sonatype)

[1] [https://jfrog.com/artifactory/](https://jfrog.com/artifactory/)

[2] [https://inedo.com/proget](https://inedo.com/proget)

[3] [https://github.com/rlidwka/sinopia](https://github.com/rlidwka/sinopia)

[4] [https://webpack.js.org/](https://webpack.js.org/)

[5] [https://github.com/zeit/pkg](https://github.com/zeit/pkg)

~~~
FLUX-YOU
>It's the developer's responsibility to handle their ecosystem's problems,
which is also nothing new in the world of software development.

This is where you have to start questioning the benefits of using node/npm/js
on the backend. We're all stuck with it on the front end for the foreseeable
future.

I've never had any of the problems in node/npm using NuGet/C# for example.

At some point, I raised my hands and emailed my boss about a new project using
node, essentially saying "this is dumb. left-pad is dumb. is-number is dumb.
is-odd is dumb. the amount of packages depending on these is dumb. And I won't
be arsed to fix it because I value my time, so I'd rather just jump back to an
ecosystem that has its shit together because the result is the same simple API
pushing json either way we go."

node/npm sucks IMO. Sorry, but the problems I have to fix daily are not
interesting, and conquering the mountain and being able to say I fixed all
those every-day problems with it is not worth much on the job market.

If the amount of web backends suddenly needs node because they suddenly stop
pushing json all day long, because node/npm is the only ecosystem that offers
that solution, then you have a compelling reason. But other languages do json
APIs just fine right now and have less ecosystem paper cuts.

------
yilugurlu
It's 2018 and still leftpad issue, really?

~~~
Hamuko
The article says that you can still unpublish packages from NPM at will. If
that's still the case, then yes, leftpad should still be brought up in 2018
before someone unpublishes the "is-string-empty" package or whatever the dumb
packages that people put on NPM are.

~~~
kabes
you can only within 24 hours of publishing. After that you need to send them
an email to request it and they determine if it can be unpublished or not.

~~~
kkapelon
Not true
[https://status.npmjs.org/incidents/41zfb8qpvrdj](https://status.npmjs.org/incidents/41zfb8qpvrdj)

"We have restored all the package-versions for the 9 packages that were
published over"

Full context here
[https://github.com/npm/registry/issues/255](https://github.com/npm/registry/issues/255)

------
amrx101
There is a npm package to check if the number is even or not, Enough said.

~~~
boobsbr
Yes, because a developer wrote a joke project, and nowthe whole ecosystem is
garbage...

------
nisten
The problem relies in the obsession with monolithic design even if it ends up
being a complete house of cards. This leads many new developers to sacrifice
basic logic and first principles when judging what good software is; often
resulting in products that no one uses and that just crash anyway. But hey,
you get this great toy-like model to show on your next interview. Security and
access rights are sadly not taught as being essential to having a proper
monolith because they'd greatly slow down development time. Whereas an
engineer or builder has no choice but to confine their creativity within those
principles because a product crash for them has much more dire consequences.

