I make a periodic practice of searching our node_modules folder for files that shouldn't be there and reporting bugs against the offending projects.
Usually that's been pretty effective, and now the total cruft is around a megabyte whereas before it was somewhere north of 50MB all told. (Conditions apply).
coverage reports, test results, build detritus, etc.
The one I'm still debating, because it's becoming a serious problem for a couple of our libraries: should the tests be included in .npmignore or kept along with the library? I'm not sure what the right answer is there. Test sizes especially including fixtures can creep up quite a lot over time. I know what I'd like it to be, but I'm not sure I can win that argument with a bunch of maintainers on different projects.
> should the tests be included in .npmignore or kept along with the library?
What are the reasons it would be a good idea to include the tests with the release/distribution of a package?
Seems like something you don't care about when you're just using it as a library, unless you want to modify something in it, but then you'll clone the library straight from a repository anyways, which includes the tests.
For packages where I don't include tests, I've had at least one downstream distro maintainer request that I include tests, since at least some of them treat npm or PyPI or whatever as the source of releases.
For packages where I do include tests, I've had at least one user request that I remove tests so that the footprint of the Docker image they're building is smaller.
Both are entirely reasonable requests, but package repositories don't really provide a good way of accommodating both at the same time, for instance, by allowing a separate upload of the dev gubbins such as tests.
As a software maintainer myself, I believe the downstream distro maintainer is the one being wrong there.
You have a software project, with a build process, and the "output" or final product of that project is the library that gets uploaded to NPM.
If they are packaging a software library, they should do it from the project's repository, not from one of its output artifacts.
They would probably reject a request if someone who was downstream of their work decided to repackage their stuff and asked them to include tests and other superfluous content on their packages.
I don't know about other distros, but Debian makes it extremely easy to download both the binary package and the source package. For instance, on the page for the jq package [1], you can download the source using the links down the right-hand side, which includes the full test suite. The key, in my view, is that Debian has a nice way to associate both the final output artefact and the source (both the original source and their patches) with a specific version.
The way it works for Debian packaging is that they usually have their own copy of the project's source code (what they call upstream). So the packaging process does start from the actual, original source code repo of the upstream project being packaged. This code is kept in a "upstream" branch, to which Debian-specific files and patches are added, usually in a different branch named "debian". For new versions, the "upstream" branch is updated with new code from the project.
All of which, if you ask me, is the correct way of doing any kind of packaging. Following that, IMO the same should be done for JavaScript libraries: the packaging should be done by cloning the project repo and adding whatever packager-specific files in there.
Notice in your link how in the upper part it says: [ Source: jq ], where "jq" is a link to the source description. In that page, the right hand section will now contain a link to the Debian copy repository where packaging is done:
You can clone and explore the branches of that repo.
(Maybe you are a Debian maintainer, in any case I'm writing this for whoever passes by and is curious about how I think JS or whatever else should be packaged if done properly)
The downstream distro maintainer is in the wrong IMO; if they want the source code, they can get the source code off of e.g. github and roll their own release.
That said, in old Java dependency management (i.e. Maven), you could often find a source file and a docs file alongside a compiled / binary release, so that you get the choice.
But this can also be done with NPM libs already; the package.json shipped in the distribution contains metadata, including the repository URL, which can be used to get the source.
I had that discussion with someone before. They were under the impression that the registry is rally permanent while GitHub could go away. Aside from the platform betting, they really thought their 2012 package (and particularly its tests) would be useful in 2052. But who am I to say otherwise.
The recent HN discussion about “that one npm maintainer” confirms please hold onto the most painful ideas.
My first thought was, "include a dev and prod version of the package" but that creates a ton of regression surface area for a feature that most people can't be bothered with anyway.
It's easy enough to have things work in pre-prod and fail in prod without running slightly different code between them.
I think there is a solution to this, but it's going to require that we change to something a lot more beefy than semver to define interface compatibility. Semver is a gentlemen's agreement to adhere to the Liskov Substitution Principle. We are none of us gentlemen, least of all when considered together.
I make a periodic practice of searching our node_modules folder for files that shouldn't be there and reporting bugs against the offending projects.
Usually that's been pretty effective, and now the total cruft is around a megabyte whereas before it was somewhere north of 50MB all told. (Conditions apply).
coverage reports, test results, build detritus, etc.
The one I'm still debating, because it's becoming a serious problem for a couple of our libraries: should the tests be included in .npmignore or kept along with the library? I'm not sure what the right answer is there. Test sizes especially including fixtures can creep up quite a lot over time. I know what I'd like it to be, but I'm not sure I can win that argument with a bunch of maintainers on different projects.