
Ask HN: How should open source support critical dependencies? - hn_throwaway_99
In something slightly similar to the infamous leftpad issue, many NPM builds started failing yesterday because &#x27;npm audit&#x27; now reports a security vulnerability for lodash. lodash is currently in use in over 7 million Github repos, and it&#x27;s nearly impossible to use NPM without some transitive dependency that depends on lodash.<p>This security vulnerability was reported in October (https:&#x2F;&#x2F;hackerone.com&#x2F;reports&#x2F;712065) and there have been 2 PRs open for 2 months that fix this issue, but lodash hasn&#x27;t had any releases for a year. There is essentially one (unpaid) person who has power to release lodash, a library that a <i>huge</i> majority of reasonably-sized javascript projects now depend on.<p>How should the open source community address essentially abandoned projects that have become critical in the ecosystem?
======
hannob
The ability to fork and gain control if necessary is one of the main reasons
one wants open source in the first place. So fork it.

Other than that: I find the microdependency / mass dependency trend in NPM
very irritating. As a rule of thumb I think you should be able to know at
least roughly all your dependencies + transitive dependencies and take some
care that they're healthy projects. If you can't name them all - you probably
have too many of them.

~~~
soneil
I think the problem with forking is you can't take the 'lodash' noun on NPM.
So you make a fork of lodash, fix the outstanding issues - but getting every
single dependency to actually accept it is a whole 'nother mission. And as
long as the original project 'owns' that noun, this is true for everyone.

Does npm have a facility to say a package provides another? Would npm resolve
that? Or at least accept that if I have package lojack-mine and it
"provides:lojack" then this dependency is resolved?

~~~
jka
Yup, you got it; the migration/reclaiming of package names is pretty key here.

One tricky aspect of the 'metapackage' approach (where, for example,
hypothetical 'lodash-legacy' or 'lodash-ng' packages could both fulfil a
'lodash' dependency slot) is that someone needs to be responsible for
determining which packages can fill which slots.

In cases where the package alternatives are close forks and API-compatible,
that's easier to agree on - but in a large ecosystem, the solution needs to
scale in terms of human and machine review time, while also providing a good
level of trust to developers when they are selecting or migrating
dependencies.

There are likely to be some tricks to reduce the assessment workload required
in each case -- running test suites from the 'original' package and
applications which rely on it against each candidate, for example -- but
ultimately either the package maintainers, the community, the NPM organization
themselves -- or some mix of all of the above -- have to determine the
packages that could fulfil each slot.

Good packaging is hard, and it's maintenance work rather than green-field
development. We (collectively) really should incentivize and fund it more, and
ideally learn from and credit existing solutions.

I'm sure there are people who enjoy reviewing (and automating review of)
packages, knowing that they're improving code quality and performance for huge
numbers of developers. And similarly, there are plenty of developers who
provide useful feedback on package quality by reporting bugs, running
benchmarks and writing qualitative comparisons.

To guess at where we may be heading in the near-term: a pure Microsoft-
subsidiary ecosystem is developing in this area -- GitHub and NPM provide all
the data and infrastructure to host, scan and gather user feedback for the
processes mentioned above.

I think they'll collectively do a good job, although any approach does take
time to build and deploy. I imagine it'll also raise longer-term questions
around the governance of open source software development and what role
Microsoft would like to take in that.

On a more abstract level the problem space relates to network effects and
influence goals. Microsoft believes that the open source developer community
is either here to stay, or is here at least for a sufficient duration of time
for them to be important -- and it would like to provide influence and guide
rails to that community.

------
benologist
I saw a great suggestion the other day -

> The npm ecosystem needs something like the distinction between Ubuntu's
> "main" and "universe" repositories, so that you have a smaller subset of
> known-good packages with harmonized transitive dependencies, stronger
> centralized curation, a lot more direct scrutiny, tighter control over
> versioning, and some party that is responsible for addressing
> vulnerabilities and ensuring appropriate maintainership.

[https://news.ycombinator.com/item?id=23680734](https://news.ycombinator.com/item?id=23680734)

------
evolve2k
Establish an initiate like ‘Ruby Together’

> Ruby Together is a grassroots initiative committed to supporting the
> critical Ruby infrastructure you rely on.

> Your financial backing allows Ruby Together to pay expert, professional
> developers to work on improving critical infrastructure projects.

> The Ruby Together board of directors, elected by members, selects the most
> impactful projects to fund.

> Ruby Together publishes monthly updates showing progress and our proven
> results.

It brings together companies and individuals concerned about stability and
health of the ecosystem, provides a aggregated place to do high impact
sponsorship of the ecosystem and the elected board then sets about making a
detailed workplan and working out the best distribution of funds.

One dev on lodash? Discuss its importance; offer the dev funds to keep things
up-to-date and if not them, have the resources to work out another way.

Check out their early reports to get a better idea of the initial refactoring
work they did within the ecosystem.

[https://rubytogether.org/](https://rubytogether.org/)

------
ddevault
The approach embodied by package managers like npm - but also Python, Go, and
others besides - is fundamentally broken. The software maintainers should not
be the sole point of failure in publishing their package.

The approach used by distributions is much, much better. Having a separate
team of people responsible for packaging up software is a much better
approach. This would allow the security patches to get in, help enforce a sane
baseline of packaging standards, prevent the introduction of malware, and have
someone accountable whose interests align more with downstream than upstream.

~~~
Kinrany
Using a package manager like npm does not prevent having a separate team of
people responsible for validating and re-publishing packages under their own
namespace. A new organization could step in and make their own whitelist at
any time.

~~~
ddevault
It is a cultural problem.

------
bjourne
Good question! One Python example is docopt. It is a very well-liked command
line parsing library but haven't seen any maintainer activity in two years.
[https://github.com/docopt/docopt](https://github.com/docopt/docopt) But
people don't realize that so they keep filing bug reports and submitting pull
requests.

There is a docopt-ng package here: [https://github.com/bazaar-projects/docopt-
ng](https://github.com/bazaar-projects/docopt-ng) But users haven't found it
yet. And sadly, there is no guarantee that the new maintainer won't also lose
interest and abandon the project.

Definitely not an ideal situation. Yes, you can fork but how do you get
visibility? How can the developer announce to the community that he or she
wants to maintain the software?

------
MattGaiser
Really there just need to be more people involved in open source. The web is
being built on the unpaid contributions of a few and when they don’t have time
or mess up due to being rushed, piles of things collapse.

I suspect if a human resource audit of open source were done, there would be
surprisingly few people involved in the maintenance of these things.

------
codegeek
I think companies with deep pockets need to step up a bit. I mean if anyone at
FAANG etc are using packages like lodash, they should ideally team up and
create some type of a funded group/co-op that funds these libraries and their
maintenance and removes dependency from one person. You cannot just "charity"
the way out of these situations.

~~~
mahmoudimus
Completely agree. I hate that companies do not give back to this -- but it is
changing and that's a positive.

Stripe & Segment have their open source grants, WhatsApp Co-Founder (Jan Koum)
donated $1MM USD to FreeBSD, DuckDuckGo donates to companies since 2011
([https://duckduckgo.com/donations](https://duckduckgo.com/donations)) etc.

At the company I have founded, I am putting my money where my mouth is -- we
have a budget to allocate up to $25,000 per year to open source products that
we find value in. As we hit certain revenue milestones, I have every intention
of donating an even larger % to funding open source initiatives as part of our
"giving back" corporate social responsibility.

It is a shame most companies do not do the same. I cannot change them, but I
can control change within what I do. So, thank you to all the makers and
builders out there, I will make it my personal mission to figure out a way to
give back to you!

~~~
gingerlime
I want to do it at my company as well (albeit at a much smaller scale), but
how? I asked previously without much luck
[https://news.ycombinator.com/item?id=23439670](https://news.ycombinator.com/item?id=23439670)

(Not to mention try to get it as a taxable donation... which complicates
things further)

~~~
mahmoudimus
I'm not sure about the taxable donation (did you perhaps mean a tax-deductible
donation instead?) but I have a few ways. It looks like that thread links to a
company that's specializing in that, were they able to solve your use case? If
not, feel free to ping me and I can connect you w/ the folks on our team who
have this as a key objective for them and have figured out most of the
logistics.

I really appreciate you attempting to do this though, thank you!

~~~
gingerlime
Tax-deductible yep :) sorry about that

I was in contact with these guys but nothing came of it so far ... I think
they’re still working on it but I don’t know. Would be great to find out
options. Thank you for your help.

Side note: lack of funding for open source is a common complaint it seems, but
I was personally surprised it’s actually not trivial at all to _give_ money
open source ...

------
zzo38computer
I mostly try to reduce dependencies, both direct and indirect, regardless of
the programming language in use.

------
pacifika
Fork it? The better solution is to use less dependencies. Or in this case
write JavaScript the browser can execute today, not to standards that might
not be implemented tomorrow.

Or work with a paid version of npm if you’re app is business critical.

~~~
renewiltord
If you're using yarn, apply the patch in your GitHub fork, bump the version,
use a resolutions block to point to your repo (I think ref based doesn't work
well, might be old info that it doesn't, though).

IMHO coordination based methods will work poorly here.

