There isn't a single thing there that seems iffy to me. Rust projects split themselves into as small of a crate as possible to 1) ease their own development, 2) improve compile times to make their compilation trivially parallelizable, and 3) allow for reuse. Because of this, you can easily end up with a dozen crates all written by the same group of people, meant to be used together. If a project is a single big crate, or a dozen small crates, you're on the exact same situation. If you wouldn't audit the small crates because they are a lot, you wouldn't audit the big crate thoroughly either.
But what about transitive dependencies? Similar thing: if you have a crate to check for the terminal width, I prefer to take the existing small crate than copy paste its code. I can do the latter, but then you end up with effectively a vendored library in your code that no tool can know about to warn you when a security vulnerability has happened.
> There isn't a single thing there that seems iffy to me.
You mean like four versions of hashbrown (which is useful, but it's rare to have to use it directly instead of `std::collections::HashMap`, never mind pulling four versions of it into your project) or four versions of itertools (which is extremely situational, and even when it is useful it usually only saves you a couple of lines of code, so it's essentially never worth pulling it once, never mind four times)? Or maybe three different crates for random number generation (rand, nanorand, fastrand)?
There's a definitely problem with how the Rust community approaches dependencies (and I say this as someone who loves Rust and uses it as their main language for 10+ years now). People are just way too trigger happy with external dependencies, and burying our heads in the sand is not helping.
Inclusion of every external dependency should always be well motivated. How big is the dependency? How much of it do we use? How big of an effect will it have on compile times? How much effort would it be to write it yourself? Is it security sensitive? Is it a dependency which everyone uses and is maintained by well known community members, or some random guy from who knows where? And so on.
For example, cryptography stuff? No, don't write that yourself if you're not an expert; you'll get it wrong and expose yourself to vulnerabilities. Removing leading whitespace from strings? ("unindent" crate, which is also on your list) Hell no! That's like a minute or two to write this yourself. Did we learn nothing from the left-pad incident?
The two options for cargo here are 1) fail to compile when there's more than one crate-version in the dep tree or 2) allow for there to be more than one and let the project continue compiling. The former would be more "principled" but in practice incredibly disruptive. I usually go "dep hunting" to unify the versions of duplicated deps. Most of the time that's just looking at `cargo tree` and modifying the `Cargo.toml` slightly. Other times it's not easy, and have to either patch or (better) wait until the diverging dep updates their own `Cargo.toml`.
> People are just way too trigger happy with external dependencies, and burying our heads in the sand is not helping.
>
> Inclusion of every external dependency should always be well motivated. How big is the dependency? How much of it do we use? How big of an effect will it have on compile times? How much effort would it be to write it yourself? Is it security sensitive? Is it a dependency which everyone uses and is maintained by well known community members, or some random guy from who knows where? And so on.
We can have a nuanced discussion about dependencies. That's not what I was seeing. There are plenty of things that can be done to improve the situation, specially around Supply Chain Security, but this idea that dependency count is the issue is misguided. It pushes projects towards copy-pasting and vendoring. That makes that code opaque to security tools, existing or proposed. Think of the shitshow it is if you have an app and decided "more dependencies is bad, so I'm copying xz into my repo"?
> Removing leading whitespace from strings? ("unindent" crate, which is also on your list) Hell no! That's like a minute or two to write this yourself.
I don't have access to the closed-source repo to run `cargo tree` to see where `unindent` is used from, but why do you feel this is an invalid crate to pull in? It is a proc-macro, that deindents string literals at compile time. Would I include it directly in a project of mine? Likely not, but if I were using `indoc` (written by dtolnay), which uses `unindent` (written by dtolnay) my reaction wouldn't be "oh, no! An additional useless dependency!".
> I don't have access to the closed-source repo to run `cargo tree` to see where `unindent` is used from, but why do you feel this is an invalid crate to pull in?
Each additional dependency imposes an ongoing audit burden on the downstream consumers of your project.
In an era supply chain compromises are increasing and the consequences are catastrophic, the security story alters the traditional balance of "roll your own" versus "use the shared library".
> but this idea that dependency count is the issue is misguided
Well, partially you're right. There are roughly two things which are important here:
1) The number of unique authors/entities controlling the dependencies. (So 10 crates by exactly same author would still count as one dependency.)
2) The amount of code pulled in by a crate. (Because this tanks your compile times; I've seen projects pulling in hundreds of thousands of lines of code in external dependencies and using less that 1% of that, and then people make surprised pikachu face that Rust is slow to compile.)
> I don't have access to the closed-source repo to run `cargo tree` to see where `unindent` is used from, but why do you feel this is an invalid crate to pull in? It is a proc-macro, that deindents string literals at compile time. Would I include it directly in a project of mine? Likely not, but if I were using `indoc` (written by dtolnay), which uses `unindent` (written by dtolnay) my reaction wouldn't be "oh, no! An additional useless dependency!".
I would never include either in any of my projects, and would veto any attempt to do so. As I already said, the 'unindent' crate is trivial to write by myself, and the 'indoc' crate seems completely not worth it from a cost/benefit standpoint in the very rare case I'd need something like that (it's easy enough to make do without it, as it's just a minor situational quality of life crate).
In general my policy on external dependencies is stricter than most people; I usually only include high value/high impact dependencies, and I try to evaluate whether a given dependency is appropriate in context of the concrete project I want to use it in. If it's a throwaway script that I need to run once and won't really maintain long-term - I go crazy with gluing whatever external crates there are just to get it done ASAP! But if it's a project that I'll need to maintain over a long period of time I get a lot more strict, and if it's a library that I expect other people to use then the bar for external dependencies gets even higher (because any extra dependency I add will bloat up the compile times and the dependency trees of any downstream users).
I also find it helpful to ask myself the question - if it wasn't easy to add new dependencies (e.g. if I was still writing in C++, or cargo wasn't a thing) would I still include this dependency in my project? If the answer is "no" then maybe it's better not to.
There are some notable exceptions, but sadly most of the Rust community doesn't do things this way.
there's 2 kinds of bugs related to security: accidental bugs, and maliciously injected bugs. xz was the second time (which you could have avoided if you vendored starting at a reviewed / trusted point in time...)
from empirical studies, we know the first kind occurs at roughly the same rate everywhere, so it's just do you have capacity to fix it. also, reusable dependencies typically are more configurable which leads to more code and more bugs, many of which might not have affected you if you didn't need all the flexibility.
dependency count is an indirect measure of the second kind, except rust pushes crates as the primary metric, so it will always look bad compared to if it pushed something more reasonable like the number of trust domains.
Agreed, the dependency list looks extremely boring and completely auditable to me.
The dependencies are modular, not diffuse.
I think people saw the title, and got triggered into hate. When actually, this seems author-submitted, and they were probably just trying to be humble about their accomplishment. It's not even the title of the article.
> they were probably just trying to be humble about their accomplishment
Thanks for your reply. To be honest, I simply recognize that depending on open-source software a trivial choice. Any non-trivial Rust project can pull in hundreds of dependencies and even when you audit distributed system written in C++/Java, it's a common case.
For example, Cloudflare's pingora has more than 400 dependencies. Other databases written in Rust, e.g., Databend and Materialize, have more than 1000 dependencies in the lockfile. TiKV has more than 700 dependencies.
People seem to jump in the debt of the number of dependencies or blame why you close the source code, ignoring the purpose that I'd like to show how you can organically contribute to the open-source ecosystem during your DAYJOB, and this is a way to write open-source code sustainable.
lots of crates by different authors: you need to trust each one not to be compromised
lots of crates by a cohesive group of authors: you "only" need to trust the group reviews each others work properly and they're not all compromised together (less likely).
The full list is linked in the article https://gist.github.com/tisonkun/06550d2dcd9cf6551887ee6305e...
There isn't a single thing there that seems iffy to me. Rust projects split themselves into as small of a crate as possible to 1) ease their own development, 2) improve compile times to make their compilation trivially parallelizable, and 3) allow for reuse. Because of this, you can easily end up with a dozen crates all written by the same group of people, meant to be used together. If a project is a single big crate, or a dozen small crates, you're on the exact same situation. If you wouldn't audit the small crates because they are a lot, you wouldn't audit the big crate thoroughly either.
But what about transitive dependencies? Similar thing: if you have a crate to check for the terminal width, I prefer to take the existing small crate than copy paste its code. I can do the latter, but then you end up with effectively a vendored library in your code that no tool can know about to warn you when a security vulnerability has happened.