I get an allergic reaction to any rationale that includes the tired phrase "easier to reason about". That's on my bingo card right next to "promote synergy"...
Now, I'm not inherently against having small modules for small tasks, but I'd much prefer if those are used like sprinkles on a more solid foundation. So a small Perl module on CPAN? Sure. Your whole web app consisting of small modules, turtles all the way down? No, sir, no.
It also means that you've got plenty of build systems, it's incredibly hard to change the output of all your dependencies to not-ES5, it's way more frequent that one developer introduces new dependencies, possibly duplicates etc.
Sadly there's no decent way out. JS is getting a new syntactic feature every second Monday, but no big std lib movement is out there that I'm aware of. Sure, you probably wouldn't want to fit the Go/Python/Java core lib into a browser, but for the backend it would be quite nice, and would make tooling for the frontend less fragile. (And to be honest, even frontends are approaching mid-tier desktop UI complexity; otherwise we could've and should've stayed with jquery)
I don't need to be able to reason about single modules independent from each other often. I need to trust the whole system enough to not lose sleep over it.
> It also means that you've got plenty of build systems, it's incredibly hard to change the output of all your dependencies to not-ES5,
On any realistic sized project you already need tooling for handling dependencies in bulk. If you want to build everything for ES6, that should be a problem you solve once in your build/dependency manager, not per-depednency.
> it's way more frequent that one developer introduces new dependencies, possibly duplicates etc.
I actually disagree with this: if you have large frameworks then you're more likely to have duplication, because you'll have frameworks that overlap but don't completely cover each other. E.g. C++ projects tend to end up with 5 different string classes, because every framework you use will bring its own (because C++ relies on big heavy frameworks because it has a very high per-library overhead due to limited tooling). Small focused dependencies make that less likely.
I don't quite get the latter. First of all, if the foundations are sound and the core language provides a good enough module already, neither large frameworks nor small modules need to duplicate that.
And if it isn't included, why would 16 small subsystems have fewer duplicates than 2 large frameworks?
By the way, C++ is exactly the trajectory I see JS moving towards. A plethora of different sub-languages and library sets. Only with the difference that it's both easier from a tooling and culture perspective to bring in new tiny modules.
> First of all, if the foundations are sound and the core language provides a good enough module already, neither large frameworks nor small modules need to duplicate that.
A language standard library is just another framework - Python's one is famously "where modules go to die", but we see much the same problems in e.g. Boost. A "good enough" solution in the standard library can be worse than nothing, as it means better alternatives struggle to gain enough momentum, and this gets worse as time goes by: functionality that's in the standard library has to evolve slowly.
> And if it isn't included, why would 16 small subsystems have fewer duplicates than 2 large frameworks?
Because the common things can be in their own small pieces. If a database-access library, a web templating library, and a config parsing library all need to do string manipulation, then if they all implement it themselves we'll see duplication, whereas if they all use an external library then we won't (this is after all the point of libraries). Maybe the ecosystem will have two different string manipulation libraries on the go with one gradually displacing the other, but that's likely to only happen when there's a genuine improvement being made. Whereas if everything's a big framework then eventually everything implements its own copy of the common utilities that just about everything ends up needing - collections, datastructure manipulation, all the really generic stuff.
FWIW, I've taken the exact opposite route. Much as I respect sindresorhus, my projects nowadays have very few dependencies; I treat every dependency as an elevated "point of failure"-risk, and I would not recommend anyone having potentially thousands of "left-pads" sprinkled across a business-critical system.
Small packages are fine, if the maintainers keep them up to date! You can't publish packages and never touch them again. This leads to situations like the one with merge-deep that I recently discovered:
All but one of those dependencies are from the same maintainer. Yet I see two major versions of kind-of, for-in, and lazy-cache. What's the point of creating reusable functions if you're not going to reuse them?
This is a huge waste. First on the filesystem, no wonder our node_modules folders are hundreds of megabytes big. Second it wastes bytes in memory and in CPU caches since they don't share byte code and each function has to be parsed, compiled, optimised separately.
I seem to be in the minority thinking this situation is insane…
It's insanity all the way down to the copper on the board frankly. People only care about making things more sane when you can prove there's a dollar to be made or saved by doing so.
Like we all know nodjs is slow(er than other things). The question is not how to make it faster by doing things like keeping dependencies up to date but how much there is to gain by doing so. I suspect very little.
All of this seems to be justifying a lack of a decent standard library. However its appears to be written from the point of view of someone who's never had the joy/pain of a decent standard library.
The problem with lots of tiny modules is consistency and power.
Sure its "open source" but each one of these modules are in various states of decay. better to lump them together in sets of functions, so that more eyes can be concentrated in the same place.
This doesn't seem too controversial of an opinion, haven't small modules already been at the core of a Unix philosophy? Once usage patterns emerge, these smaller libraries are composed into frameworks which ultimately build end-user applications.
The entirety of engineering is premised upon leveraging powerful abstractions; cf. the reason we're not still coding via electric charge.
Edit: To expand on the analogy: "A two-person startup already uses twenty-eight other tools" [1].
>, haven't small modules already been at the core of a Unix philosophy?
I assume you're talking about userland programs such as "ls", "cut", "grep", etc. ?[1]
But C Language is also part of UNIX and programmers typically link against a large more comprehensive "standard library" such as glibc:
libc.a, libc.so <-- includes printf
... instead linking of a hundred little object files specified explicitly like this:
printf.o
strcat.o
strncat.o
malloc.o
leftpad.o
... <hundred or thousand more .o files>
What happened with Javascript is the language didn't have a comprehensive "standard library" like C, C++, Java, C#, Python "batteries included", Go, etc. Therefore, NPM became an adhoc "standard library" via bottom-up crowdsourcing. There are both positives and negatives to that approach. One negative is that leftpad() isn't produced by a canonical entity like Netscape or Mozilla but a random programmer. When he wanted to take his ball and go home, he broke everyone's build that depended on it. That type of event didn't happen with "printf()" in the C Language world.
I just looked at the source code of left-pad for the first time, and it's strange. They noticed that concatenating a bunch of single-space strings is O(n^2), and optimized it to O(n log n) by using repeated doubling. So in the end they take longer than O(n) to fill an array with a constant value. Why?
Why write code to left pad your strings, when you can get an O(n log n) implementation just like that. You only need a "minimal" amount of bundle tooling and a few hundred MB of node_modules. Count your blessings. This is the way.
Small focused modules dont need much maintenance or braking changes, so you basically only have to download them once, and treat them as if they where your own code.
The article highlights a huge pitfall of having lots of small dependencies, provides no real remediation, and says small dependencies are easier to reason about, so they're better anyway.
I disagree. I personally find that with large projects, small dependencies actually become much harder to reason about - they sort of get lost in the codebase. Not to mention the duplication of small packages on npm doing similar things - with a large team, it's easy to add both to your codebase and then we're in a new sort of duplicative mess.
Small modules are only useful when they're self-contained. Leftpad was mocked because it brought in three dependencies on top of itself. The more dependencies one has, the more breakage hell one must deal with. So please, keep your code self-contained when sharing with the world.
> Regarding Jobs' quote, here's another way to look at it: the most beautiful hotel in the world wouldn't feel like such if all the toilets overflowed when flushed.
But this would be a functional problem, irritating at best and a potential health hazard at worst. Using plywood for the back of a chest of drawers is no such thing.
Now, I'm not inherently against having small modules for small tasks, but I'd much prefer if those are used like sprinkles on a more solid foundation. So a small Perl module on CPAN? Sure. Your whole web app consisting of small modules, turtles all the way down? No, sir, no.
It also means that you've got plenty of build systems, it's incredibly hard to change the output of all your dependencies to not-ES5, it's way more frequent that one developer introduces new dependencies, possibly duplicates etc.
Sadly there's no decent way out. JS is getting a new syntactic feature every second Monday, but no big std lib movement is out there that I'm aware of. Sure, you probably wouldn't want to fit the Go/Python/Java core lib into a browser, but for the backend it would be quite nice, and would make tooling for the frontend less fragile. (And to be honest, even frontends are approaching mid-tier desktop UI complexity; otherwise we could've and should've stayed with jquery)
I don't need to be able to reason about single modules independent from each other often. I need to trust the whole system enough to not lose sleep over it.