Occasionally while waiting for npm install to finish downloading, I think about how there are websites like http://www.urbandead.com/ which is probably a single 20k line perl script and mysql server running on a single box with 12 years of uptime that somehow manage to get along just fine with no javascript and apparently no input from their owners.
When I started doing webdev 8 years ago you shipped a recentish copy of jQuery, maybe jQuery UI if you were feeling fancy and that contained pretty much every library function you needed. Now we NPM install thousands upon thousands of modules to get by.
These days, browsers have tons of library features baked into them - but with APIs so obtuse and difficult to use you have to install another library to make them usable.
Last week at work I had a 2 hour meeting where we tried to work out how to ship the 2MB of JS our website apparently needs to clients without Google Lighthouse penalising us.
I don't know what to do with any of these thoughts but they often make me want to quit and take up a profession that doesn't involve computers like blacksmithing :/
It's pretty crazy. I have just about the exact same experience I'd say. On personal projects I'm moving back towards straight CSS, Vanilla JS (maybe I'll use jQuery if they get more complicated, they're very bare bones at the moment), and server generated HTML with Django templates.
Web Development seems to move at this incredible pace. I also started just about 8 years ago and I've gone through jQuery, Angular 1, Backbone, React, React with Classes, React with Hooks.
And even the tiniest project I do with NPM ends up being hundreds of megs on disk locally. I was cleaning my harddrive recently, MomentJS is _over 100mbs_ cloned from GitHub, I don't even know if I actually did an NPM install in there.
Material UI was _nearly a gig_. I'm assuming there are assets in there somewhere.
Meanwhile we have really neat improvements in CSS and HTML that I want to go and use but I'm to busy learning how to do CSS in my JavaScript, and validation in my JavaScript. Reinventing accessibility by deploying ever more bloated projects filled with bugs and inconsistencies.
I wonder if we'll see a gradual return to less complicated stacks as people realize a lot of work can be accomplished pretty reasonably with a lot less and a greater reliance on the browsers to do a fair amount of water carrying.
I'm not sure... but I have to get back to learning how to create APIs in GraphQL ;-).
Well SPA can do route rewrites if it's taken care of properly (Vue Router makes this easy). I'm always pushing for it on my projects for usability and link sharing. There's no reason to not have it.
There's nothing wrong with Vanilla JS and jQuery, but if you need more interactivity, it's possible to use modern libraries like React (with htm [1]) or Vue without a compilation step, all with zero dependencies. The size of the libraries is only 10k to 20k bigger than jQuery.
If you need even less bytes, then Preact is also an option (also with htm [1]), and it clocks at around 3k + 400 bytes for htm.
If you're only targeting modern browsers with ES6 support you can even use `import from` statements.
I don't buy it. I mean the entire society is set up this way. If we tried to reduce reliance on others and others systems, are we saying we should all build our own operating systems that are hardened from solar radiation?
About one week ago I woke up to my great uncle knocking on my RV (we live in his driveway temporarily) and he said, the fastest moving fire in CA history is moving toward Paradise again and we need to get ready to evacuate. We had no electricity and so updates were bad. Later he updated us after driving to the top of the hill that the fire was 40 miles away. Later that day I found out he was wrong, it was 10 miles away :P. The sky was red, the world had stopped, everything was hopeless.
I went to gas up the RV, hundreds of people doing the same, all scrambling trying to avoid a repeat of mistakes from 2 years ago. Everything is nuts, no one is calm.
Except the world didn't stop. Mail was delivered in Magalia and Paradise that very day. UPS and Fedex dropped off packages in both cities that very day. The world is actually a lot more robust that we give it credit. Sure, maybe we are constantly 2 minutes to midnight. But maybe, just maybe this modern society can handle pandemics. I know the literature is saying the exact opposite. And maybe in a year we're all going to be starving to death. But if that's the case, then the least of my worries is how many dependencies I have. If anything, if I have more, perhaps we're strengthening our society just a little by paying 5 SAAS companies to keep mine running.
Just tying it back to my fire experience, I depend on daily, so many freaking companies and people, ACTUALLY doing stuff that day, that I don't care that my code is resting on the backs of 4000 developers I haven't met, that haven't touched 90% of that code in 3-4 years anyway.
Not saying I wouldn't enjoy a completely different society, but that's a different essay.
I think the point is resiliency. The real world is resilient. In the face of massive threats to the infrastructure people will find new ways and keep living. But the average webapp is not resilient. If one out of 3700 libraries breaks then the whole thing can go down. A webapp is a very complex and rigid machine, it does not have the organic nature needed to be resilient. Making it simpler makes it sturdier.
The author is absolutely right. And to add one more point: Debugging such a system has become so difficult that many do not even attempt it anymore. I have seen experienced operators treating their stack like voodoo ("I have no idea why it acts up, but restarting this part seems to help sometimes. Oh and I increased memory on that VM, because, why not.").
For any reasonably successful web application there comes the day when a developer has to investigate the cause of a bug who wasn't there when the application was "young and agile". They will scratch their head above old libraries, out-of-fashion frameworks, tools, and languages, and finally, more often than not, argue that the whole system needs to be reimplemented in X.
Definitely some issues raised in the article and particularly the idea that even if you need a little bit of something, you inherit the entire library with dependencies you don't need!
However it is a little short on solutions. The truth is that I don't want to write my own PayPal API when there is one that PayPal produce and are likely to keep more up-to-date than me. Writing my own means that I only rely on my team but it is a big overhead for any reasonable size company that use dependencies from possibly 20+ major libraries and the hundreds of lower-level dependencies.
What would be cool would be a webpack style dependency manager that can remove things you don't use and then only show you updates for the bits you do.
It seems like "the hundreds of lower-level dependencies" is a problem that is unique to JavaScript due to its lack of a standard library.
JavaScript has a huge ecosystem, and as a result produces many new, interesting ideas, but its also the source of a lot of churn.
I remember working on large-ish (+300k LoC) .Net applications that were essentially dependent on just the .Net Framework itself.
For those looking to prioritize stability, the "solution" would probably be to avoid Node.js based applications altogether.
Use something like JavaScript/Node.js where it makes sense, in the front-end, and for developing thin API layers to support your front-ends. For heavy lifting (data and complex business logic) look towards languages/tools that prioritize stability.
I ran into this recently. Firefox has a "packager" for putting together add-ons. It uses "node.js". All it really does is apply "zip" to some files. I tried to install the "packager" on Ubuntu 18.04 LTS. It had several hundred dependencies, and wouldn't run because of some version dependency in node.js for something totally irrelevant to the task at hand. Mozilla support suggested upgrading the operating system on the development machine.
I wrote a one-line shell script which zipped up the proper file types into one file to package the add-on.
The overreliance on NodeJS in "modern" Enterprise(TM) JS development is pretty weird in general, even for folks who have an affinity for having all their tools written in JS. A bunch of the NodeJS tooling could just target the browser's native JS runtime instead, which is already installed. E.g. to build the project, you clone it, double click the project's Build.app.htm file, and then drag and drop the project source tree there. I've actually already prototyped this sort of thing. This is how I plan on doing anything when working in JS going forward, whenever possible. That it's incompatible with the vast majority of the NodeJS/NPM ecosystem is a bonus.
I use the `npm-check` [0] to help me with that problem. Might not check all of your boxes, but it certainly helps me manage dependencies in my JS projects!
Reminds me of this movie illustration the complexity and human collaboration required to produce a pencil: https://youtu.be/IYO3tOqDISE (6:33 in total; it’s a bit “dreamy” sometimes but still worth a watch, you can skip the first minute).
Basically entire human civilization is based on collaboration, learning from and relying on each other (among other things). The process kickstarted with developments in agriculture, freeing up Human Resources to work on other things. We’ve been perfecting this for a few thousand years, of which we’ve been automating things using software for only 50 years (being generous here). Give it some time - with the current rate of development we may not need another thousand years to improve software development to near perfection as well ;-)
(Or more pessimistically: there’s still a lot of hunger in this world so even after a few thousand years we’ve not even perfected the most basic of human needs - doesn’t bode well for something as fancy as the typical glorified beans counter that is most enterprise software...)
Disclaimer: This video is typically used as a pro free markets argument to illustrate the level of cooperation required to produce even the simplest things, arguing that such collaboration is only possible by free actors in a free market - that’s not the point here but still interesting tangent to think about in this context.
Sure, supply chains exist everywhere (and managing them is a field of its own..), but usually they are more explicit at least on each "link" of the chain. Can you imagine producing pencil in full anarchy where people just do random stuff and somehow you have to fish out the things you need and hope they keep the flows going without any explicit agreements or contracts? Where you get wood from logger because he happens to like logging that much, or he had some leftovers after doing a gig for someone else? That imaginary world resembles more the current state of software.
Of course, the other side to this coin is assuming that you can do better than the folks who have specialized in solving particular problems, and open-sourcing their solutions.
I'm not arguing that package bloat is a good thing, but I do think it's an acceptable trade-off for the speed and functionality improvements.
If long-term stability is a very high priority for your project, you should look at using the right tools for that job. A language like Clojure that actively seeks to avoid "churn" while operating on top of common runtimes is a good choice for things like that. There's no need to compromise on speed of development, either :)
This article mentions node dependencies and ignores the innumerable number of libraries down the stack to the bare metal. Even if you wanted to spend the time writing your application without using any libraries you still must have some level of trust on the platform you are running on.
I completely disagree. Stability isn't better when you do everything yourself, actually in my experience, it's worse.
For example, if I want my users to be able to scan a generated qr code, I am not going to write a qr code generator myself since that is quite a lot of work. I will use a trusted library to do that for me.
In this way, the library will have had a lot of eyes, patches all the time fixing bugs and so on. If I would try and make a QR code generator, I would most likely make a lot of mistakes and bugs that more seasoned people that have experience in making such functionality already have experienced and fixed.
If I stopped writing code for the company I work for today, the application would work just the same in one year, in two years etc. I don't get why a desktop app would be more stable than a web app in that sense. I would argue a web app is more stable since we don't really break backward-compatibility in a browser in the same way people do in desktop operating systems.
Sure there is something to be said to not use a big ass framework for some small app. I think frameworks usually brings more pain than it's worth in the long run. But having dependencies in modern applications is a must, even in desktop applications because people expect and demands it. If you would not use dependencies, your customers would choose your competitors most likely.
A web app cannot be more stable than a native app but there are trade-offs, it's simpler to target an app that would open on most browsers even if it does't work too efficiently. If there was only one browser in the world web apps would be a whole lot more stable but that's not the world we live in.
I think the only reason people stopped writing native apps in favor of web apps is manly for the cost benefit and we as users are getting a good enough experience but not perfect. The landscape is changing quickly, not sure how the whole webassembly thing is going to run but there's some hope.
Semantics. Because the web changes more than native. Like com 32 been around for 30 years, doom still runs in windows 10. We still cry when microsoft patches... Well you can't have your cake and eat it.
"As if source code rusted. The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it. It doesn’t acquire bugs just by sitting around on your hard drive."
By that token, blindly updating your dependencies sounds more like the problem and so does writing your new code in place of their old. Security patches may force your hand but apart from that, if you want stability just be deliberate about upgrading and it will be rare for old dependencies to suddenly generate new bugs.
As far I see it, the right way to think about good, reliable code is not as a compact, resilient thing like a baseball but like a matchstick sculpture glued together with maple syrup. It is both inherently very fragile and very unlikely to spontaneously break if good decisions went into the design of its supporting structures and because it's mounted to the sturdy, climate controlled plinth of the silicon chip.
In the article, these supporting structures such as compilers, OS and firmware are mentioned but if we don't trust these then we simply should all stop trusting computers. Most of us believe that this broad societal dependency has been a good tradeoff bringing us an amount of good which is so hard to outweigh even by an imagined disaster.
The same goes for most of the dependencies in the average ruby project (don't know what to say about JS). They are battle-hardened and bring value that companies would pay massive sums for were they not free.
There are of course the odd suspect, unnecessary dependencies in most projects. Usually they're for trivial uses and easy to remove. When you discover these few they do not indicate that the rest of your package list was a mistake. The fact that 90% of the problems you have come from 5% of the dependencies (made up statistic) should tell you that your project will be dead before the other 95% let you down.
But one thing I can't help but agree with: Upgrading to a newer version of your language implicitly creates a jolt to the maple syrup glue in every one of your dependencies. There's only so many years of that your project will stand before the unmaintained ones bring it down and an overhaul is needed.
I don't know what author is trying to say here. Should I develop everything from scratch and myself? I would actually recommend opposite.. look what you can buy before deciding to build it yourself(considering a lot of factors).
When I started doing webdev 8 years ago you shipped a recentish copy of jQuery, maybe jQuery UI if you were feeling fancy and that contained pretty much every library function you needed. Now we NPM install thousands upon thousands of modules to get by.
These days, browsers have tons of library features baked into them - but with APIs so obtuse and difficult to use you have to install another library to make them usable.
Last week at work I had a 2 hour meeting where we tried to work out how to ship the 2MB of JS our website apparently needs to clients without Google Lighthouse penalising us.
I don't know what to do with any of these thoughts but they often make me want to quit and take up a profession that doesn't involve computers like blacksmithing :/