I think this is one reason why doing your design work on pen and paper in a quiet place, before you sit down to write any code, can be a game changer. Thinking on the fly and trying to keep a bunch of state in your head is difficult. Mathematical notation is super powerful and there's a reason it exists—use it!
I also find that some programmers' bad habits naturally lead to "fragmented" thinking or lack of sustained focus. A lot of beginning programmers I encounter are sort of like information mice. They have some specific problem that they need a quick answer to, so they hunt around and scavenge for it, skimming various resources like crazy until they find it. While this works for basic, entry-level work and tasks, the same behavior is a complete achilles heel as you progress in you career and have to solve more complex problems. It teaches you to abandon detailed, sustained, focused thinking and building of deep understanding—the kind that allows you to solve problems in a novel way—for a mode of consumption and thinking that is shallow, fragile, and built upon expectations of instantaneousness and immediate gratification. I realize that having the time not to settle for scavenging is sometimes a luxury, but resist the impulse if you can!
This is wonderfully put, and it perfectly expresses why I'm frustrated with the current state of my career. In terms of fast/shallow vs slow/deep thinking, I've gone through a bell curve of sorts.
When I was a junior, I did exactly as you said - skimming around Stack Overflow every few minutes. But as I progressed, I did that less and less, and entered into a blissful state of my career where I felt myself really sinking into deeper and deeper concepts.
But in the past 3-5 years, something has changed. I don't know if it's me, or my industry (frontend web dev), or both. But it just feels like there's this overwhelming amount of constant change and detail, and I feel like I simply don't have the time or energy to keep up with it all. It's forced me back into a kind of frantic work process, and it makes me just want to give up. I've honestly thought about just trying to get into OS-level programming just so I can work with something that doesn't change every 2 weeks.
The cost of change is real, and I think you've touched on it.
When I started developing, Stack Overflow was in its infancy and occassionally had helpful answers, but "search the internet" was usually the fourth or fifth option when you couldn't find answers using the following:
1. Look through the documentation and/or man pages
2. Look through (one of) the book(s) you have on the subject
3. Ask a knowledgable co-worker (depending on what you're doing)
4. Try a few things to see if you can figure it out. If the source is available, dig through the source code of the framework you're building on. This was especially useful when it was a Qt-related question for example.
So I came up with a balance of having theory/foundation in place before starting . There were pros and cons to this approach of course. It meant a bigger investment and higher barriers of entry to learning a new language/framework, but it also meant I would really learn those things and understand them deeply.
The javascript world around 2011 or so when I got heavily into it (more than just jQuery, I started learning Backbone.js) was a change. It felt slower then than it does now. My approach of book-based learning worked great for learning Ruby and Rails, but not JS stuff. There just weren't many great focused books, and when I found one it would be outdated quickly or lose relevance (for example, because we moved to Ember.js, and then to React, etc). Eventually I did get the foundation in JS, but it was painful to get at especially with how many levels of abstraction there is (or used to be, now that browsers support a lot of ES6 and above features it's much better).
Anyway, all that to essentially say that yes, the javascript world makes this very hard by being so fast-moving and mercilessly fragmented. I feel mostly insulated from that world now that I'm mainly doing Elixir (Phoenix and LiveView), but if you're doing anything web it's impossible to avoid it entirely. If Alpine.js disappears I'll be much regressed!
Same, I started writing JS around 2011. Back then, even Grunt and Gulp hadn't been created yet. I was literally optimizing my files by hand, and deploying by dragging my files via an FTP client. It was tedious, but also transparent.
Software development keeps expanding in functionality and the level of detail. Twenty years ago you made a simple desktop application and that was it. Today, the same application would need to be online, which of course requires REST, authentication, load balancing, etc. You don't save information in a text or binary file, you need a database, which means object modeling, transactions, indexes, partitioning, etc. Instead of writing information to the standard output, we have log file, dynamic configuration of loggers, log rotation, infrastructure to move the logs in cloud, etc. Just editing the source code already requires several tools, including version control, which includes conflict resolution, pull requests, pre-commit hooks, etc. Nothing is ever simple; any functionality can be expanded into a separate library, parts of which will soon require libraries of their own, etc.
But ironically, instead of specialization, the trend is the opposite. Everyone is full-stack dev-sec-ops working in several different programming ecosystems, sometimes on multiple projects in parallel. You also do the testing and write the documentation, of course.
At the same time, you are supposed to be agile, and sprint, sprint, sprint, until you collapse. You can't really spend a week just sitting in silence and studying one of the hundred technologies you are supposed to master. So instead you learn everything on the fly. But as you keep jumping between thousand topics, it becomes impossible to remember them. If you accidentally memorize a library, it gets soon replaced by something new. So you keep googling endlessly, and maybe soon you will just keep pressing Ctrl+space and letting an artificial intelligence write your code... and then pray that it compiles and contains no bugs.
Makes me wonder whether it really increases the productivity, or just creates an illusion of greater productivity because people are kept visibly busy. I guess we keep writing more, but a lot of that is rewriting, or writing the supporting infrastructure.
If this keep accelerating at the same speed, in another twenty years only the artificial intelligence will be able to write code. It will use a different programming language every week, reinventing all the tools and libraries all the time. Bugs probably won't get fixed; humans will no longer be capable of that, and if the AI couldn't write the code correctly in the first place, it would be probably simpler to just throw the entire application away and rewrite it from scratch, using this week's programming language instead of the previous week's one. The resulting applications will be slow and require the latest hardware to run.
> just feels like there's this overwhelming amount of constant change and detail
The good news is, you can mostly ignore it. I have 20 year old server-side rendered apps that are still running just fine today. Why run on that treadmill? Find something that looks solid and makes sense, and stick with it for awhile.
For my own tech yes, I can ignore it. But for job opportunities, the unfortunate reality is that recruiters, hiring managers, and interviewers will be looking for knowledge about relevant tech. I don't want to learn NextJS, and you will not catch me trying to use it in my free time. But if one day I apply for a job that's looking for people with knowledge of NextJS, I probably won't get that job. And if it comes to pass that the majority of the industry moves to NextJS, I'm now at significant risk.
Of course, when I hire I look for fundamental skills, not familiarity with XYZ technology. But I've seen people hire that way, and I know it's fairly common in our field.
> And if it comes to pass that the majority of the industry moves to NextJS, I'm now at significant risk.
I think "significant" is overstating it. Learning new tech is not that hard, particularly if you're already familiar with adjacent tech. I recognize that some people look for checkmarks on a CV, but if that's how they operate then you probably don't want to work there.
Does reading a dozen papers on different algorithms to accomplish a specific goal count as scavenging. Asking for me because I've gone down a rabbit hole with recent work. Not trying to be antagonistic or humble brag either. Actually not sure what I'm trying to accomplish with this question.
I think most papers should effectively just be scavenged through. In (mathematics) grad school I’ve only had time to seriously study a single digit number of research papers, but I at least feel like I gained a lot from looking through many many more than that.
John Cassian's "The Conferences"[0][1] is a collection of interviews with the Desert Fathers, and one of the subjects discussed is that of focus and concentration (a selection of excerpts was recently compiled into a small book entitled "How to Focus", a review of which was posted here on HN two months ago [2]).
At one point, one of the monks interviewed by John Cassian and Germanus (either Abba Moses, Abba Serenus, or Abba Isaac, all of Scetis) compares the mind to a millstone driven by a water wheel. As long as the water flows, you cannot stop the millstone from moving and grinding whatever has been put into it. We cannot stop the millstone, but we can control what we feed into it and what is ground up by it.
Of course, there is also a long tradition of mortification of the flesh ("The Conferences" also discusses such things), crudely parodied and ridiculed by bad literature and trash Hollywood films. The basic understanding is that our bodily appetites can be unruly, grasping, wanting, desiring, imprudently and with no regard for the objective good that the intellect knows. Mortification is a way of training these insubordinate appetites and passions into obedience to the intellect through exercises of denial. Fasting is one example. An element of discomfort, and even suffering, can be involved as the lower is sacrificed for the sake of the higher, and the appetites throw their tantrums as we persevere and dominate them, reigning them in like a man on horseback reigns in his horse.
Our culture has sold us the idea that indulging our whims and desires with no regard for our objective good is freedom, but nothing could be further from the truth. This kind of lifestyle is a recipe for misery and enslavement, both to the chaos and disorder of unruly and even disordered appetites and passions, but also those who gladly exploit the appetites of such weak and blind men, herding and using them like mindless beasts. As Augustine of Hippo wrote, "a good man, though a slave, is free; but a wicked man, though a king, is a slave. For he serves, not one man alone, but what is worse, as many masters as he has vices." The relevance to focus here is that if you do not learn to exercise restraint and self-mastery, you will have trouble with focus as your appetites and passions will pull you away from the matter at hand. Perseverance is thwarted by such softness [3]. Pour only good things into your millstone. Do not pollute your millstone with darnel. Endure truth, whether pleasant or painful. Let is grind away your marble you bring forth your sculpture. It is not mastered, repressed appetites that come back to bite you. On the contrary. It is repressed truth that does.
I also find that some programmers' bad habits naturally lead to "fragmented" thinking or lack of sustained focus. A lot of beginning programmers I encounter are sort of like information mice. They have some specific problem that they need a quick answer to, so they hunt around and scavenge for it, skimming various resources like crazy until they find it. While this works for basic, entry-level work and tasks, the same behavior is a complete achilles heel as you progress in you career and have to solve more complex problems. It teaches you to abandon detailed, sustained, focused thinking and building of deep understanding—the kind that allows you to solve problems in a novel way—for a mode of consumption and thinking that is shallow, fragile, and built upon expectations of instantaneousness and immediate gratification. I realize that having the time not to settle for scavenging is sometimes a luxury, but resist the impulse if you can!