What taught me the most about this was managing my parents' home internet from a distance. If it went out, they could hardly even communicate with me to fix it. I quickly learned just how many things can go wrong with Ubiquiti wifi APs or other more advanced tools sitting there.
The final working setup: Basic ISP-provided modem/router/wifi combo, cheap near-zero-config wifi APs to reach distant rooms, entire setup on a lamp timer to cut power for 30min a day at 4AM. The thing is, that solution is decent even if you understand the more advanced stuff.
Now at work I see how many SWEs are, in a way, too smart for their own good. I keep telling them, make your solutions less galaxy-brained. No you are not making another custom DBMS-type thing just to serve a few kinds of requests from adjacent systems. If random other people can't understand it, it'll get rewritten in 2 years tops.
> The final working setup: Basic ISP-provided modem/router/wifi combo
And imagine how many person-hours went into making that remotely manageable, when unlike you, they control both ends of the connection!
I have my ISP’s hardware in dumb bridge mode because otherwise it won’t do what I want. And all the things they block or cripple? I firmly believe they were right to do so — few people want to do the weirdo stuff I want.
Like you, I set my parents’ setup to the generic defaults.
I'm curious what kind of stuff they block or cripple that you're interested in. I don't think I've ever run into such a limitation with my Comcast internet. Granted, I have other issues with Comcast, but none of the nature you speak of.
I'm a big fan of straight out of the box solutions. I make a point of starting projects often and from a clean slate and learning to use the tools in their default config. If defaults aren't good, change tooling. Customization is super fun but it's just another layer of complexity at the end of the day.
Every time people say that you need to first install 30 pluggins before you can use gnome I just think “uh no, you don’t. It works just fine out of the box.”
Rather than turning a bunch of preferences and habits in to hard requirements, I prefer to just go with the flow. If the gnome design team says workspaces are now horizontal instead of vertical, I just go with it instead of building a plug-in or fork to turn them vertical again.
Same with Apple products, they are designed well, so just go with it.
I had this revelation around 15 years ago. I was fiddling with custom i3wm setups and complex hotkey setups and my .emacs was longer than some short stories. I even had Windows with a custom explorer replacement that re-did the whole UI.
At some point I figured out that I spent so much time configuring everything at home, but I couldn't do the same for the other multiple computers I used at work (work machine + remoting to client's computers) and that started to grate on me.
I just gave up on customising and started using everything pretty much on the default settings. Haven't regretted it at all.
The only "custom" bits I have is a homeshick[0] setup and a Brewfile that install some basic shell tools for me on a new computer.
I'm with you. If I can't either take the custom config with me (.emacs, for instance), and it goes beyond a few simple things, like changing a couple of defaults on OSX, I just don't mess with it anymore. It's not worth the overhead of either having to use an unfamiliar configuration in other contexts or having to spend the time to completely reimplement my custom tweaks again (which might not even be possible due to IT policies).
> At some point I figured out that I spent so much time configuring everything at home, but I couldn't do the same for the other multiple computers I used at work (work machine + remoting to client's computers) and that started to grate on me.
I get the best of both worlds with nix and home-manager.
I’ve heard this called “the defaults lifestyle” on the Software Defined Talk podcast. I’m forever locked out of nirvana because I have an iPhone but not a Mac, but that’s fine.
Occasionally plugins are worth the expense though. I finally discovered Argos[0] and I’m using it to show time in UTC and a couple other time zones. Super handy.
But if I try some new software and find that I have to tweak it a bunch to make it usable, that means the devs have different aesthetics and I should probably try something else.
> Every time people say that you need to first install 30 pluggins before you can use gnome I just think “uh no, you don’t. It works just fine out of the box.”
Emacs enthusiasts seem to be the same way: oh, it’s so configurable that let me recommend you do all these crazy things just to get started.
Default setup (absolutely no customization) works great for newbies right out of the box.
(And I say this as someone who’s been using emacs for over 45 years; it’s still my daily drive for editing, web browsing, mail reading… but a beginner doesn’t need to know any of that)
if i don't like the defaults in an application i tend to look for alternative apps that have better defaults before grudgingly changing the defaults in this app. i just don't like spending a lot of time fiddling with settings.
the hallmark of good application design is to have defaults that are good for the majority of users and don't require fiddling to become useful.
just recently i decided to look for a better way to switch windows in gnome. alt-tab is not convenient because the window previews are to small. the activities overview was a little bit better but slow because i had to use the mouse (i just found i can use arrow-keys too, but it's a bit awkward).
so i went out to search for an extension that would let me cycle through the full sized windows. that let me stumble over a comment that pointed out that i could do that without an extension but with alt-esc.
> Customization is super fun but it's just another layer of complexity at the end of the day.
Feel like this often is a recipe for fragility too. I used to be big into customizing a linux environment and editors/IDEs for personal use, but often turned into an enormous time sink when troubleshooting would be required just to maintain usability.
Yeah, I constantly heard people at work complaining about whatever minute changes broke their ultra-customized workflows. Past tense cuz they got laid off.
I hate the rewrite treadmill. I'm dealing with this now. One of the APIs we use is migrating to a different API. Of course they completely changed the data structure. So now we have to rewrite entire components (they don't want a service layer to restructure the data), there's nobody left from the original team that wrote it, and the only requirements we have are "make it work like it did with the old API". Surprise surprise, we have a lot of bugs showing up in prod.
This is how you burnout people - have people do boring yet frustrating work, make them do it the hard way, then put them on a deadline with other competing priorities.
> This is how you burnout people - have people do boring yet frustrating work, make them do it the hard way, then put them on a deadline with other competing priorities.
The only way to get off the rewrite treadmill is to have a good API. Half of ours suck because people focus more on the implementation details, also caring more about unit tests than integration tests when it should be the other way around. Similarly, those failing to see the value of a good API end up deprecating it for no reason.
We had zero say in most of the APIs we use. When I started demanding clients take part in our API design, they were pleasantly surprised.
I don't remember the specifics from that long ago, but basically they're too complicated. I didn't even get the settings right the first time, and it was too confusing for my parents to log in and fix things if needed. There was also some weirdness with needing a Ubiquiti account that I'm told you could avoid, but why is it even a thing.
Indeed. Treating every topic as something I should get my mother to do over the phone while driving gives wonderful results. It improves communication, documentation, code, stacks and logistics.
Lol, the ones there don't even have a brand name, so I don't remember what they're called. MAC address manufacturer field just resolved to "Shenzhen," a city in China.
There are tons on Amazon etc that do basically the same thing.
The wifi APs too. And not "will" stop working but "might, and if so I won't be around to reboot it." Also regularly tests that stuff will return to health after a real power outage, and that nothing is depending on ephemeral state like DHCP leases. Basically, I want to experience the same circumstances during the week I'm visiting that they're going to experience for months after.
Ubiquiti APs failed this test. The moment one started an OTA update, my mom temp unplugged it to make room for a blender. It soft-bricked itself. Parents couldn't figure out how to reconfigure it after a hard reset.
I usually SOIC clip/backup the firmware on Ubiquity and other devices we own, before doing anything like this (more recently). Its usually so much faster and reliable than an RMA in most cases for a simple firmware update fail, especially with the post pandemic supply chain the way its been.
Took awhile to get the approval but when you have a large pile of RMAs sitting for 6mo to a year due to a bad firmware update, for only a standard 1-2 year limited warranty, and they stop responding, and replacements were back-ordered for ages. Well, you do what you have to.
For wifi in the kitchen. The walls have wire mesh that happens to block RF really badly, so wifi will only work in the same room. If internet went down, house was also a cell dead zone so nobody could talk to me while looking at the router or something.
Usually the power brick is poorly isolated and/or you have dirty electricity, it results in errors on an ongoing basis that get logged to the small amount of onboard memory. (not usually a circular buffer.)
When the errors have sufficient volume the memory fills up, and it stops working at resource exhaustion.
In a real pinch, I've resolved this in some of the more milder cases by simply using one of those walmart plug timers that turns off for a few minutes in the morning when everyone is asleep.
Well, at least they were on UPS and got power conditioning that way. But there's also just crappy programming. It's not only ISP-provided stuff, it's cheap home routers in general. I don't trust them.
> It's important to keep in mind that most of us sit not far from the middle of the bell curve. Most people are, by mathematical definition, close to average, as skill distribution is not from Taleb’s Extremistan.
Is this true? Why should I believe it? Certainly unclear to my sub-average brain why it's a "mathematical definition." I'd call it an empirical fact, but has anyone bothered to measure? Logically speaking, the curve's hump could be biased one direction or the other, or bi-modal, or whatever else...
Also unclear to me why it follows that "top of the curve" people can't explain things to their lessors. Sure some may not possess that skill, but you're telling me there aren't good teachers/tutorials out there? My brain's too soft, so pack it up? Seems presumptious.
Anyway the message of "know yourself and your limitations" is fine and well.
PS. Thanks for noticing I'm a tad above average, statistically speaking!
Nassim Taleb has a great bit on it: characteristics of "natural" systems tend to settle on gaussian distribution on a long enough time frame. Weight, life span, force, height, skills... On the other end, systems that we have created have metrics that tend to not follow the normal law: money, electricity consumption, size of houses, etc.
It's not a perfect match and I'm not trying to write a white paper on anything. It's just a way to warm up the reader to the essay point.
Great point, I could totally see the "skill of an IT person" being skewed towards the majority being "bad" and a few being "good". If nothing else, simply due to the fact that there are fewer people with more years of experience
After reading this article, is it strange that I've always considered "IT" a separate discipline from "software development/engineering" ? In these parts, the connotation of "being in IT" means you're managing networks, servers, fixing people's email, etc.
I think this is a cultural distinction that depends on where you are. I have seen many Europeans group both engineering and “operations” stuff like help desk into IT. Whereas in the US we limit IT to only operations and in practice may also exclude highly technical operations like SRE or devops from the “IT” bucket.
I see it as like distinguishing the engineers that design cars from the mechanics that help people with their cars. The distinction blurs when your “mechanics” are operating on bespoke and highly complicated machinery that may have maintenance that includes extensive engineering knowledge (like race cars, or large backend production systems) or requires the operations to be done by the same engineers that created it. But being an IT person that specializes in printers, or internal networks, or Windows configuration and policies is more firmly in the “mechanics” area. Which is not to devalue their skills as setting up a big network can be very complicated and difficult; just as being a mechanic on something like a nuclear submarine is also probably pretty complicated and difficult.
I don't think so, that's the connotation I usually see. I've always found that dichotomy rather odd though; I work in systems design/administration and I think a lot of common development mistakes could be avoided if devs had more extensive support experience.
I frequently see line-of-business software that's likely self-explanatory for a 20-something developer, but that's nearly incomprehensible for the target audience. In terms of basic computer literacy, most developers are far above average.
For example, a friend of mine had to teach his elderly coworker at a machine shop about copy/paste. Not the keyboard shortcuts, not how to do it, literally the concept of the system clipboard. He had learned computing on DOS, and had learned everything since by rote memorization.
That's a rather extreme example, admittedly, but it does illustrate just how unpredictable the skill level of end users is. Even a minor UI redesign can throw off someone like that for weeks, and I don't think most devs fully appreciate that.
I think you're right, depending on how you define "IT".
The operational part of IT wants zero change and high levels of predicability to maintain reliability. It thinks long term.
The development part of IT wants to add features and functionality to satisfy business requirements, which means lots of changes. It's forced to think shorter term.
We still lump these two disparate parts together. I wonder if having the operational part report to the COO, and the development part report to a CIO (who ideally is a peer of the COO), would work better. I've seen a lot of organizations, but never seen this organizational structure.
This is why DevOps became so popular: throw the two types of people in one team and make them responsible for the whole lifecycle of the products they develop and maintain. Report to whomever requires those products.
It all really depends on your definition of IT person.
Some IT Helpdesk people are most definitely not IT, they just have a knowledge base they look things up, assuming they can' understand what you are saying.
Many companies offshore this to places like Jakarta, and you are left with really dismal service many times, and non-existent escalation paths almost guaranteeing corporate escalation for important issues.
This hits home quite hard. I get depressed sometimes working on teams where I'm the most technically proficient on the team and I want to solve a problem in a way which will save everybody not just a little time, but probably save us enough time that we could remove one or two staff.
But then I remember how I got the skills to do such things. Hours of work a day for years at work. Hours of interest a way in my personal life. I did that for decades and I had a natural knack for such things. So it turns out the solutions I build would fall apart without me holding them together. As soon as I leave whatever I make will be thrown out. I'm not good enough to unilaterally make so tools that they are easily maintainable, debuggable, extendable, and so on by just anybody you hire off the street.
The more I understand how to reconcile these problems, the more I understand managers and architects.
This makes a simple point I've needed to hear for the past thirty years. Thank you.
I was just saying today how realising you have absolutely no idea what you're doing is a prerequisite for competence in tech, and you need to watch out for the ones who don't realise that.
"I know only that I too am a dumbass," as Socrates said.
The topic of information disparity is highly interesting to me. Knowledge workers in general have very interesting challenge relating to other workers at different expertise levels, & to others in the org. Each person is somewhat rather isolated in their own knowledge domain & that is the highest loss transmission medium I can think of.
I think there's a lot good points for consideration in this article. It mainly favors a KISS approach. Reciprocally, I hate to say it, but I also feel like there's an overly compulsive YAGNI behavior (you ain't gonna need it). Developers are drawn like hawks to a possible whiff of over complexity, & will descend for the kill. Ambition is talked down & avoided by many voices in most orgs, even when the plans can be quite sound.
What can be really sad is that some devs will convince themselves of doing all kinds of absurd highly compromised labor that they know is a better choice than a slightly more adventurous route that requires some discovery & exploration of known unknowns. The last principle engineer we had kept demanding to make Api calls for a migration because they didnt feel very comfortable with sql, but my heavens the work we went through to do the job thousands of times slower to bend to that was staggering. People will tool away doing things poorly, slowly, repetitively, glued to the safe path, and not advancing.
Becoming a really great developer is hanging in the balance, between letting yourself spiral & lose traction/understanding/focus/being-overwhelmed, and between being relentlessly curious & hungry to learn all the options, how each option works. Finding stuff out is the job, at least as much as doing the work, but training ourselves towards initiative is the hardest practice of all.
There's plenty of other ways to be a very fine developer too. Not everyone has to sign up to contort themselves, to a predisposition of being always staying slightly uncomfortable. But those that can start to lean in & appreciate the endless possible quest-opportunities about us, & who can latch on & dive deep on an option, courageously go find out and see: those folks tend to do very very well. (If placed in an environment where this let's say "ideal" behavior isnt resisted by the org.) It's an amazing life.
> Developers are drawn like hawks to a possible whiff of over complexity
In my experience, juniors tend to pick solutions without considering complexity and people factors — at least, I did. There’s a very good reason to avoid over-complexity: if you have too much of it, it kills your velocity.
> to a predisposition of being always staying slightly uncomfortable
I love that feeling of always moving forward. Tech that you didn’t know about a year ago, is now part of your normal workflow. That feeling when you start exploring something totally new and it takes a week to get comfortable with hello world, and a month (or years) to learn exactly where it should fit and what the sharp edges are.
In all fairness, to me this reads as more of a rant man-splation for why thing's aren't productive, and largely misses the point.
Its also fairly niche and specific to developers that have a very developer-only mindset. Anyone who has spent time and designed systems with the bare-minimum feature sets needed to make operations supportable understands that most of the examples provided are very brittle, and purely opinion with little backing them. Its an overbroad generalization with a poorly thought through hypothesis that doesn't really match observations I've seen.
I've worked as a System Admin and System's Engineer over the years, most of the issues are solved when you follow good documentation practices.
If you aren't generating documentation in a way that others can read and understand, you simply aren't doing your job.
A proper IT Operations team understands this, the developers in general that I've met seem to think these rules don't apply to them, this includes the supervisors managing the business unit, and then they raise holy hell when SHTF like it wasn't a completely predictable outcome. There are exceptions but they are just that, 'exceptions'.
Average people should be able to read, and comprehend documentation needed for their position. If you can't do that, you aren't doing your job and deserve to be fired for misleading business about your skill set.
If you don't have good documentation for an existing system, you need to generate that first before you can really do anything new, its deferred maintenance that needs to be paid. Its impossible to put any reasonably accurate scope of work together if you have no documentation on a system, and only generalized requirements. Documentation is often a critical part of the initial requirement scope/process (before any work gets done).
You skip this phase, and you are tacitly approving a project that either will never complete, or will complete egregiously over budget.
Honestly, based off what you said, in my opinion you clearly lack credibility.
The very least you could do is properly quote what I actually said instead of poorly paraphrasing, and then holding that paraphrase up as an example (with a different context). That's no better than putting words in other peoples mouths, its just flawed reasoning, and the tactic comes off as plainly manipulative and biased.
If you read and comprehended what I said, and knew what good documentation practices are, you'd agree if you had the experience.
Every single time I've seen a developer say differently and get their way, its resulted in project cost overruns by integer scaling factors greater than 1, many times in the double digits, assuming the project even completed (some don't).
Availability of good documentation on existing systems makes and breaks project budgets.
I really don't see how you can reasonably think this is a good example, let alone a great example. You of course are free to think what you want.
Fundamentals are covered in Chapter 31 of the Limoncelli books (TPOSA) covers the broad strokes, and Redhat Documentation pre-IBM takeover are examples of good end-product from proper documentation practices.
Em, this isn't meant as a snipe but has to be said because you clearly didn't read and comprehend.
Read what was said, and make sure you look up the definition of words like over-generalization beforehand (its a logical fallacy). If you can't pass reading comprehension at a grade school level, there seriously is almost no skilled job your suitable at. This is basic fundamental stuff. You need to do this for almost everything in a professional environment.
> And these people, there are the ones telling the average rest what they should do
Following that reasoning, you can literally claim anyone who communicates anything is telling someone what they should do, and the author tends to fall into this bias to group people doing just that. So anyone they want or perceive as negative then falls into this group which in fact encompasses all humanity (because communication is inherent in almost everything we do). This is literally in the definition of over-generalization, which is what I said (if you go back and read). Its also very common and manipulative tactic in identity politics often used for malign infuence against those that don't critically think to dupe them.
Unfortunately, this lacks any kind of rational thought, is biased and follows flawed reasoning, and as a result lacks any kind of credibility, and rightfully so.
Man, this article hit me hard. Years ago I wanted to blog about more complex subjects to a beginner audience. As I built my mini blog series, I started to realize there was so much to explain. It took me much longer than I thought to write it. I decided it was way too much work and decided to never write tutorials again. Kudos to those who do.
Well that is a very common trap with writing, not choosing an appropriate audience for what you are writing about will send you down a rabbit hole.
There are some things you simply cannot explain to a beginner audience because they lack the fundamental context for comprehension.
Some may ask questions, and give it a good go, but really without that context its an unproductive use of both parties time. Some will misunderstand this as being a gatekeeper but it really has nothing to do with that. Its a problem you only really accept and learn through experience.
The final working setup: Basic ISP-provided modem/router/wifi combo, cheap near-zero-config wifi APs to reach distant rooms, entire setup on a lamp timer to cut power for 30min a day at 4AM. The thing is, that solution is decent even if you understand the more advanced stuff.
Now at work I see how many SWEs are, in a way, too smart for their own good. I keep telling them, make your solutions less galaxy-brained. No you are not making another custom DBMS-type thing just to serve a few kinds of requests from adjacent systems. If random other people can't understand it, it'll get rewritten in 2 years tops.