I swear this guy exists at every company I've ever worked.
He's the guy you go to when you find some legacy code which you have no idea how the hell it works and end up getting a 2 hour history lesson into a decade of company politics and failed replatforming projects.
"... And the AIX machines were a real beast. We had 20 of those things, and each had its own 20 amp circuit. Had a tendency for the grounding to the steel case to fail. If you wanted to upgrade it you'd first need to grab a pair of thick leather gloves...."
You kids sitting there. All you do is change your EC2 instance size and restart it. Back in my day I had to go and install the RAM by hand. And you know 128Mb of RAM then took up a whole suitcase and weighed more than your laptop
The I fall asleep at my desk, periodically waking up shouting "STOP-A!" and hallucinating about being bitten by thick ethernet vampires.
When the Sun 4/330 I used to have needed a RAM upgrade someone flew up from London to Edinburgh with the RAM and installed it. Fortunately we were in the Grassmarket about 10 minutes from the Castle so he had plenty of things for him to do to keep him amused until his return flight.
Edit: I checked and the max the 4/330 could take was 96MB so I suspect the upgrade was an awesome 64MB. That machine cost more than most cars I have owned - even without allowing for inflation.
I worked in some ex-USSR government facility, and they had an old hp-9000 system, that needed ram upgrade. HP wanted some ridiculous thousands of $$ for like 16MiB of RAM. After some inspection it was discovered that the RAM is nothing more than 72pin SIMMs. We took some ram from nearby old 486 and it worked.
> HP wanted some ridiculous thousands of $$ for like 16MiB of RAM. After some inspection it was discovered that the RAM is nothing more than 72pin SIMMs.
Yeah crazy money. My father used to import RAM when it was really really expensive in the 1980s. He used to take it on planes from Taiwan as hand luggage handcuffed to himself because it was that valuable.
This brings back memories. I got my start in parts wholesale selling 30pin SIMMs in static baggies out of my leather jacket in a Taco Bell LMFAO. It took some convincing to convince the management I wasn’t selling drugs, but the getup helped me to not be noticed by people that might otherwise know that I was carrying thousands of dollars in cash and chips daily.
I would sit there and wait for RAM hungry customers that would call me on my 3 lb cellphone lol. Each one has to buy at least a $0.39 taco so I wouldn’t get kicked out by the manager. They had armed security, who I tipped 20 dollars a day. I hung fliers all over town with tear-off tabs.
Then I went to COMDEX, it changed my life. Within 2 years I was in my early 20s and moving 2-3 million in hardware a year. It was a heady time.
Then, the business dried up overnight with online sales.
Fortunately I saw that coming and had moved into technical services and contracting, and sold the parts wholesale business to one of my competitors lol. The local wholesale distribution biz was dead within 8 months.
Hahaha hooky RAM dealer. Much respect though for being there at the start. I used to do something similar with pirated Amiga games but that was a limited market!
At first, it was an opportunistic grift by a teenager looking for party money.
As prices fell rapidly at the supply level, local box stores would occasionally sell off stock at deep discounts (reflecting the real market price). Being a teenage sociopath, I would buy every last bit of it in town and then coast on the uninformed price perception for a week or two.
Prices were falling so fast at that time that the stores were always wrong footed, so I was able to keep up that racket long enough to get to the bottom of that mechanism and start ordering in bulk directly from city of industry California.
Place was called Ma labs, iirc. After that it was obvious to start moving disk drives, video cards, processors, and later everything else too. I was just exploiting basic market inefficiencies , since the only other sources were the big box stores that were constantly 2-4 months behind the price curve.
Then COMDEX happened. It got me tied directly in to motherboard and peripheral card manufacturers. I also got introduced to the cocaine and extremely friendly booth-girl shmoozefest, and NGL for naive Alaskan me it was heady stuff. The real deals and the points negotiations were made in the afterparties with piles of coke and very persuasive “account representatives”. Memorable times were had in the hedonistic environment of an exploding industry.
At that time there were a lot of small manufacturers in industrial parks. They were usually pretty small outfits usually just a shop with a few hot air stations for warranty work and a wave soldering table, 15-20 employees at jet labs, for example, which was one of our big suppliers of motherboards, video cards, modems, lan, and other peripheral cards.
My activity festered a plague of tiny pc builders selling systems, and launched the careers of at least 50 technicians and small businesses. We used to put on big public LAN parties and public PC repair clinics as promotions for our “dealers”. I worked with a training/testing outfit to get new dealers A+ certification to reduce the damage to society and actually grew into a semi responsible citizen during that time.
Fond memories but I’m glad to leave that hustle behind.
After that, I unwittingly became a software developer for a major criminal organisation. I thought they were a car dealership, but gradually it became apparent as I built out their backend that the dealership was just a small part of a very large laundromat.
It took me years to extricate myself from that and for years afterwards they would give me gentle reminders that they were keeping an eye on me. Free 500 dollar an hour lawyer’s though. They never let me get near a courtroom or a police interaction without “proper” representation showing up even for years afterwards (statute of limitations?)
The fact that they always knew when I brushed up against the legal system in any small way, even traffic tickets in rental cars, gave me great respect for the scope of their concern. That and their obvious love for mid range turbine helicopters.
I worked in a shop in the early 90s with a bunch of Suns, and we had hired this new sysadmin. One day we noticed that half the RAM was missing from a workstation, and few minutes later noticed half the RAM was missing from all the workstations. Turned out the new sysadmin was a cokehead and had stolen it.
Oh man that's funny. Only because something similar happened to us. Hired new cleaners circa 2001. Got into the office one morning and all the PC cases were open. All the RAM and CPUs had been stolen.
In the mid-90s, when CPU heatsinks didn't need to be screwed in, thieves discovered that the new ZIF/zero-insertion-force sockets also make good zero-removal-force sockets.
When I was in Korea in the 90s you could get LG RAM for about half what it would cost in the US. I had a scheme to buy a bunch of it to sell when I came home, but in retrospect it would have been the nerdiest case of "guy goes door to door trying to sell rapidly expiring meat".
I went to the closing sale of a local computer store that had been in business since XT days.
Owner lamented finding 4MB sticks of RAM that had fallen behind the workstations that had cost hundreds of dollars back in the day, as he was selling (and this was a while ago) laptops with 1-2GB of memory).
Continuing with the Monty Python vibe, the first code I wrote was in machine code (ie hex, not even assembler). Though I worked with a guy who wrote machine code on a magnetic drum and they used the delay of the rotation of the drum for their timing. The good ol' days.
My thumbs still have blisters from manually stuffing memory expansion boards with 1Mb of DIP RAM for our 80286 PCs. We were doing image capture and processing, long before it either usable or useful.
"Back in my day, we only got one compile a day with our punch cards. Now you can compile willy nilly without having to put in the forethought we did" - ex coworker
> Back in my day, we only got one compile a day with our punch cards
You were lucky! In my computer class at school in 1979 we got one compile a week. Punched into cards which were taken to the local insurance company who actually had a computer.
Stop-A is the key combo you hit on Sun workstations to drop into the ROM monitor. Usually when they are locked up solid with a crashed Xserver and need a reboot.
I did a real internship (as in it was for my diploma and lasted only a few months) in a medical facility operating Big Machines. I remember a guy soldering a new battery to some old DEC/Digital hardware.
"They were afraid to turn off or reboot the DEC Vax for fear that if the disks stopped spinning they wouldn't start again. I saw it once at the retirement ceremony and it didn't look like a server but more like a set of industrial washer and driers. The users transferred files using the Kermit protocol and printed locally by a telnet client that could understand the special characters that redirected character output from the terminal to a lineprinter."
An old IBM tech told me that hard drives failed when power cycling equipment could be temporarily resuscitated by placing the drive on the floor, twisting it hard by hand, then as quickly as possible connecting it back into the machine and powering on. Apparently getting the platters to spin helped them get around a worn out motor that couldn't make the initial spin-up.
I remember upgrading from a 4.3 GB HD to a 20 GB HD around 2001. I had the old hard drive sitting on my desk, and the new hard drive sitting in the case, both attached to the motherboard via IDE ribbon cables.
Most of the way through backing up the 4.3 GB HD to the 20 GB HD, I heard a screech and the old hard drive twisted slightly on the desk... conservation of linear momentum. The drive was visible to the BIOS, but refused to spin up after that.
I put the hard drive in a ziplock bag, put it in the freezer to give the parts slightly more clearance from thermal contraction, and slammed the hard drive nice and hard on the desktop to free the stiction. That revived the drive long enough to finish my copy.
I did have backups of all of the really important information, but restoring from a stack of floppies is more tedious, slower, and less fun than percussive maintenance on a hard drive.
Back when I had an internship working on MEMS gyros for GPS guided mortar rounds, we had to sometimes perform similar percussive maintenance if static electricity had caused the moving parts to contact the substrate. We took the gyro, and smacked it hard on the desk in an "eyeballs out" orientation to give it something like 10 to 100 Gs of acceleration in an attempt to un-stick the MEMS gyro.
Yeah, did same in 1999 or 2000, but we _did_ _not_ take the HDD from the freezer, just pulled the ribbon and power cable through the narrow slit between the door's rubber gasket and the fridge's body.
Yes, stiction is a real issue in old hard drives. In fact something that used to work well if you were cold booting an old server even in the mid 90s, with the expectations of an immediate migration, was giving the drives a good sharp slap to help break stiction so they could spin up. The only issue is it could cause a head crash. Another fun trick from back in the day was sticking hard drives in the freezer to cause the platters to shrink enough to dislodge the head and restore the air gap if you had a head crash so you could try to recover the data.
As much as I love hardware, I much prefer our current solid state wonderland.
> to shrink enough to dislodge the head and restore the air gap
Huh?
What I heard is what a cold drive would be a bit more magnetically stable and IMMSMV that surely worked, because I never had a drive with a stuck heads but I did had drives what would just abort the read or return gibberish, but after an hour in the freezer they would read just fine, till they heat up again. That's for the drives mfg after 2000, if that matters.
My understanding may be incorrect, after all the freezing the drives trick is mostly something that was shared among sysadmins like an old wive's tale. Nonetheless, it does work (or did).
The way I had always understood it was that if you had a head crash (which was caused by the head physically contacting the platter, overcoming the resistance of the air gap between the head and platter, usually due to physical impact) that the magnetism of the head would prevent it from lifting back up on its own, and that freezing the drive would cause the metal in the platter to contract away from the head, which was restricted in movement by its armature, thereby restoring the air gap and lifting the head away. If you started the drive spinning before it heated, the head would stay out of contact and you could successfully read data (some of it, for awhile).
Every Friday I had to take a bunch of magnetic tapes to the bank so they could look after our backups ( which was a good excuse to pop into the pub next door to the bank...)
Oh, and we never ever tested that the backups actually worked
My first job out of school was engineering consulting, writing software models of pre-production military radio hardware. The client was a military contractor, and they paid a crazy sum in order to have 3 DAT tapes: one tape always in the drive used to make the daily backups of the shared working directory, one tape always stored in a bomb-proof bunker, and one tape potentially in-transit via courier. I think it was once a month that the client's sysadmins rotated in the "unused" tape, sent the latest tape to the bunker, and waited for the old tape from the bunker to return as the "unused" tape.
Due to absurd government requirements, we were not allowed to use any version control software other than PVCS, and PVCS could only be used for storing official releases of the models. So, the last person in the office on Friday needed to make a dated zipfile of the shared project working directory.
So, one day I convinced the project manager to just let me rename our working directory, create a new empty one in its place, and tell our sysadmins that I accidentally deleted the contents of our working directory. It turns out that the client's sysadmins hadn't set up anything to expunge the oldest backup and had dutifully ignored the alarms that the tape was full. So, the latest available backup was from about a week after they last changed tapes!
The senior sysadmin prominently had a sign on his cubicle reading "Programmers are the ditch diggers of the 21st century." Those same sysadmins were supposed to keep the server room door open while I reinstalled Solaris on a Sunfire V1280 via serial connection from my laptop, because there were cables carrying TSSCI (Top Secret - Secret Compartmentalized Information) traffic. However, the V1280 consumed so much power that the server-room was under-cooled, so the sysadmins illegally shut the door to keep their cubicles cooler. They had top-secret clearances. They knew their obligations to keep me away from those TSSCI cables. Those lazy lazy arrogant sysadmins.
Anyway, unless you have recently passed a restore-from-backup drill, you don't really have backups. Even if you pay absurd amounts of money for sysadmins with top-secret clearances and absurd amounts of money to store your backups in a bomb-proof bunker.
Do you know why I am giving you the two hour history?
First, I recognize that I need human interaction, but do not really like people.
Second, I am hoping that you will learn & understand instead of just memorize & regurgitate so in the future you can resolve it yourself instead of bothering me.
Now, go away; I had my week's fill of socializing.
For me, it's usually trying to rapidly justify the existence of whatever Rube Goldberg machine I'm having to explain. There's always reasons behind the madness, even if they're not particularly good ones.
> Second, I am hoping that you will learn & understand instead of just memorize & regurgitate so in the future you can resolve it yourself instead of bothering me.
Where does all this hope come from? I’m barely 15 years into my career and I’ve nearly given up…
I've started to attempt to gently grind that into juniors. After a certain point, hope is a vice you cannot afford. It is simply something you outgrow, if only because of the unceasing retirement from the mortal coil of those that can bail you out.
Hope is for the young and inexperienced. Everytime I come to your rescue, you'd best pay attention. I have an expiration date.
I’m not trying to explain why you shouldn’t change it. I’m trying to help you understand what competing forces made it that way in the first place so you can decide if any of that’s still relevant.
I’ve seen a few of those. Some are worth their weight in gold. Most are set in their ways from the trauma of trying to make everything not fall apart for so long. Almost all have stagnated and are incapable of bringing in new ideas. Almost all are husks of human beings from putting up and personalizing so much of company dysfunction
This reminds me, in a sad way, of a friend of mine who stayed on when a startup we worked at was acquired by a huge corporation. He has an amazing mind for technical detail, and at the startup, he planned and executed some hugely ambitious technical initiatives. But management at MegaCorp treat him as an obstructionist. They want to announce an initiative, launch it, declare it done, make up a big dollar figure for the "savings," and gather a promotion. They don't need the initiative to make sense or work in a technical sense. But he understands the immensely complicated legacy context, all the work required to accomplish anything, and all the things that will break if they leave out parts of the work, and he won't keep his mouth shut. He would love the challenge of running a big ambitious project, but the only way to get anything big approved is to drastically underestimate the effort, so he's stuck rearranging deck chairs until they inevitably eliminate the product and lay him off.
A real life application of Odysseus's Oar. I have mentored junior devs who have "heard" of SVN and also think CVS is just a pharmacy. It was a real gut-punch the first time.
They also tend to be brilliant engineers. It’s a heck of a lot easier to always be writing 1.0 of something. It’s a hellaciously difficult engineering task to improve a system that’s been improved upon for decades. These folks should be admired and respected, they’re by far the most crucial people at any company older than 30 years old. Instead we get condescending garbage like the linked article.
I just wish hiring managers saw it the same way. It's hard to explain how your last major project was a web enhancement that would have taken a couple days on a virgin codebase, because for you it involved a month of excavating through multiple strata of code dating nearly two decades old, spread across three codebases, two backend languages, a code generator which no one working there understands anymore, and a library who's documentation now only exists on the wayback machine.
A secret I’ve learned is you only need one job, so you don’t need to take just any job or be hireable at every job. It might take longer, but finding a manager that “gets it” is very important. The most important thing is to find someone who recognizes further that a team is more than the sum of its parts, that each individual is unique and not a cog to be evaluated on a metric rubric and forced into compliance with some idealized engineer. They do exist - I was one for many many years. I’ve recently given up in disgust though and switched back to IC, but senior enough I can shape how managers manage engineers. Regardless, my advice is, if it’s helpful, find the right job with the right team and the right company. It’ll take longer, may pay less at first, but finding the right place that respects work at its value not at its marketability is key to any successful career.
Or a library who's code now only exists on the wayback machine... and of the modified version that you actually use only a single copy exists on an old machine that one of the devs in the company never bothered to hand in for recycling...
Never mind that it's part of a decade old contract stipulating a quarter century support...
I wish people like this would architectural decision records so people inheriting it later wouldn't be left in the dark by the decisions made in the system
Because sometimes on the outside systems look confusingly written, but it could be a reflection of the scope of confusing business rules to begin with that they had to scope out
What I propose wherever I go is writing a deprecation plan before production release. It may not be kept up to date over the lifetime but at least a plan for what it would look like to exit to another platform exists documenting assumptions and one way doors, why things were done, and what a replacement would look like and why. I’ve never had great success though, people are too eager to make the next 1.0.
It doesn't seem like a bad way to go. If the company is stable enough and you can cash in your engineers salary for 25 years and have crazy good job security.
Seems better than being a 50 year old trying to wow a 28 year old interviewer with your skills on whatever hot new framework just came out.
...these OS/2 machines were once interconnected by a networking technology called 'token ring'.. no joke. It was basically an entire network where if one single node on the network misbehaved it would bring down the entire network - they didn't self recover either! you had to reboot them all. I remember once when I misconfigured a token ring card on a PC and it brought down the mainframe! It shut down the bank for 30 minutes"
I used to work in a place where every office on the floor was connected to the same ethernet hub. We also tested a server product that became a DHCP server if you checked the wrong box, so periodically everything would stop working until the team lead came out of his den and went to yell at the new guy.
What do you mean hubs? Back in the day we manually pierced thick Ethernet coax to get access; of course you could ruin the whole segment running tens or hundreds of meters if you didn't take care.
Hard to be anything else if the entire job is spinning up a yet another microservice and then spending most of your time debugging in production because shit keeps breaking all the time.
Maybe we're thinking of two different archetypes, but I wonder if their "hard not to love" nature is part of the survivorship bias of why they're still there.
They are worth their weight in gold, truly. I had the pleasure of interacting with one such man at my previous company weekly for an extended period of time while planning/designing a rewrite. Listening, thinking, and questioning his stories and perspectives allowed me to formulate problems and architecture in a more first-principles, fundamental way and I believe that experience really helped me mature as a young engineer (as well as the project).
This is where another one of your Pied Piper boxes would go. Okay. Let me show you the next location where we would install one of your Pied Piper boxes. ... okay
Not necessarily hate...there's always the 1-2% of new hires that are highly impressive and gives us hope for the future. They tend not to last too long. The rest seems hell bent on never even cracking open documentation, and keep asking "where are the architecture videos?" (at least where I work).
This is actually something I've identified as a major con of working at startups (there are a lot of pros too, and I'm at a startup right now that I love): It is impossible to have these people, because the company is just too young. And also I think this kind of person doesn't usually join a startup for its first, like, decade.
"Hard not to X" is not the same as "Easy to X". It may be easy to do something but also easy not to do it. It would be easy for me to yell "Elephant!" right now, for example, but also easy not to do it.
I often search for a clearcut answer to a technical question and I'm met with a 2 hour history lesson into a decade of company politics and failed replatforming projects.
Yeah, thanks for telling me why John from accounting was a dick 10 years ago and you had to code this module in a certain way. I really don't care. I'm new to the codebase and I just want to know how it (the codebase) works.
I'm currently in this situation and a colleage never gives straight answers to anything. It's always some little rant about something and when it's done, I still haven't got my answer.
> I'm new to the codebase and I just want to know how it (the codebase) works.
You should want to know _why_ it works that way too if you want to do any meaningful work with it. Context matters. I’ve seen many cases where the way something works seems dumb, only to learn later they had already tried the “smart” way but ran into some obscure problem which the “dumb” way solves.
This is important. As soon as we come to the understanding that the coders that came before were not all idiots then we are forced to ask the "why". There is generally a decent reason why something is coded the way it is. Could be as simple as it was an emergency and meant to go back and fix it but never had the time or it could be a valid business edge case that absolutely had to be wedged in. There is almost always a why. Otherwise you tell everyone about a fix you made that 3x performance and you see their faces go gray as they explain you just shut down some archaic but mandatory process in Singapore.
It probably depends on the person, but I personally enjoy such tales. Gives me more context about why a piece of code was written the way it is now. But then again, I also enjoy scrolling through the commit history of a repo like it's an archaeological dig site, so I might be the weird one here ahaha.
It's full of "fix" or "fix typo" or "update blah.c"... tiny commits straight to the master branch without CI.
Anyway, I really don't care about the past in this way. A simple "accounting needs this for that" is enough for me. No need to explain that Adam was getting divorced at the time, so he was grumpy, and, and, and...
Easier said than done, but try telling them that. "I appreciate you sharing context, but it's too much to take in at once. It would work better for me to get a more clear-cut answer."
Sometimes, the 2 hour story isn't worth the time it's told. Sometimes, the 2 hour story gives you the insight necessary to satisfy Chesterton's Fence. It seems like in this instance, that's not the case, but I'd definitely encourage you not to dismiss stories in general because you don't think you need the history to achieve your goal.
Because I don't want to hear about personal stories? How is that related? I do very much want to know how code works. It's the very core of my argument.
And this propagates the problem. Getting an answer without understanding. Sometimes the history is necessary in order to not repeat botched attempts at "fixing" something that people have already tried.
Man spends entire career convincing himself the reason he keeps job hopping is because everyone else is writing crappy code.
Karl Hackerman shows up to each meeting complaining. He seems sour that he hasn’t been given the go ahead to rewrite the system in Rust. He knows a lot about bleeding edge frameworks and best practices and always seems annoyed that he can’t use them at work. He thinks it’s because everyone else has settled in to becoming wage slaves instead of striving to be the best programmers they can be. Everyone puts up with him because they know he’ll quit in six months anyway.
Karl is 47 now. Hasn’t held a job for more than a couple of years. Has mostly been freelancing lately and trying to sell books on tech and frameworks nobody cares about anymore.
I love this so much. I've totally been this guy :) Learning how not to be an insufferable doucheb*g was a surprisingly hard challenge for me
I've been writing a longer fiction piece about overzealous know-it-all engineers in the style of Confederacy of Dunces. Probably won't be done for a while, 'cause there's a lot more of my soul in it haha
Yeah this is ruthless. On the other hand, I feel ok with my present course because the intermittent chance to use new shit is the only thing that keeps me interested anymore. That and the exorbitant pay, I guess.
I felt this way earlier on, but quickly learned to get fulfillment in my day job purely from shipping and contemplate the mysteries of computer science largely on my own time. Sometimes they overlap, but good day jobs are often 80% boring in my experience.
I lost it at "best practice". I once worked with this guy. He would use that term whenever he didn't have any good argument, which was basically all of the time. He would have the strongest opinions, whenever it didn't matter...
For ten years, I worked on codebases written in C++ without using any third party libraries (no, not even STL -- we needed faster, thread safe versions of strings and maps which were not available at the time; Boost? What's that?), its own custom messaging middleware built directly on sockets, its on distributed process management system built with unix syscalls (obviously). There was not a single line of HTML, Java etc. anywhere. It was a comfortable, high-paid life and I solved cutting edge problems (HFT, distributed, concurrent computing). In 2013, I forced myself to quit, start with a small company that paid me half the salary so that I could learn the web tech stack from the ground up. It was all to avoid this fate. I'd like to think I've succeeded.
I don’t think anyone working with low-level C++ for HFT can face similar fate. Most of skills gained at such job seem very transferable. The article seems to be more about jobs where 95% of work consists in implementing CRUD in accordance with very specific business logic of the company.
The only reason to change such job I can imagine is to sit out your non-compete package. But I am not in this industry so maybe I’m missing something.
It's not that you won't have the skills, it's somehow demonstrating them.
I also worked at an HFT C++ shop that used no STL. Every data structure has a weird name and API that you get used to. The whole architecture has its own idioms, and hasn't needed much upgrading in terms of language features.
Imagine you learn the grammar of English. You learn how to pluralise nouns, when to put an s at the end of a verb, how to to use commas, and so on. You then go to work at a place where they have a different word for coffee, drink, and order. Along with every other word. You end up learning where everything is and how to talk about it, and you're productive.
How do you apply for another job? Chances are you will be asked how to order a coffee and drink it.
This kind of issue cuts both ways - how do you hire new people and get them productive quickly on a code base where they will almost certainly understand nothing?
Getting rid of weird data structures that now have STL analogs then is a productive exercise - existing devs get to learn more STL and new devs have more familiar ground. You get to delete code too.
Yeah, I find it amazing someone would transfer to standard web CRUD development from that. Sure you can have enough of all the custom stuff, but there are plenty of companies that will hire you for your current salary for non-web work. (you can probably guess how little I like web work)
Same! The poster's pivot reads like horror to me. Like they lost their mind and joined a cult where they worship Memnon with sermons of JavaScript.
> It never sat well with me. Colocated systems saw latencies no mom-and-pop investor could ever dream to achieve. That is one of the reasons why the second company I joined was a non-profit that built systems to help track human rights violations, enforce labor rights, monitor democratic elections etc. Third job was in the food industry and the last one, in housing / property management. So I'd like to think I've made sufficient amends.
Ah, nevermind. Turns out they were worshipping Memnon with C++ and renounced that cult with web tech! :p
There is a massive advantage in being able to see the web tech stack for what it is and wielding it without over-engineering or fanaticism, while your competitors are spending 2-3x on monstrosities that essentially does the same thing, but slower.
I was originally concerned about your parent comment regarding your new foray into web dev (that C++ path sounds amazing), but this comment has put my soul at ease.
If you walk into web dev with aggressive principles and a willingness to be controversial, you can get a lot of shit done and it will feel great. Watching others struggle under the various JS framework circuses has almost an opulent/aristocratic vibe in 2023.
I have a pretty similar story. I spent 7 years working as a subcontractor for a big telco with CORBA, pthreads and the ACE reactor library. First week, I halted a whole contact center service by adding a sleep call to the codebase (it was on UAT environment). Fast forward several years and I could effortlessly debug cores with thousands of threads, reason about semaphores, reentrancy and even tell the difference between the POA and the BOA Orbix adaptors. I began to become the mythical guy who knew what everything did not because I was technically good, it was because I understood the business and how it mapped to the different architecture elements.
Then I felt like I could spend my whole career there and decided there was a whole world out there in the web, began to learn PHP in 2004 until the mythical "blog in 5 minutes" Rails demo appeared. I switched jobs in 2007 and I have never looked back.
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
Interesting. I've used semaphores with pthreads (only once -- after that I realised plain old mutexes will do). Like your sleep call, I once brought an entire service to a grinding halt by adding a simple `fsync` call (to fix a recovery failure because once a sequenced message we thought we had written to disk before a crash had actually got lost in the buffer). Understanding things at that level makes any new tech fairly easy to grok.
C++ is such a damn chore. This is partly why, everyone always assumes they can do it better, they don't trust any other code, so you end up with monoliths of NIMBY code and esoteric build setups. Code reviews are incredibly painful for the same sort of C++ reasons, everything can effect everything, the whole idea of encapsulation tends to fall apart when memory, time, ordering behavioral effects get relied on after some amount of time usage. Every change is a footgun to be found in production under just the right circumstances.
Hard to be productive when you can't trust the language, the tools, or other developers to be correct.
I did the same, I thought C++ was heading for obsolescence. Now a master of Java, Javascript and can write great full stack apps. Meanwhile HFT C++ jobs are paying double, I'm thinking of going back, even if the systems are boring.
been working on java jobs and easy code bases since I graduated 10 years ago, nothing had been really challenging and I'm not complaining. Would be interested in a job that pays more even do you think working on C++ jobs is more challenging or just boring?
Depends on the job of course. Probably more technical problems and tuning. Often code is low level which is interesting if its your thing but not if you like business problems.
Those stacks can be full of 'not invented here'. They are unique and have their own quirks. Quirks that you more than likely can not google for any sort of guidance as to what is going on. I wrote one of those stacks a few years ago. No std libs at all. Why? must fit in 32k and most of that space is needed to hold data. That means throwing a lot of those helper bits out or spend weeks figuring out how to bend the compiler to your will and not include everything. When it is easier just to roll your on CS version of whatever. But it sucks because you probably have odd edge cases.
Yeh it's a rough but rewarding life; I spend most of my hours on problems that aren't on Google, or any LLM, so I'm kinda used to it now. (Stockholm)
Would love to know of more spaces which require this level of programming - usually I feel kinda underworked at most shops.
Though, true enlightenment is combining the urge to NIH with some, light, sensible borrowing from those smarter than us :) very much a key skill unto itself.
Embedded programming is this 99% of the time. Abstracting hardware away is a silly notion. Docker? Cloud? Hard to do when your code only runs
at all on one specific custom board. The protocol you wrote to talk to that one chip isn't working? Time to break out the oscilloscope and look at the actual data going to the pins.
Join us, the pay is worse and the work is harder, but also the prestige is lower.
I sometimes really miss embedded programming; it felt like real engineering in a way that higher-level software development often doesn't. Alas, the shop I worked for could not keep me busy enough, the projects began to feel repetitive, and I got really bored; it'd be hard to go back.
The space I was in was IoT type items. Usually hub controllers. The thing is that is getting less and less as you can get a fairly robust thing now with a couple hundred meg of flash and ram for a very reasonable price. So those 'must fit in 32k' items are few and far between these days. With that sort of space you may as well use the proper libs that are usually nicely documented and all the quirks have been seen by hundreds of other poor souls such as yourself.
I know it's not all about the money, but over the course of your entire career, do you think that move was financially positive or negative?
I'm very roughly estimating that you could probably have worked in the HFT job for around two-thirds the time you're going to have spent working on generic web tech, and still been better off.
Absolutely 100% positive. I was largely an individual contributor all that time, and only did some unstructured, untrained, unguided people & project management. Perhaps because it was such a niche, engineer-dominated environment. Several years after leaving that company, I quickly learned to effectively manage people and projects, architect systems using more mainstream/open source tech and eventually reached exec level (but that's a different story).
Ah, that's the missing punchline: you had ambitions to be something else other than an engineer, and you (quite correctly) recognised that you were never going to grow your wider skills at such a place.
For someone who had wanted to stay an IC, I don't think the move would have been quite as wise. Being in a niche, engineer-dominated environment, and being paid extremely well for it, is nirvana for many lifelong ICs. It certainly feels very different from the experience of Arthur from the linked article.
That's really some fields I'd love to get into -- less need to talk to people and more need to talk to the machine. What a dream! Alas I only scratched the surface of C++.
I hope it worked for you, but I don't see any big difference. Web technology will become dated faster than the old C++ codebase. If you learn some web tech stack today I expect that it will be obsolete in 5 to 10 years. What are you gonna do next? Move to another small company for half the salary to learn everything again and compete with 20 year olds?
What about it would you like to avoid? I don't understand the fears that people have in this thread. To me this sounds like the Platonic ideal of a programming job.
EDIT: I'll reply to my replier here because /u/Dang rate limited me (again), though I must commend him for letting me have one more comment than usual before he pushes the button ;)
> I would say the ending -- the part where you are unemployed and unable to support yourself in your old age because you invested decades of your brainpower into a highly specialised, non-transferrable skill.
I would contend that such a person, unless very reckless with finances or extremely underpaid, should have enough money saved up to make what is mentioned there very unlikely. That said, I know some high-income earners struggle to not blow their entire bank account each month
I would say the ending -- the part where you are unemployed and unable to support yourself in your old age because you invested decades of your brainpower into a highly specialised, non-transferrable skill.
Most of my early programming work was with C and C++ (after I left academia, where it was mostly Fortran). I am, simply put, a better programmer for it, objectively so when compared to my peers who've never touched it, at basically every place I've worked at since (including a FAANG-level company, currently, where I'm mostly writing in go and python). At least, that's the feedback I get from them.
I think it's fair to say this skill difference is largely from that work in the C family, where I learned a number of different paradigms for design and development in that world, with all its footguns and low-level "gotchas". It has always been easier for me to parse and understand others' code, identify subtle bugs, to use debugging tools, and to identify some optimizations by examining the code that others might spend days using profilers to find.
Some of that would come with engineering experience in general, regardless of languages used, but not all of it, but I picked these skills up faster and earlier in my career, thanks to that early work. Skill with C++ is definitely transferrable, in at least some cases.
I should think that years of specialized C++ work in hft environments would have come with impressively high pay… in which case one could retire early and self study anything for fun, not hope to find a job for financial reasons.
Ideally, yes! Unfortunately, there is such a thing as being young and not knowing your own worth / not knowing how to negotiate / not having a manager who looks out for your career growth.
In that case, if web development is your interest, then you have many possible paths.
The primary choices are full stack, backend, or frontend.
Obviously full stack is most flexible because you can theoretically do it all. But keeping up with so many technologies simultaneously is really tiring.
Take a look at Phoenix framework. It is a fantastic way to get started, and it will grow with you into just about any future project need. Plus Elixir is an excellent language built on top of the incredibly powerful Erlang ecosystem.
It is a transferable skill. But not one many people want to pay for except in very niche areas these days. So if you are out of a job expect to grind for awhile trying to find someone to hire for C++. Unless you have network connections then you might be ok. Being good at C++ only goes so far, you have to have skills in techs people are willing to pay for. Not many companies are willing to wait while you skill up in their stack unless you are willing to work jr but they will not hire you anyway because 'you will jump ship'. Its not great.
That's mostly the guys fault and not the jobs fault.
Even in a bad code base you can start making things better, apply modern techniques and learn new things.
Also most of the skill you gain there is transferrable. E.g. just because you have to work with some legacy PHP project, it doesn't mean that you would suck at creating modern applications in JS. It would just take a couple of weeks to get use to it.
> Even in a bad code base you can start making things better, apply modern techniques and learn new things
All th bad codebase I've seen were a result of fossils who are still with the company gatekeeping anyone from fixing their code.
A few years ago, I had one architect stubbornly refuse to use source control, rejected all PRs that didn't use his homegrown buggy C++ libs for strings, refused to let anyone do the http server bits unless they used his C++ server and so on.
If there isn't a gatekeeper, you wouldn't have the bad codebase in the first place.
I've got to disagree. not every contributor is making the code base better. I'm in a situation where we are such a small team that there really is no gatekeeper, and we only require one approve per pr...and I've come back from vacation to some monstrosities that I have to look at with no time to fix.
After a pairing session with a a fellow developer, where I took the extra effort to refactor something along the way, probably an addition 15 minutes of effort...she quipped- no wonder it takes you so long to finish your stories sometimes.
I mean, that's not why- I'm ADHD, so my lack of productivity sometimes is invariably due to a day where I forgot my meds or didn't get a full nights rest.
Making things better and applying modern techniques don’t map to a feature story. In the environments being discussed you’ll have to wait for a cadre of architects and program managers to convince the C levels to commit to a multi year, millions of dollar effort to have any hope of improving a code base.
> "just because you have to work with some legacy PHP project, it doesn't mean that you would suck at creating modern applications in JS"
I disagree; working on a legacy PHP project isn't just opportunity cost (failing to sharpen your knife), it's actively harmful (leaving the knife in salt water, accelerating the rust).
that part is a complete fiction. People who have a lot of experience in the embedded world or performant but esoteric codebases are highly thought after. Even if the literal job isn't transferable, the skills for sure are.
My mother was a COBOL developer and didn't work for 20 years because she stayed home. She recently returned to her old bank she worked for because they are dragging people out of retirement as they can't find anyone else. She earns more than I do now.
People are crazy to give a job like that up for generic web dev work.
The web stack might give you more transferable skills but how did you calculate the trade off vs the 50% pay cut? Was the salary not actually very high and at a low ceiling compared to web work - i.e. you expect your salary to soon exceed what you made working in HFT? Or was there a serious risk of layoffs happening far in advance of your retirement? If the goal is to support yourself after retirement, are you saying in certain circumstances halving your salary 10 years into a tech career ultimately optimises your entire-career-earnings?
It wasn't such a purely material calculation. Less stress, more time to spend with family and friends, learning more things and doing something that you know actually adds value to the world -- those things matter.
High frequency algorithmic trading. Often ran on colocated hardware within the exchange / trading system data centers. Started with stuff like Sun SPARC, Solaris and Oracle and we slowly made our way to Suse, Intel and... surprise, our own database optimised for fast writes.
The whole business model is misunderstood by your average internet commentator.
When you have a market, you need someone to be there to provide liquidity. Imagine if you're a farmer and you show up to the market with your wheat, but all the bakers have gone home that day. Or the baker shows up and there's no farmer. The market maker stands around all day offering to buy and sell so that you don't have to wait for the guy you're really trading with. Of course this middle man wants to get paid for it, but your cost as an average Joe is next to nothing. This is trading in time.
Now imagine you want to cook a meal and your ideal meat is beef, but actually you're ok with pork, so long as the pork is cheap enough to make it worth it. How much cheaper should it be? Well your fellow who knows all the pork and beef people will be able to gauge where the balancing spread is, given the amount of interest. In fact he will from time to time do the trade when the spread is out of line. This is trading in space.
So why all the fancy tech? After all market makers used to stand around in a pit in a colored jacket, and they didn't have degrees. My first boss in the market was one of these guys.
Well, things have gotten very tech heavy because as soon as prices are out of line, there is money to be made. Or rather, lost. As a market maker, you are constantly out there with your prices, offering to buy or sell at a very small spread. If some news happens that affects prices in a big way, you can be sure that you will buy when it's going down and sell when it's going up. In order to both have tight prices and avoid this "adverse selection", you really want to be able to react as quickly as you can when your system decides that something's up.
Markets were possible. But what about the current offering of ETFs and similar products? Despite all the fancy talk, many HFT firms are just market makers for the products that enable average joes to invest well and cheaply.
You don't have to be that old to remember how terrible the experience used to be, and how easily even sophisticated individuals were ripped off at every point of the process.
I explained that. It doesn't matter whether it's a computer doing it or a guy in a jacket, but once one guy gets a computer all the market makers need it.
Retail consumers aren't exactly disadvantaged by the gradual narrowing of spreads, so what are you trying to fix? The latency arms-race is exclusively professionals fighting each other.
Getting back on topic, being a software developer for an HFT shop can be a lot of fun: serious technical challenges and a rapid feedback loop in a role where you may be the revenue stream rather than just another cost center. But posters highlighting NIH syndrome are spot on: you risk pigeonholing yourself.
You could easily solve all of this without HFTs - you might introduce a few seconds of latency but virtually no real investors care about that level of latency and it would save them billions per year.
It absolutely would if you remove the cut HFTs take out of the market. I'm sure you'll make some argument about liquidity and spreads but, in the aggregate, the money from higher spreads goes to market participants so the market as a whole still saves by eliminating HFTs, even if the spreads go up a bit.
This is like saying if we eliminated millimeters from rulers, and only used centimeters, everything would get closer together.
HFTs do not take a cut out of the market, their cut comes 100% from other market makers. Those are the only folks who would benefit from eliminating HFTs, and unless you’re one of them, who cares?
I don’t care who buys when I am liquidating a position. That’s the whole point of a market. Therefore I don’t care about HFTs.
Unfortunately, yes. It never sat well with me. Colocated systems saw latencies no mom-and-pop investor could ever dream to achieve. That is one of the reasons why the second company I joined was a non-profit that built systems to help track human rights violations, enforce labor rights, monitor democratic elections etc. Third job was in the food industry and the last one, in housing / property management. So I'd like to think I've made sufficient amends.
Generally, it's not about front-running normal people. Instead its racing other HFTs. Either a race to capture arbitrage opportunities, or to update your orders on one market based on trades made in another market.
The aim of the game is to collect a small spread on a lot of transactions.
The scary thing for an HFT like that is being 'run over' by a big party selling or buying a lot over a day.
If a pension fund is dumping 10٪ of their holdings of shell, it's hard to collect a spread because you need to find buyers for every stock you buy from them. Meanwhile the price is dropping as they sell off.
Hence you occasionally see 'front running' with price improvement. Instead of buying from the market at 1.02 a HFT gets to sell to you at 1.01, yielding you a better price. This is a good deal for the HFT, because a normal person isn't going to run the HFT over. And it's a good deal for the normal person because they get a better price.
No, it's a zero sum game. HFT firms fight to be the first to pick up a penny, but they can't force anyone to drop more pennies. They don't cause prices to go up more than they would have otherwise, or extract more value from the poor and middle class.
And now you are jumping from one "stack" to another, none the wiser, your knowledge hopelessly outdated just a year in. The principles you learned in HFT? Those apply all the same.
So you left a job that paid 2x and exposed you to hard problems that require reliable solutions for a job inside the horror clusterfuck that's the modern web where nothing really "works" and there's no real engineering to speak of besides monkeys throwing crap in a shared pile and shovelling it out the door ASAP.
Most people I know, inc. myself, would call this a "bruh moment".
I can't fathom willingly entering the web mines if you've already sunk in the hours to master high perf native code, OSdev, dist systems etc. People usually enter web to avoid having to learn such things.
To discard that knowledge and instead compete with an endless stream of fresh grads, boot camp grinders and LLMs for less money? A madness if I've seen one.
Web tech is more immediately rewarding. With C++ (and especially with modern web dev agile processes), you can spend a week building enough framework and infrastructure crap to stand up the barest inkling of a demo of the solution you plan to implement, only to have the whole thing pivot out from under you and become wasted work.
More complicated languages and frameworks that manage memory and do UI and everything for you let you get to a prototype much faster, so you can see whether it'll work or whether you want to toss out the idea much sooner.
Even if the decision he made was a bad one, you don't know anything about why it made sense for him at the time.
People aren't algorithms running in complete isolation and silence. They live in nuanced circumstances you can't possibly understand by his short history in a comment.
I hope you can find understanding and peace in the future.
It's hyperbolic but it's a common sentiment that I tend to agree with. The web dev world changes too fast and too often, many stacks involve absurd complexity for dubious performance and maintainability and other metrics.
I've been doing web dev for 20+ years. Which part is wrong?
It's not hard to find feature support or implementation differences between browsers. But these days, that's not really the problem. I can go read the ES language specification, and generally find user agents/node/deno/bun have compliant implementations.
The problem is brittle "ecosystem" stuff. How are the create-react-apps going? How is module bundling going amidst the UMD/CJS/ESM war?
Don't live to work, work to live. I bet Arthur partied his ass off back in the day, stumbled into work without sleep, napped during retros and refinement. He might be the least useful dev but he's a king at Burning Man. Now with his flush 401k, he can finally retire to make little wooden figurines to sell at the farmers market, where he really just wants to regale people with stories of the biggest fish he ever caught, or his dream of opening up a ramen food cart if his in-laws will go in on it with him. Arthur's lived a life - just not at work.
Hard to do if most of your life is work. I find it very hard to find space for an actual life after subtracting a full-time job, exercise, personal hygiene, nutrition, etc. from my waking hours. It's essentially ~2h a day + the weekend, which often also consists of getting stuff done you didn't manage during the week.
I hate the thought of my life starting in retirement. Especially if I might be 67 by then, which is the legal retirement age, where I live.
> Especially if I might be 67 by then, which is the legal retirement age, where I live.
When you say "legal retirement age", what do you mean? Is this to receive some sort of retirement benefit like Social Security in the US? Or is it something else?
The retirement system is completely broken over here. As an employee you have to pay into it. I pay > 500€/mo, which a big part of my pay check and one of my largest expenses. My employer has to pay the same amount on top. You might or might not consider this essentially an additional cost for me as well. It is intended to be the primary pension plan for most people, though by the time I will go into retirement, the whole system will be unviable financially and I might get nothing. If I do get something, I get it starting after completing my 67th year of life. Of course I'd rather invest that money privately. If I choose to go into retirement early, I will have to pay hefty life-long penalties - 0.3% per month that I retire early. Obviously a couple of percent over 20 years of retirement or so can hurt quite a bit.
It might sound silly, but it's one of the main reasons I want to leave this country.
> by the time I will go into retirement, the whole system will be unviable financially and I might get nothing.
What do you mean when you say "financially unviable"? There have been similar concerns raised about the US retirement system, but AFAIK that only means less money coming in than promised going out. The income is predicted to stabilize at ~75% of expenses after the "Baby Boom" generation finishes retiring. At the worst, I'd expect a pension cut rather than a complete collapse to 0.
I don't know how the US retirement system works, but I think Germany is pretty unique in that the current working population is directly paying for the pensions of the retired population. There is no investment of those funds of any kind or something like that. The state is already taking up huge loans to finance this, as we don't have enough payers. We have about two payers to one pensioner at the moment, but when I will be 67 with the current population development it will be one payer to two pensioners. Even with more massive debt one payer will not be able to pay the pensions of two other people. It might just be a huge cut, but if it's not enough to live off of, large parts of the older generation might end up on the street. A large part of the younger generation is aware of this, but 23% of Germans are over 65 and the median age is 45 so older people generally have a lot of voting power and the issue is largely ignored. Very recently there were proposals to switch to a system in which the pension payments are invested into index funds, but it was decried as "gambling away our pensions" and the proposal was shot down.
The US retirement system (Social Security) is paid for the same way. The population pyramid for Germany does show a much larger hump of upcoming retirements compared to the US though. The ~30yo population cohort has grown in recent years though (through immigration), so maybe it won't be as bad as you expect.
Why do you think Index Funds would solve the upcoming issue?
mate you think retirement age is 67, thats very cute of you. just wait for 20 years and see how many times this milestone will change with the way the world population is going :)
I think this was a contrived example, but I really appreciated all the little details here! They struck close to home and felt very validating, even if accidental :)
Yes that would be great and I do hope for mr A the reality is some variant of this.
More often than not though, its not so rosy. Arthur would be overweight, riddled with medical issues, solitary guy with unhealthy attachment to XYZ (anime, gaming etc).
But if he had kid(s) and raised them well, well as a parent too I do have tons of respect for him for that alone, rest are details when looking back.
There are talented people everywhere, the probability of somehow being ground breaking and world impactful is ridiculously small. Anyone pegging their self-worth on that goal is setting themselves up for a lot of mental health struggles.
My levels of ambition and relentlessness absolutely shifted after having kids, I used to be absolutely furious about being the best. Now I am very much "fuck you pay me" and my priority is having as much free time as possible for family and friends. I for one would be fine with Mr. Westbrook's career although thankfully I haven't had as crappy colleagues as him throughout my career.
There are people of all types who will want different things, no point in judging. As the great thinker Alicia Keys once said: 'you do you boo!'
> I used to be absolutely furious about being the best.
I found out that just being better than yourself takes you a long long way.
I have never competed, but just did my thing. Then found out that I've already know more and ahead of many people I'm inspired by.
Yet, this was not my aim, and still is not. I'm just one of the so-called dark developers. I just do my thing, try to do it better every time, and don't care about hall of fame of anything casual or serious.
I recently put a little modal on my phone that just says "Compete against yourself" every 30 minutes, and I'm finding it remarkably helpful at focusing my efforts back to what I want to see in the world.
I like the mindset of trying to be just a little better than you were, every day. It feels... correctly scaled, somehow.
Everyone has their quirks. I've got my computer speaking the time every 15 minutes and everyone I know would be driven absolutely insane by that. It helps me keep track of time.
Some people put a postit on their mirror or laptop screen.
This is a great comic. Understanding and appreciating other people's differences goes a long way. So much harm comes from being fearful or judgemental of people who are different.
Now I'm curious - what else is in your "life changing comics" folder?
Hey, C'mon: https://www.viruscomix.com/page528.html
The Human Race: https://viruscomix.com/page531.html
So Mature: https://viruscomix.com/page532.html
Sexier Th>n: https://viruscomix.com/page542.html
Copper: https://www.boltcityproductions.com/copper
There are other another honorable mentions, but I was not able to find the original URLs, so I didn't add them here.
I'm intrigued by human nature and why are we the way we are, and search for a calmer, more peaceful self inside me. Apparently meditation helps a lot.
Mine is currently on a 15 minute loop as well, but it just plays prerecorded numbers from 15 to 0 telling me how many minutes left in my 15 minute time block
The time notification reminds me of a screenless "watch" 10 years ago. It would buzz every 5 minutes as a supposed aid in perceiving the passage of time for various tasks.
I have a GPS disciplined rubidium clock with a 100MHz output hooked up directly to my sinuses. The slow sensation of waves washing ashore keeps me awake long enough to interact with the rest of you when I am taking my daily luxury 3ms break to read HN. Further, I have a script that, when my break time is over, automatically terminates any conn-
Great for the upside of growth - how do you change it for when you start to age and skills erode a little bit. That would be deeply frustrating. Need to change the model at that point! What to though?
The AI helps a bit, I'm feeling a bit recharged with it, like I have a few more extra years of capable contributing in me. Easier to get stuff done, not getting worn out with technical minutia as much.
Move more into to people skills and relationships. It could be management, marketing, sales, HR, recruiting, mentoring, training. There's a plethora of side functions. Or take a lower paying less demanding role. Do harder technical work that isn't time constrained like open source or personal projects.
What does it mean in this context to compete against yourself? Anything at immediately at hand or on the todo list? Or is it just a reminder to get something done you thought of doing but normally wouldn't otherwise?
If you find yourself becoming desensitized to the notifications on your phone, I suggest training a parrot to stand on your shoulder to liven it up. Great for parties too.
> I found out that just being better than yourself takes you a long long way.
That resonates with me.
There is a saying where I'm from that says "most people are just kicking dirt" that basically means people are doing the bare minimum or don't care at all, and if you just do your thing and really care about your stuff, you're already ahead of the majority.
Being anxious all the time, I use that saying in the opposite way though: just relax and do your thing... you don't need to worry about not having a job, all those other people are barely doing anything and you can't be the best at everytime. Just do your thing are relax.
> I found out that just being better than yourself takes you a long long way.
I usually hold vast disdain for the "self-help" genre of books, but after a recommendation from someone whose opinion I value greatly, I read Atomic Habits by James Clear. I've never found myself agreeing with and finding revelations from such seemingly mundane and obvious statements. Sometimes seeing a statement or opinion you take for granted can make you cognizant of that point.
I've since tried to adopt such "agile" methodologies in personal improvement. Clear says 1% improvement each day always sounds attainable, and such iterative improvements add up.
Yeah, that's true. I have a blog, and share the code & docs I write for myself, but my blog is not about programming, and the docs are more like cheat sheets for general public.
Well, the code is just GPLv3 licensed small tools which I develop according to my own needs. Nothing impressive.
I don't use Twitter, Facebook and Reddit anymore. I just converse here, and with a Discord server I really like.
I'm in a constant state of pendulum swing with this. (Not with the goal of being "ground breaking" or "world impactful", but just doing "great" work within the range of my own self-perceived capabilities.)
When I first had children, my pendulum swung way over to disinterest in seeking anything in my work beyond financial support for my family and time to spend with them (while still supporting and not breaking trust with my professional colleagues).
But my kids are growing and becoming more independent all the time, and I'm currently in a phase where my mental cycles and energy are on an upward trajectory over time, and now I'm back to being excited to use those cycles to do impactful work. But critically, I think, with significantly more wisdom about how to strike a healthy and happy balance for me and my family. I mean, I'm not naive, I don't think it will be easy to strike a good balance, I think it require be constant work and effort at it. But as I've emerged from the baby-brained haze, I have renewed clarity that neither focusing my energy entirely on family or entirely on work is the right path for me.
> the probability of somehow being ground breaking and world impactful is ridiculously small.
See I have the opposite take. There are so many damn facets of modern life/society, and new avenues/facets/domains coming out every year, that it is actually pretty easy to find some niche where something hasn't been done yet that is world-shattering for some specific community or niche.
World-shattering for everyone is pretty overrated.
Agree with you. I don't understand the point of this article other than some sort of tongue and cheek snobbery. Hope it maybe made the OP feel better about themselves.
I will not call it snobbery but a difference in generational culture. I am assuming you are below the age of 45 and came into the industry where jobs were aplenty , super low interest rates and the industry was rapidly innovating and risk taking is encouraged . For a lot of folks from the Office Space days of 80s and 90s that wasn’t necessarily the case . Tech jobs were limited and you probably had to post or submit a paper version of your resume.
I would encourage you to take a look at JavaScript code from 2003 or a Java codebase from then and see how dreary and different the tech industry was back then . As for transferable skills there is a reason IT workers were stereotyped as autistic nerds before the brash,well rounded, rock climbing risk takers came in. Culture, risk, opportunities and interest rates had to do a lot .
Edit- I was probably like Arthur Westbrook when I was working on a visa and had a family to support . As soon as I got my greencard my appetite for risk taking increased and showing my transferable skills increased . Situational context .
This level of reaction tells me more about you than the author. Referring to it as 'snobbery' shows that you interpret satire as a personal attack on someone, probably yourself. I clicked to their home page and immediately figured out that they dabble in tech and like to joke around.
This is as much a tip for myself as for you: take a deep breath before reacting.
> There are talented people everywhere, the probability of somehow being ground breaking and world impactful is ridiculously small.
No there aren’t. For every Jonathan Blow, Salvatore Sanfilippo (redis) or Mike Pall (LuaJIT) there are probably thousands of run of the mill developers working at feature factories. You don’t build amazing software by being lucky. Your boss at GenericCo will never go out of their way to ask you to build that thing you’ve always been dreaming of making.
You choose to work on software you find interesting. Or, for any of 1000 totally valid reasons, you choose to be small.
If you think you’re all that, don’t blame your company for “not getting the opportunity”. Please. You already have all the tools you need to code. Make something cool!
You underestimate the odds of talent in a population of 8 billion humans.
> If you think you’re all that, don’t blame your company for “not getting the opportunity”. Please. You already have all the tools you need to code. Make something cool!
There are neurosurgery robots operating with my code running in the background for several years now. I am now working on ophthalmic surgery software for sub-retinal operations. I would think that is actually kind of pretty "cool!".
However, I still know full well that I am a cog in a big machine. I still know full well that my impact is not world shattering. And I still very much think "fuck you pay me!".
My self worth is pinned by family and friends. Work is fun but it's a side quest. If you think differently that is ok, and so am I.
Stephen Jay Gould in "The Panda's Thumb": "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops"
> There are neurosurgery robots operating with my code running in the background for several years now. I am now working on ophthalmic surgery software for sub-retinal operations. I would think that is actually kind of pretty "cool!".
That’s super cool dude! Surgery robots are pretty world shattering for the people whose lives are saved by them. Contributing to that sounds like something well worth feeling proud of too.
I just finished 1 month at clown school doing an intensive performance class. In clowning we talk a lot about coming on stage with a dream. A dream to do something great. To impress the audience. To move mountains with your words and presence. If you come on stage with no dream, the show is dead.
I think work is the same. If you go into work each day with a dream, it has life. You are there for a reason. You love your work. You want to change the world. Whatever. But if you’re going in to work each day with no dream of your own, you are a walking corpse.
Redis will bit rot. LuaJIT will bit rot. Jonathan Blow did make some fantastic games, but even then it's in a sea of games that are pretty great. Everyone is pretty small in the long run.
- Talent - obviously have to be really good to even do anything in the first place
- Vision - Need to have a goal of some sort to accomplish something
- Decision making - The goal needs to actually be something the world finds useful.
- Drive - Be able to take constant set backs and negative feedback and believe in yourself to an almost delusional level.
- Consistency - This means doing and thinking about your "thing" for a REALLY REALLY long time. Like weekends and holidays and ignore family and etc...
The talk above goes into it pretty in depth, but theres no mystery about it. You need ALL those things and no less. People who achieve at this level aren't accidents. They are simply not only talented in their fields, but also in areas of being able to consume volumes of information and handle failures. They're the ones getting up at 6AM every day working and thinking about their "problem" and going to bed dreaming about it, for years.
You can be a 10x developer, but you also need to be 10x in several other areas to actually accomplish valuable things. Theres a grand canyon of difference between them and everyone else.
Been in software dev for 43 years. Done the 6am and 96 hour weeks. Then realised I'd missed a big part of my kids growing up, and once they were dead, spending little time with my parents. Too busy you see.
I'd recommend people really think about the costs.
The truth is almost certainly between those extremes. People who never put themselves in a place where they can do something amazing are obviously much less likely to do so.
Equally, there are many, many talented people out there who never achieve recognition for what they do or were capable of.
You may not want to hear this (I certainly didn't when I felt similarly)...but perspective does change with age/personal life...with all due respect do you have kids? A partner? Any major commitments outside work? That's an enormous part of OP's statement.
People don't stop being talented/smart when they have kids/families/personal lives...but they will have less time and their perspectives do change.
It's a balance/spectrum where most of us end up somewhere in between two cliches: do what you love and you never work a day in your life on one end & work to live not live to work on the other.
Engineering is one of a few special disciplines where peoples' hobbies align easily with ability to earn money at most levels of professional performance; being mediocre is still a recipe for a generally acceptable standard of life. Compare it to things like sports or acting or music. It's possible to make a living at the lower echelons (read: everything below elite level), but it's hard work or mundane and often not well compensated.
While I love programming and computer science that is one of the reasons I chose programming over other fields. I figure if you want to be a journalist you’re basically broke unless you’re one of the top few that are also making money writing books or doing podcasts. If you’re a historian you are basically not doing well unless you are one of the top and get a tenure track professor position. If you are a programmer, and aren’t even that good, you have a comfortable life in the us. The bottom 20% of software jobs are still in like the top 75% pay range.
Having kids is such a traumatic, life-changing event. I mean this in the most positive way. I can see why so many people see this as the before/after watershed moment of their lives and ambitions. However, it's understandable that not everyone needs to have kids to come to this realization.
For me it was the profound perspective shift from being a leaf node of a tree so to speak to becoming just one link in a long chain. It tends to relax the do-or-die drive quite a bit.
It also makes it feel all the more exploitative that young developers are encouraged to work long hours and put their everything into their job. And that women are de-incentivized to have children by the market, if not capitalist society at large - don't do it, it'll cost you your career! We only give you two weeks of leave after you gave birth. You can't afford child care, or all your wages would be spent on it! etc.
It’s not, really. Japan is absurdly good if the only thing you want to do is have children, and don’t care too much about how or where they’re schooled.
1 year of leave per child, inability to get fired while on leave, free childcare.
The only negative is that all those make advancement options for working age women 25-35 whom are expected to have kids ’soon’ nonexistent.
Why do people feel they can make sweeping assertions about other people's experiences like this? You don't know anything about the lives of the billions of childless people out there. Many of us have livestyles that wouldn't be possible if we had children, and our "moments without kids" are totally, incomparably different from your moments without kids.
The smugness of people with kids is just annoying.
> The smugness of people with kids is just annoying.
The ability of people without kids to have a firm belief on how they’d feel with or without them is also astonishing.
I doubt people with kids are smug about it. It’s just been life changing for them, and to some extend they must regret that other people will never get to experience that, especially if they were initially hesitant too.
Anyway, fully agree it makes you appreciate your time more. I enjoy adjusting the flow of time to school age again, even if it’s not my own.
> If a non-parent told you that you can't truly enjoy your free time because you have kids, wouldn't that be a bit rude?
Nah, I’d nod my head and agree, but I imagine that’s not the point you were going for xD
I feel it’s like growing up, it’s hard to appreciate childhood, or university, or any of that until it’s passed.
By the same token you start to appreciate the seas of time you had without children once you have them, but by then it’s too late of course, and you can only appreciate them in hindsight.
I’m sure the same thing will be true after my children grow up. I’ll appreciate all the time I got to spend with them more in hindsight.
In general I appreciate things a lot more in the moment than in retrospect. And I'm much happier now than I was as a child or a university student--the thing I appreciate most about those times is that they're over.
And look, it's fine we have different experiences--I'm not here to invalidate your experience. But when people say stuff like "You cannot fully appreciate your moments without kids until you have kids though" you're invalidating my experiences. That's total bullshit--you don't know anything about my life and whether or not I truly appreciate it.
If you want to talk about your experiences, that's cool, and I'm actually even interested to hear them. But don't pretend like your experience is some universal truth. It's not.
Maybe I should be smugger about myself having kids, but it was really meant as nothing else as a lightheaded remark. From my own perspective of course - there is clearly my (nick)name over it.
Sure, and if it was just your comment it wouldn't annoy me. But these sorts of comments about non-parents from parents are constant. I totally parents that they enjoy having kids--why is it so implausible to parents that I can equally enjoy not having kids?
This is stated as if the only way for someone to experience kids is to have their own.
I didn't need to have my own kids to appreciate the peace of not having them (although at some point I do still intend to have kids, just not soon). I just had to frequently babysit my niece and nephew.
Nowadays I visit them only for a few days every few weeks and even then it's blatantly obvious that kids would mean a sharp drop in my ability to have as many different projects and hobbies as I do right now, for at least 5-10 years. Makes me really appreicate my moments without kids, as much as I also appreciate being able to see my niece and nephew grow and develop into their own selves.
It seems tragic to me that you had to procreate to appreciate your time on earth. I am grateful and thankful for every moment. Life is full of wonder to me without the burden of children.
To me I enjoy every moments with my children. Not only enjoyment, I couldn't have imagined how much I'm learning from them and I'm experiencing life so much fuller now. For one example, empathy, all the books about how to be empathetic can't make me a bit more empathetic. My children do. Another example being learning to listen, it is humbling to have your children asking you to listen them more. Ok, one last example is gaining inner peace of people not "understand" you. It happens in work place a lot that sometimes seemingly good idea goes unheard. There are lots of things I can share with my young. It is okay if they don't understand now. And I'm learning from them too so what I think I know might not matter 10 years down the road. Live the moment
The rational side of me tell me it is all my selfish gene in action. And I'm loving it.
Writing new code with no existing constraints or requirements isn’t some masterful feat of engineering, usually.
Working in an existing code base, on a team with multiple features going in, gathering requirements from stakeholders and negotiating deliverables on a timeline while refactoring is happening, libraries being upgraded and CVEs being fixed - navigating all of that while being a useful, compassionate team member - that’s priceless on a team.
You're just going to off load your lost potential and lack of ambition on to your kids, and so on and so forth. It's the main reason why people have them.
You're being negged into oblivion, but you have a point.
There is "I'm OK with my pedestrian job but I still try to do my best and actually like the people there" and "my job sucks all joy out of my life and provides me with nothing of value except money".
I have kids, but I somehow skipped the "I want to be the best"-phase and went straight to "I'll make do with whatever you give me" so nothing really changed. I actually work a bit harder now so my kids can see work is good - also to pay for their fancy shit.
Even stupid jobs can be sources of relative joy if you let them. Don't put up with abuse of any kind, though.
The opposite is no good. I had 1 or 2 opportunities which a lot of people of my age were looking at with envy. I was not super gifted but they came out of passion: by working on the things I loved relentlessy, I ended up being noticed. I was good because I worked a lot. It was not hard work since it was stuff I enjoyed a lot to do, but I spent countless hours on it (about 25-30 hours a week in addition to my studies or day time job).
Problem with the kids is that when I tell them about that, they feel they don't have enough passion to succeed. So you see, eventhough I was on the right side of success for a while, it scares the hell out of them...
So true, I wish I had this perspective before having kids as well, I was so focused on the rat race and on "being the best", to the point of being hospitalized with intestinal problems. Now I have a great job, that gives me joy but the two things i care the most in life are my two little devils.
I largely agree. My goals have shifted from having an impact moving the technology world forward to not being frustrated and depressed from work and getting enjoyment and satisfaction from personal small projects with no deadlines.
Since having kids, most of these have shifted from a few months in time to a few years, but they are unambitious, poorly defined and unimportant so who cares if they take a little longer?
My main goal at work is not getting annoyed or frustrated and getting paid well for my time.
It was once said that the national anthem of Hell is "My Way", and that the motto over the main gate reads "Do what thou wilt".
Pride, radical individualism, self-promotion. These are extremely common today, especially in the US (just compare an American resume with a European CV). Both the US and Europe have their problems, but this particular obsession tends to be more conspicuous in the US. Paradoxically, it doesn't bear the desired fruit because it misunderstands the nature of personal excellence.
This isn't a vote for collectivism or mediocrity. Only dead fish float downstream. It's a vote for the notion that the riches of society are the result of countless incremental contributions up and down the hierarchy of value for the sake of the common good. But liberalism embraces an inverted view of reality. All our relationships are viewed instrumentally and transactionally. We place desire above reason. We've become atomized, because nothing sensible unites us (perfect for those in power who view people as "human resources"). The "social contract" view of society is a disaster.
Exactly this. Guy had a comfortable stable job for years. Probably knew codebase and technology enough to have lots of time for family/friends/hobbies. More than this story scares me how HN reacted
It’s not, but from the outside the Bay Area’s cost of living looks impressively bad. A typical house in my neighborhood in the Northeast goes for less than 1/4 of that.
I can only assume it's because HN is full of young, inexperienced people whose main wish is to stay on the cutting edge of technology rather than sinking into a stable profession
You may be right, but I don't think experience is what makes people prioritize stability over anything else. I'd be so depressed at a real job like the one described, it would be irrelevant that it was stable and gave me extra time. Some people are ambitious, some just like variety.
Only if you don't have other aspirations in life. My day job is solely to fund my personal projects that I actually enjoy. Not everyone is out to be insanely rich.
If you fairly profit from your direct efforts in a lifelong career as a productive programmer, you will merely become rich, not insanely so. You appear to have constructed a strawman.
No, perhaps I was too subtle. My point is I don't care about the fact I might be making someone else rich. Life isn't about coming out ahead of everyone else you interact with. Some people simultaneously understand your point, and also don't care about it. Only within your own subjective opinion do you think they must.
Wait you mean life isn't a video game where my position in a company completely defines me as a person, and if I don't destroy all my colleagues (read: competitors)with cruelty and cunning schemes I still have value?
It's 2023 and today companies will quite happily lay you off the moment you aren't profitable to them.
This fictional person made it all the way to retirement and good for him. But the guy who followed this strategy for 15 years and now is job hunting because the company needed to cut 18% of its staff? He's got a huge gap in useful skills and decades left before he can retire.
This is the real reason I put moderate effort into staying current: I've interviewed too many guys with 15 years in the field who can't write actual code at all because they only have written a few dozen lines ever. They are a sad lot and it's embarrassing to even interview them.
If I could be guaranteed I'd have the same job forever with the same tech then who cares, but I've never met anyone for whom that was true. The people that don't learn either retire early or are forced to switch careers.
Post hoc ergo propter hoc. They aren’t incompetent because they worked the same job for 15 years. And they probably weren’t even laid off because they were deemed incompetent.
You just got unlucky enough to interview them instead of someone who got laid off after 15 years and were amazing at their job and could pick up a new tech they’d never heard of and immediately be off to the races.
I met a thru-hiker on the Appalachian Trail that carried a few pouches of cat food. You need to carry emergency rations and they need to have decent nutritional content. But how do you stop yourself from eating it when mildly hungry? His theory was, it should be so gross that you wouldn't possibly eat it except in a true emergency. He made in from GA to CT without dipping into the forbidden slop.
In our company we do medical devices. It is a hard SDLC business. Once a colleague of us left and shifted over to another (much bigger) medical device company. I met him some years later and he told me, retroperspectively, our company is a high-performing, super agile group. His new employer needs a year for a two line code change without really changing but only adding something. And that despite him pushing and pushing.
To be fair: to modify blood dialysis systems is no joke business.
AT&T 5ESS switch had a similar atmosphere back in the day (no idea about now). The rule was no function deletion so you delete the code in the function instead. I remember reading about a very talented guy from Bell Labs being on the 5ESS team and writing some ridiculously small number of lines of code in one year due to all the rules in place to prevent major telephone system issues.
There is a lot of analysis that goes with it. I worked in both medical devices and also transportation.(Rolling stock signalling systems). It is definitely not a joke when you are moving such huge mass of metal with people in it.
This is like the programmer equivalent of WH Auden’s poem, The Unknown Citizen [0].
When I first read this poem I thought it was sad and about a person who wasted their life on mundane, average things. But now I think it’s a positive for a person who lived an average life on the outside but likely had a rich internal life that they didn’t share with the world and that we shouldn’t expect people to have metrics that we can externally measurable as to whether they were great or not.
I think it’s fantastic they this man’s passion is Soylent cat food hybrids for human consumption. He worked all those years and finally broke free for his passion.
The concept of life as meaningless, to be happy with the drudgery, or repetition, instead to be happy with the 'journey', to be in each moment.
Very common throughout literature/history.
I'd like to think we are somehow different, that modern man with technology is reaching some new level, to find some purpose. But now, not so sure, it's still just day-in-day-out, then you die.
That sounds like a well above average life. No problems and a functional member of society in every respect? Good relationships? Five children?!
It seems obvious this guy was in fact happy, despite the last lines of the poem, or else he would have acted out at work, become an alcoholic, divorced, or whatever. That or a supreme stoic.
If this is a jab to supposedly mediocre programmers, it fails to be funny.
I have a friend, let's call him AP, who is a programmer in a local java shop. He's been working in the same place for 12 years since he graduated. He kind of fits the description on this article, but he is one of the happiest people I know.
He gets paid above average wage, and his work is easy and not very demanding. He knows the codebase inside out, and gets along well with his colleagues. Outside work he goes out cycling or for a beer with his friends or travels the world with his wife.
Different people have different priorities, and some are not obsessed with excelling at work.
Everyone can build something new and shiny on the green field. If it doesn't work out you can always find someone or something to blame, because what you set out build was an awesome, highly innovative[1] endeavor anyway. Moving on to something else new and shiny after a while is not only ok, it is expected.
Sticking to something, through good and bad times, on the other hand, that's not something many people can do, and while it is by no means sufficient in most cases it seems to be necessary for sustainable success.
Examples: Guido van Rossum[2], Linus Torvalds, Daniel Stenberg
[1] You wouldn't even have to put the "high risk" nature of it in the fine print, because it's readily understood.
[2] I could not find the source but I remember him saying in an interview when asked about Pythons's breakthrough, that it never had one but that it constantly and steadily grew.
starts promising but ends unnecessarily snarky and condescending.
why dumpster diving? and those guys cannot be replaced by any number of junior devs - that is exactly the problem with them retiring. they know the legacy stuff nobody wants to touch but everything depends on.
so, from that angle this writing reflects very poorly on the author and his work experience.
having said that - the title could fit just about any lawyer. law is the crappiest codebase that you have to spend a lifetime learning. especially the german tax law - but it guarantees a very comfortable income.
also in defense of people like that man i'd even argue that mastering a legacy codebase is mentally and motivationally not that different form mastering a game like chess.
Maybe dumpster diving is extreme, to drive home a point.
But, there are a lot of old people that are hungry, and work as Wal-Mart greeters.
If you didn't save/plan well enough, and 30 years of cool software development ends as being a Wal-Mart greeter, that can seem equivalent to dumpster diving.
guys from that generation (with its past financial opportunities and real estate prices) and of a character required to even work like that - humbly and diligently maintaining a complex legacy code base, very rarely fall victim to financial issues when they retire. those men (and women) are usually very zen and collected. loneliness and suffering from a feeling like no longer being needed is probably the biggest (mental) threat. but the aforementioned personality traits will provide for developing new opportunities like voluntary work or maintaining some open source repo.
don't worry about them ... worry about yourself (if you are young and in IT) and especially about those poor junior devs put in place for this guy.
I'm in medical device software. I'm mid-way through my career (15+ years). I've been on my current codebase (which is 20+ years old) for 5 years. I work with a coworker who's been with this code base for 20 years, but is 45 years into his career. I push back hard when it's suggested that we bring on a fresh college grad (who wasn't even born when this codebase was initiated) and it's suggested to give him/her one of the harder tasks with no supervision. Management: "They have a degree in computer science, why wouldn't that be enough qualification?" Me:"For the same reason you don't name a recent MBA graduate CEO of our fortune 500 company".
Note: I'm all for hiring recent college grads, but they need to be paired with a senior dev, and management needs to understand that you're not speeding up the project this month. Ideally, you're investing in the robustness of your team. The challenge I face is that so many junior engineers are job hopping the first ~5 years. That's not a smear on them. It's good for them. But it's not great for me...
Parent and great grandparent comments are correct. I'm sure the author of the blog had fun with that, but it's fiction. These kinds of pragmatic guys are not the ones you have to worry about, they will be ok. I think they included it to take the steam out of the obvious counter to their snark: Arthur got paid well for his work and did alright, so who cares about the sniping and denigration?
>It’ll take two junior devs and a Keurig to replace him.
Also out of touch. You can throw N juniors at this kind of codebase and without the mentorship of an Arthur type, they will get nowhere. They'll do a "rewrite" with 1/10 of the features.
If you are the kind of person who thinks the future of IT looks bleak right now, imagine what it must have looked like in 2002 when NASDAQ was in a three year death spiral and down 80%, which is an experience this guy lived... and then in 2009 when the stock market was down at levels previously seen in 1997...
If anything, the lesson should be not to worry about yourself because the only way this man could be forced to resort to dumpster diving is if he was given too much to worry about his retirement accounts.
I would dispute that the person that plugs away at a legacy code base, does not necessarily translate to someone that is also good at personal finance.
They can actually be quite opposite, and finances are easy to let slide, ignore as the days go by, while you are zen'd out on some cool technical problem.
There is something to be said for stability. Since our protagonist retired early, the pay must not have been bad. The glass is half full, or so I tell myself.
I know people who enjoy street performance and dumpster diving. Bringing your juggling balls (or guitar) to a street corner and trying to make money gives some pressure that makes it fun - and that you don't need the money means if people don't like you enough to pay it doesn't matter. Likewise people throw away a lot of treasures that you can fix up and sell/give away.
If he needs to be a street performer and dumpster diver to make ends meet that isn't good. However people do those things as a hobby all the time.
I worked with a guy who made easy into the 200k range in the early 2000s two separate pension plans. Total dumpster diver. He loved the thrill of finding stuff. Wore complete rags at work. Drove a car from the early 70s. Ate simple homemade leftovers ever day for lunch. He loved the idea of racking up a high score.
True, but people usually don't start with this on retirement. And the end of the story is about him planning to eat cat food(?). Not sure how to take this, as it also talks about Soylent and Premium Mix, which are not the cheapest.
Retiring at 58 would be awesome. Some people live to work and others work to live. I like what I do but I also have hobbies which I really enjoy.
I get that the post is satire but at some point in your career there are diminishing returns on learning new things. If you are somewhat specialized say in backend systems involving scaling, message queues, clustering, etc. it probably isn't a good time investment to learn whatever is new in the frontend world. And by learn I mean to become proficient professionally and get a job doing it, not just a "hello world" learning. Doing "hello world" learning on a technology is like learning how the yolk and rudders work in an airplane, you're not going to be flying professionally anytime soon.
Learning new things is how you keep your mind working. People who sit around doing nothing in retirement tend do die sooner than those who keep busy. What you keep busy with doesn't matter, just do something challenging.
Strong agree. There is enough to learn in one's professional career that keeping up to date on tangential technologies may not be worth it until you have a need.
Having hobbies where learning and applying things is not just fun but can also scratch that itch to keep your mind fresh.
Agreed. I feel the need to learn new things just to keep my learning muscle strong. Sometimes I actually need to learn a new thing, and if I haven't done it in a while, it's scarier.
I've spent 10 years as the lone dev at a small medical facility, writing database software and dealing with lab hardware etc.
I guess I've touched 100% of the codebase because I wrote it all, but I always wonder if my skillset is so niche that I'll always be at small companies.
Here's the deal though, work isn't everything and I'm at a job that recognises that.
You wrote it all? By yourself? That sounds ideal to me. The horrible part of this to me is that he inherited a mess and left without having done much to it, evidently being unable to do so for whatever reason. The headline says he "mastered" it, so clearly other forces were at work than his skill. That's the depressing part to me, pulling my hair out to try and understand someone else's antediluvian code base which was built under duress.
I think devs imagine someone else out there is having a better time at their computer than they are. I definitely do.
It also said he only wrote hundreds of lines in 35 years. So maybe one line a month. Either he's a wizard that spins gold from straw or "mastered" is being used ironically. Given that he could be replaced by two juniors I'll assume the latter.
I think you are correct. It seems like all of the other prices align with the weekly rate as 5 days. It also makes the daily rate seem more reasonable, at a yearly rate of $240,000. In a better interpretation of this, I could have maybe also assumed that this is just a fun way of stating something like "hiring to work on doubling users costs this much, but there isn't actually a guarantee that the userbase will be doubled."
I don't get these threads, it's impossible for everyone to do a job that they love and that is meaningful ... it's the ultimate luxury. Try to find an alternative way to give meaning to your life. (I have not yet succeeded in doing that by the way ...)
For decades, Arthur raised flags to upper management about the high complexity of the codebase, and begged them to hire a couple of junior devs. Only then could he delegate enough work for him to take on the refactoring project. The software's uptime and stability - he reminded them - was critical to the survival rates of children stricken with spinal meningitis. But management churn and budgetary chaos at the org resulted in the project fluctuating between "investment" and "divestment" states for decades, leaving him to fend for himself.
He told his manager that the data access layer - consisting of 5k LOC batch scripts calling Microsoft Access - needed to be brought up to speed. Maybe they'd consider PostgreSQL this year. Containerization, he protested, isn't going to fix the problems inherent to the system's labyrinthian integrations with countless software vendors and proprietary components. He also reminded them that it's 2023 and Windows containers still don't support RDP.
They still needed a replacement for "Bob". Bob left the org back in 2008. Under duress, and impossibly tight sprint deadlines, Bob was asked to make surgical modifications to the WingdingsAbstractFactory.java class - to be deployed directly into production no-later-than-end-of-sprint. Bob complained that there were no conceivable DR plans, or QA engineers assigned to the project. After a ten hour deployment - and a highly blameful postmortem - Bob exited on record-short-notice.
Arthur was forced into an early retirement under the recommendation of his physician. It was said that his stomach issues were brought on by the stress at work. Only Soylent and tinned meat were within tolerable levels of discomfort.
Arthur will be remembered by his loved ones for his patience, steadfastness, and creativity.
I've done something like this for a couple of years, albeit not full-time. The project was written in Symfony 1.x and got stuck there, as Symfony moved on to higher versions and wasn't backwards compatible with 1.x. The codebase consisted of thousands of lines of poorly documented and commented code, intermingled with custom JS modules that nobody understood. When management wanted to implement changes, it would take days to comprehend the potential effects on different parts before I could initiate any modifications. The project would have required a complete rewrite, but the management was staunchly against it. This has been the most frustrating and unsatisfying tech work I've ever undertaken, alongside past tasks of migrating MS Access apps.
I sometimes find myself pondering why I didn't cease working on it, especially given the compensation wasn't particularly impressive. It's crucial to recognize and act when you sense that a project is no longer benefiting you, and simply move on.
That is one extreme. The other one is developers hopping from new project to new project without ever even seeing the product going live or getting user feedback/bug reports from the live product.
This is quite funny in a painful way. I identify on a few different vectors. I started my "professional" career in telecom. There I really did become an expert on some very small percentage of code. It was exhilarating to do patches in 286 assembler for an obscure language (CHILL). Now many years later, I work in NPD at one of the small divisions of a giant Medical device company. I wrote 90% of the legacy code. It is embedded and I have not needed to update any of it in 10 years. With changing management styles, the new push is to buy new products from smaller companies, and just to the required paperwork for regulatory approval.
Hadn't heard of Poe's law.
I'd say the best sarcasm does contain a lot of truth. Think it used to be that the best sarcasm did leave you wondering a little if it was meant to be sarcasm.
Interesting idea.
It can be that everyone has different knowledge base, so everyone has individual interpretations.
For the post in question. I've seen it in real life, so it didn't seem 'extreme' enough to be obviously 'sarcasm'. Though, I could tell it was sarcasm, it was perhaps the good type of sarcasm in that it contained a lot of truth.
There will be a lot of these people maintaining the currently fashionable microservice/multiple languages/use as many AWS services as possible/scalable systems in a few years.
I have a friend who just started a job working with AS400's, learning RPG. He's not much of a programmer, but I am, and this made me think, hmm... maybe I should learn it too. There's going to be some serious job security in that field, once AI is writing all the CRUD web apps from scratch in Nodejs.
I’ve tried to look for the COBOL and other legacy languages jobs where there is a shortage for. I came short. And the few I found didn’t pay like there was any kind of shortage. There was a lot of willing offshore sweatshops offering their services though.
There is something admirable about someone who commits to maintaining a codebase, no matter how crappy it is. We don't see this often in tech and often being a person who is more reliable and stable is more desirable than two junior devs and a Keurig machine. Speaking from my first hand experience when reverse engineering a poorly documented product for a small company these people were the real heroes. Thanks to all the "Arthurs" that keep the world spinning.
Most coding is pretty mechanical... and most juniors can pick that up easily. In spite of the mysticism surrounding software development, higher level languages are actually intended to be easy to pick up. If you have good problem solving skills, those are harder to replace with a couple of juniors (or ChatGPT).
If coding is mechanical, then the code will often turn out to be overheady, uninspired suboptimal solutions á la everything is a class, not well extensible and maintainable, or the task at hand truly not fit for the skill of a good developer.
Coding is a craft. There is an art aspect to it and there is a lot of room for inspiration and ideas to improve things.
I don't disagree, but in my opinion code organization (which is what most of these things are) is mostly subjective, in other words, beauty is in the eye of the beholder. Good organization certainly helps with maintainability and extensibility, but it's not a prerequisite, and it won't on its own solve tricky or vague problems.
Question: Wouldn't a couple of juniors be more expensive? I don't think I ever work at a company where junior developer was paid less than half of a senior developer.
That's me, got fired 4 years before retirement. Made them rich. Young guys came in, it all had to be rebuild in microservices. Last press release, they have tripled their staff, financial turnover went down, the new system is working but is lacking the functionality of the old systems. All the pitfalls i warned them fore and got me fired, became reality.
Programming is a job, it is not a mission to save the universe. A few programmers become famous for a combination of talents and luck, but nobody can control either of these things. In the end, 99.99% of programmers do the same things that this guy is making fun of.
Life is too short for working with crap code. Unfortunately there is more crap code out there, than there are opportunities of working on beautiful code. At least in most jobs there will at minimum be compromise and part of the job will be dealing with ugly code.
Recently on a call my client introduced me to his $BigCorp whale client as "my IT guy" and his client said "oh yeah I have 12 of those here on my team".
I'm cool with that. I keep doing work valuable enough to pay well for, and the rest is semantics.
That's not how it works.
Those who've been there 20 years and who would have difficulties finding another job are those who get low salary increases. On the other hand they've been there for so long without looking elsewhere that they probably have no idea that their pay is low.
> He probably made fat bank which turned out to be golden handcuffs.
This is a big assumption, not everyone in this field is a high earner - plenty of people make okay-ish salaries without being anywhere near highly paid.
That's what Dan did at my old job. Retired at 50. We would go on per diem travel and he would bring food from the hotel. We called them 'Dan-wiches'. He was smart and frugal. Now living his best life.
Wasn't this a bit of a self deprecating summary of his own career?
Many of the comments seem to assume that it was written by his colleagues and that the description of his career is condescending or snarky.
Personally, I find it really funny, and that it probably describes many of us, despite the fact that we think we are way too cool to ever be like the character described in the piece.
Most of today's new tech will become tomorrow's 'legacy'. Imagine how your awsum multi lambda hairball deployment will look in 30 years. (If it's even around in three decades.)
This is how it should be on salary. If you're good enough then you can get time back from the job and work less per week. It starts to look crazy good when you are doing like 5-10 hours per week because you're so familiar with a code base and have a decent understanding of the business. If they aren't going to give meaningful raises for doing maintenance, then time is the only thing you can get more of.
If companies want people to work harder, they'll have to give more compensation that scales better with performance.
The idea that two interns are going to be able to take over for this guy was the funniest part of the article. Maintaining a large old legacy code base is a hard job because there is so much to learn before you can even start. Codebases like that live that long are often used in important places where a YOLO approach of “test it in production” can cost tremendous amounts of money or even lives. The article makes fun of the guy for only committing hundreds of lines of code, but they worked perfectly the first time and every time.
Einstein once said "Once you stop learning, you start dying". I feel like in IT it's the case for most what people are thinking. In my case I have been switching tech because I felt I wasn't learning anything new or simply because it's was dying.
I feel like these guys like him are probably too confortable in their confort zone or too lazy to learn anything new (the less you do the less you've got motivation + you can't teach old dog new tricks).
To be fair you can learn outside your job. I wouldn't call him lazy just because he stuck with a company for a long time. Most jobs will only require a small percentage of a programmers' knowledge, so people supply their urges with outside endeavors. Hence reading books, doing Open Source, side projects, etc.
Lots of significant inventions in computer science began as a side project only the creator cared for initially and had to borrow his own time to do it.
This is quality satire :) it also works somewhat for larger codebases, even non-crappy ones.
> Arthur had a knack for working hard without learning transferrable skills. He will be dearly missed
When I was at Google during the pandemic, I noticed more and more people saying they needed to learn open source tooling so they can have skills that transfer to the outside world.
That company is full of brilliant engineers who have mastered (part of) a great code base but have to start at the beginning with popular non-Google tools.
Nothing against Arthur Westbook and his career choice [;)] but I think the proper way of turning mastery of a crappy codebase into a career is joining a bug bounty program. Some are big enough (think Yahoo's scope) that you can earn a living mastering them and finding bugs. But won't those bugs dry up eventually you ask...well thankfully new features, API endpoints, (sub)domains etc. are intruduced constantly.
I'm unfamiliar with bug bounties but how often do you get to peruse codebases? I thought most of the time you're working in a blackbox setting with just an outside interface that you peek through?
I mean, a lot of people work on things that, despite being billed as a Great New Thing, are throwaway projects that will disappear without a trace in a few years (once management changes, the market turns, or the founders/VCs cash out, or the SEC comes knocking, etc.)
At least a 35yo codebase probably actually does something more useful than sell a narrative whose purpose is to enrich a few random jackasses.
Often times great ambition requires moral concessions that are untenable even in the face of the end result.
Politics is a helpful lens. Either you compromise and muddy the purity of the result, or you take the high risk path of being uncompromising and greatly reduce the odds of achieving any result at all.
When facing this, many individuals optimize for lower-risk self-fulfillment over great ambition.
This type of cynicism/snark used to be a lot more funny before it dominated all internet discussion and interaction. Maybe the people who spend all their time online are the bitter/angry ones and it just has taken over all discussion?
In any case it's just a job, and anyone harboring illusions otherwise will learn the truth soon enough.
From 1997 to 2012 I was the sole IT guy at a trade show company. We had a database called Smiley.... I wrote 3 different replacements for it, they were always afraid to convert over, even though I was fully prepared to run them both in parallel.
I've been this guy, sorta.
I've been afraid to even try personal projects since then... that job broke me.
I left my previous job because it was obvious this was my future (or would be my future if the company could stay in business, which was unlikely). My mandate was to fix all the problems without changing any of the founder's work that caused the problems. My new job is terrific.
"Two junior devs and a keurig" - that will totally wreck the system by refactoring it to "modern patterns" without bothering to understand what it actually did. But it's OK, they're younger and therefore better. Thus, the cycle begins anew.
I think that was the point, but see all the sympathising comments here. Being full-on driven by your day job is only one way of having a good professional life, and I wouldn't say it's the best or the most common by far. The way how software development is democratised and slowly becomes the next blue collar job, the engineering hero ethos will fade.
If you serve plain static HTML from S3 or Cloudflare Pages (or even a vanilla nginx on a vps) and put a decent CDN in front of it (and don't load 3p resources), it will astound people how fast your page loads. I regularly get this feedback on my own webpage.
It's not rocket science - it's how we made webpages in 2001. Browsers are really good at that.
I had about 15 jobs over 40 years, not including 9 years at the two companies I started. But some were short term contracts, and several (in a row, sadly) ended in my employer going out of business, mass layoffs, etc. I always found doing something new was revitalizing. Also found that sometimes something new was awful. Being comfortable changing what technologies you are expected to use helps, taking advantage of new tech being new, and not many people knowing it, is good too. At my last job before retiring I stayed for 5.5 years, and could still be working there today, except it was enough of a career.
If I don't have easily-transferrable technical skills, how much additional value can I offer to a new potential employer in any engineering (non-management) role above junior or maybe middle-ish? I obviously recognize that there are other areas of skills, and that learning new tech gets easier with experience, but isn't there a limit to that?
I've seen it. Some people are weird and just want a stable job that pays them alright so they can focus on their family and other things they care about.
A few times in my career I've seen people with a decade-plus experience leave companies they would have stayed at for another two decades if it were within their control. It's sad. They struggle a bit because they often realise that they've over fit their skillsets to that single company.
One guy I knew was hired as a "web developer" in ~2005 because of his HTML/CSS knowledge. He continued to do basically just HTML/CSS until he lost his job in 2020. Lovely guy, but he saw his job as just a job and couldn't give a damn about learning new technologies. I often wonder what he's doing now... I doubt just knowing HTML/CSS is a hireable skillset anymore.
> Some people are weird and just want a stable job that pays them alright so they can focus on their family and other things they care about.
What's weird about that? It's how things work in every other sector, why would our sector be any different?
I wouldn't want to be stuck in the same position for decades, but that's different than being at the same company for decades. If I found a company that treated me right, paid me enough not to worry about money, and allowed me to progress internally (at a reasonable place), I wouldn't complain.
That’s a common story on HN but not at all a reflection of real life. Be careful letting the internet convince you of something without doing your research. There are lots of us with deep tenure at a single employer. It would not surprise me to find it’s still the majority of professionals.
Yes, I know many people in software/tech who have been 20+ years with the same company. Hopping every 2 years is not common.
That said, it is a red flag for me if most people in the team have been there for very long: it means that as a new comer I will always be bottom of the pile and have no career growth opportunity.
I'm that guy at work. I solved the problem by completely rewriting our huge and aged out internal application. The plot twist is that I also completely wrote the original crappy codebase.
If he already mastered the crappy codebase, then "two junior devs and a Keurig to replace him" or even a senior dev might not be able to replace him in a short amount of time.
There was a Dilbert once (no longer available, for some reason), where Phil, the Prince of Insufficient Light, curses Dilbert to select one of two choices:
1) You will be paid well, but your code will be destroyed every night.
2) You will be poorly paid, but your code will be useful and appreciated.
Dilbert says "Hey! They're both better than what I have now! Wally, you should come and get some of this!"
Lots of reasons. Many folks are afraid of the unknown, so it's the devil you know, kind of thing.
That's why most folks aren't entrepreneurs. There's a ton of risk, involved in that kind of stuff, and most folks are risk-averse.
In my case, my job was sort of soulless, but only in parts. Many folks here, would say that it was a horror show, but I also worked for a company that gave me business cards that would open doors, I worked with some of the finest engineers and scientists in the world, enjoyed considerable trust, and had great stability.
Nothing comes for free. There's pluses and minuses to everything. I find it doesn't do me a whole lot of good to judge others on the choices they made, and, in the aggregate, there's not much in my life that I regret, and that's worth more than money.
are you also opposed to higher education/research in anything other than hard science, or do you think people in those fields should just stick to pencil and paper?
Well I guess it depends on what we define as art. I don’t think putting a piece of cheese on a table in the middle of a room is art or thought provoking beyond “this ‘artist’ is fucking shit”. Or running and jumping while drawing a line on the wall. That’s just insulting to actual artists who have talent that we can appreciate.
There's plenty of low effort garbage on the internet that I would consider a waste way before this piece. Just the reactions of some other commenters making them rethink their career trajectory shows it has value.
He's the guy you go to when you find some legacy code which you have no idea how the hell it works and end up getting a 2 hour history lesson into a decade of company politics and failed replatforming projects.
They're hard not to love.