I periodically think "Man, I really shouldn't be getting paid for this. Any idiot could stick these two APIs together."
In my more sensible moments, I remember that that idiot only needed to know HTML, CSS, the DOM model, Javascript, jQuery, HTTP (mostly headers and status codes), Ruby, Rails, basic MVC design, Oauth, SQL, SQL performance, five-ish APIs to support the two that needed integration, how to configure and maintain a server, system architecture, a bit of security, etc.
[There's also object oriented programming, class inheritance, polymorphism, algorithms, time complexity, discrete math, data structures, imperative programming, and a few other things I forgot the first time around. Lest we forget, we might take these for granted because we've been programming since we were $ARBITRARILY_YOUNG, but to most people these are just black magic. My girlfriend, a smart cookie, asked to see "how I made the phone ring" and, after I showed her the code that did that, told me it had never even occurred to her that every program she's ever seen was once a collection of special words placed in a particular order.]
After that, it was pretty much done, except for the marketing.
On the plus side: you can learn one or a few of these things at a time, and get more comfortable with how deep the rabbit hole goes as you go along. I coded my website in static HTML written in Notepad with no JS or CSS, crikey, only four years ago. (And my brain still recoils at how much better I'd have to get to do e.g. web scale work.)
It's like maintaining a skyscraper built out of Tinker Toys.
What I wonder is: Does the following sentence run through the head of every expert in every field?
Man, I really shouldn't be getting paid for this. Any idiot could [do X].
It certainly felt true of most of my day-to-day work in graduate semiconductor electronics. It felt true of a lot of biology research. My conclusion is that there are a mere handful of days in your career when you will have to do something so hard that you will feel smart about it. But most of your career as an expert will be spent doing stuff that you know so well that it's kind of routine, even if it is totally esoteric stuff that only you know how to do.
Feynman said it well when he teased the mathematicians: "Mathematicians can only prove trivial theorems, because any theorem, once proved, is immediately seen to be trivial."
Man, I really shouldn't be getting paid for this. Any idiot could [do X].
Another variation: what I do is not easy but every person practicing in my field can do this just as well as I can.
I worked very hard for many years to master my craft and always felt deficient along the way because the individual pieces seemed so daunting as I encountered and dealt with them. Not daunting in the sense of insurmountable. Just difficult. Long hours, Lots of sweating it out to get a given project done, one step at a time.
Nothing new here. Everyone who works hard to try to excel in his field goes through it.
It took me 10 years into my career to realize that many in my field (law) were not like me but contented themselves with cutting corners or otherwise coasting along while seeking to avoid the hard challenges.
Doing something right, and learning to do so consistently, is hard in any complex field. It does seem easy once you have mastered the difficulties but not that many are willing to do what it takes to get there. People are all too often content with mediocrity.
"It took me 10 years into my career to realize that many in my field (law) were not like me but contented themselves with cutting corners or otherwise coasting along while seeking to avoid the hard challenges."
I remember going through that moment. It puts the quantity of "best practices" chatter into perspective. People say this stuff over and over because it seems so widely ignored.
"It's like maintaining a skyscraper built out of Tinker Toys."
Actually, it's like maintaining a skyscraper built from steel, concrete and glass integrated with complex HVAC, plumbing, electrical and elevator systems.
This is also true. But it's a question of levels of abstraction. The skyscraper is built out of (in part) a plumbing system, the plumbing system is built (in part) out of pipes, and assembling pipes is like assembling Tinker Toys. Which is why we give a five-year-old aspiring plumber a set of Tinker Toys to learn with. [1]
But this is only a few years after the web was invented, and the average website is still sufficiently ad hoc that the Tinker Toy level is a lot more visible.
---
[1] Break any human-designed system down far enough and you'll often arrive at something as simple as a Tinker Toy. Although, of course, Tinker Toys are not that simple, so this fact may not help as much as you'd think. And this is not true of all systems: In metallurgy, the "simple" parts are atoms, quantum mechanics is not like Tinker Toys, and even if it was nobody has experience with a Tinker Toy set containing 10^28 pieces.
"Actually, it's like maintaining a skyscraper built from steel, concrete and glass integrated with complex HVAC, plumbing, electrical and elevator systems."
Adding, "Designed by a committee. Of MC Escher fans."
Two different development processes, two different outcomes. And unlike other 'construction'-type businesses, in software we have many different development processes.
Man, I really shouldn't be getting paid for this. Any idiot could [do X].
I definitely think that periodically about technical translation. Any idiot could read some text and type the exact same text. Then I remember that not everybody can read German, French, Spanish, Italian, and Hungarian, and still stay focused on being able to write readable English.
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision.
> Does the following sentence run through the head of every expert in every field?
> Man, I really shouldn't be getting paid for this. Any idiot could [do X].
That's because most experts are humble. That is what allowed them to become experts in the first place.
I think the comparison was with embedded programming, which can be pretty hard in its own right.
Also, with lots of embedded stuff, you don't get the "do over" of being able to update your server with the latest code. Once it has shipped, it had better work. In some cases, people's lives may directly depend on it, something that's probably fairly rare in the world of web programming.
I'd hire someone good with C pretty quickly, if I had the time to let them get ramped up, and they demonstrated the flexibility and willingness to learn.
I hopped on your website, sir (madam ?) and no doubt, you are a rockstar. I'm talking about typical jobs -- typical embedded vs. typical desktop vs. typical web. Yours does not seem like a typical job, nor do you appear to be a typical developer. Most typical desktop/embedded jobs out there do not require keeping up with constantly changing technologies the way typical web jobs do.
My husband's just starting to get into web development in the hopes of being able to do it with me in a few months. He's a C programmer from years back and has a great head for sorting out and designing how an app should run. His theory is solid, and he has real programming experience to back it up.
I've been bringing him up to speed on the web stuff, though, and he's said the main thing that gets him is that he can't just learn a language and build an app. He's got to learn HTML, CSS, JavaScript (the DOM, jQuery), SQL, PHP/Ruby/Python and any accompanying framework we choose to use, etc. and piece it all together.
It's not that he finds any one of them particularly difficult (he's picking up on them quickly), it's just that there are so many pieces to the puzzle that all need to be understood and fitted together (and they often inter-mingle so much that you can't always separate them out and do one, then the other). It's just a large and varied volume of things to process.
I bet 90% of the time, any "expert" is working on things that any idiot can do. But that isn't why people pay experts!
I think experts are paid so much because of the other 10% of the time. When convoluted edge conditions have made a mockery of all things abstract and sensible, they are capable of diving into the black boxes and figuring out what's really going on to save the day.
A good friend of mine is a master carpenter. When I need something done on my house I'll always hire him or his equivalent in electrician or plumber. And it's mainly because I just want the peace of mind of knowing the job will get done properly. He charges a lot but things always get done quickly and without hassle.
I've watched other home owners I know try to do things on the cheap and they're constantly getting burned. Things take longer, there are unforeseen problems and jobs quickly develop into hassles. If you can't afford to do it properly then just don't do it. And if you can't afford to hire the right person for the job then don't hire anyone.
As a cheap programmer, I disagree. An expert may spend 90% of his time doing something he feels is drudgery, but he does it properly, so as to prevent the 10% all-hands-on-deck emergencies from occurring.
If you hire cheap programmers to handle the 90% under the expert's supervision, the expert still has to painstakingly review the work so as to ensure it was done in a way that at least sort of resembles the method the expert himself would have used. And having reviewed other people's code, I am of the opinion that process is sometimes soul-crushing.
As a cheap programmer, I appreciate the consideration, as I like to feel I have some utility. But as an entrepreneur, I think it's a bad idea.
As a not-so-cheap (by local standards) programmer, I'd have to agree with this assessment. Sometimes, customers will ask if it's possible to reduce the budget for a contracted piece of work by getting some of their internal staff to do parts of it. I've tried to make this work, but it basically doesn't in most cases, for the reasons you mention. On the occasions I agreed to try it, it would have ended up more expensive, even when assuming internal staff time to have a cost of zero. Fortunately I agreed to try it only on the condition I could bail out and do it all myself if it wasn't working. Which I did.
I realise this may come across as me being arrogant or unable to work in a team. I can't with good conscience pass judgement on the former; I am however frequently contracted as an extra brain and pair of hands on projects with people of comparable skill who could do it without me, save for the project's time constraint. This generally works very well.
It's conceivable that the model might work if the skill gap isn't so big. Presumably that makes the programmers less cheap, and the scheme less attractive. Plus, they could probably muddle through without me, thus never feeling the need to hire me in the first place. Which, I assume, is why this has never happened to me.
I concur, a sort of an apprenticeship program would be quite useful. If an expert can guide 1-3 people to do most menial tasks, it would serve the dual purpose of getting stuff done and educating a lot of interested people.
Currently we have university projects, internships and the like. However those are often slow to react to technology changes, or are too rigid and lacking proper employee orientation procedures.
It is a nice pipe dream to have a sort of a board where one could find a mentor in a particular subject, while working on a real-life project that shows all the dirty stuff that most undergrads are sheltered from before they enter the job market. E.g. topics of the "I'm doing a Django project and I could use some slaves to do this, this and that" sort.
Being a web-programmer - those C and GUI stuff looks hard to me. The grass looks greener on the other side, the tech looks harder on the other side. I am humbled too.reply
It's interesting, I definitely see myself thinking the same thing (which may lead to myself undervaluing my skills because "any old dummy!" can learn this stuff). There are days where I've joked where I'm a "glorified typist" :)
I think in order to become more and more skilled you tend to abstract certain things out and take them as a given. Basically you know how things work, you just never think about them.
I remember when my company had an intern, and his only real education was college based (he hadn't really worked on any of his own projects). I had to explain how our stack worked, and generally these are a given when you're working with the stuff. But to someone who has never worked on this stuff it's very odd and unfamiliar. I also think that in order to build a good product for someone it helps to get to that point. I don't really need to spend a lot of time thinking about implementation these days and can focus on what would make clients happy.
To users, this stuff just works and I forget that sometimes because I'm in the weeds.
Integrating various python modules/ruby gems that all monkey patch over each other?
ducks
On a more serious note, the closest equivalent I can think of is porting games to work on windows, linux and mac platforms. You would have to have a good understanding of the entire renderer, any main event loops and physics, as well as experience in writing very low level code on all those platforms.
Yes, this would be very challenging. Making games cross-platform is not the typical GUI app out there, and is not the sort of app I was thinking about on my blogpost.
You forgot the architecture needed to sustain a development enviornment - the VCS, bug tracking, deployment (ssh? shell scripts? something more modern?), and so on. But that's probably because it's all so obvious.
I've always been a hobby programmer, but over the last few months I've been developing some web sites and trying to bring my skills up 'to the next level'. The things you mentioned- vcs, bug tracking, deployment- are definitely non trivial. Partly because while most programming languages are well documented and have lots of tutorials, the things you mention are almost a black art, something every programmer does a bit differently (I think) and rolls their own scripts.
For example- there is a php project I want to contribute to with bug fixes. I can fix the bugs no problem, but submitting a patch & using the bug tracking system? Way beyond me!
VCS and bug tracking are not difficult. You can get it all by just installing fossil. If that's not your thing, you can install tortise hg (on Windows) or some other dvcs and get started in next to no time.
Deployment is definetly ugly. There's stuff like pip, fabric, and virtual-env (all on python) which are meant to ease the pain ... but the fact that agile geeks reccomend 3 different frameworks just to make things simpler show how ugly it is. I don't know what the cool kids use for PhP though.
"only needed to know HTML, CSS, the DOM model, Javascript, jQuery, HTTP (mostly headers and status codes), Ruby, Rails, basic MVC design, Oauth, SQL, SQL performance, five-ish APIs to support the two that needed integration, how to configure and maintain a server, system architecture, a bit of security, etc."
You should talk to my new boss, who just tore a strip off my ass for not getting somethings done in 3 days, that took another team in the company 3 months. Oh, you just have to "do this", no problem. Invoke Magic, upon 10 year old hardware.
In my more sensible moments, I remember that that idiot only needed to know HTML, CSS, the DOM model, Javascript, jQuery, HTTP (mostly headers and status codes), Ruby, Rails, basic MVC design, Oauth, SQL, SQL performance, five-ish APIs to support the two that needed integration, how to configure and maintain a server, system architecture, a bit of security, etc.
[There's also object oriented programming, class inheritance, polymorphism, algorithms, time complexity, discrete math, data structures, imperative programming, and a few other things I forgot the first time around. Lest we forget, we might take these for granted because we've been programming since we were $ARBITRARILY_YOUNG, but to most people these are just black magic. My girlfriend, a smart cookie, asked to see "how I made the phone ring" and, after I showed her the code that did that, told me it had never even occurred to her that every program she's ever seen was once a collection of special words placed in a particular order.]
After that, it was pretty much done, except for the marketing.
On the plus side: you can learn one or a few of these things at a time, and get more comfortable with how deep the rabbit hole goes as you go along. I coded my website in static HTML written in Notepad with no JS or CSS, crikey, only four years ago. (And my brain still recoils at how much better I'd have to get to do e.g. web scale work.)