In addition, engineers have commoditized many technical solutions that used to be challenging in the past 15 years. Scaling used to be a tough challenge, not any more for many companies. In fact, part of my daily job is to prevent passionate engineers from reinventing wheels in the name of achieving scalability. It's not because we don't need to solve scalability problems, but because the infrastructure is good enough for most of companies. Building and operating so called "big data platform" used to be hard, not that hard any more. Building machine learning pipeline used to be hard, not that hard any more for many companies. Of course, it's still challenging to build a highly flexible and automated machine learning pipeline with full support of closed feedback loop, but many companies can get by without that level of maturity.
But maybe it's actually an appropriate term, now when our interview processes so clearly have ceased to be job interviews, and have become auditions - because nothing less satisfies our own grandiose self-importance.
> "On the other hand, how many companies desperately need product talents to figure out what is valuable to produce?"
These middle-management positions are so unmeasurable that I think they may be greatest source of bullshit jobs in tech. So if it's true that their success is largely unmeasureable, I wonder how you've come to that conclusion?
Yes. This one is accurate.
And yes, in a past life I was a master licensed plumber.
But, product improvements shouldn't be immeasurable. Through analysis of engagement metrics, talking with users, etc. one can measure improvements. Sure, it isn't a direct measurement, but pure unmeasurable intuition is not how good PMs function
It's the credit assignment problem IRL.
If this were true, hard-won experience would be prized, as it is in law, medicine, and all real professions, instead we have rampant ageism and statements like “young people are just smarter”.
Programmers that call themselves "engineers" are fooling themselves. Unless they are doing the sort of programming that the NASA shuttle team used, none of us are "engineers".
There is a career called C.S. Engineer and it has an huge amount of math and physics (equivalent to an undergrad + master's in the U.S.). Most of us take the career that's equivalent to an undergraduate degree though, the extra math (and grueling attrition) is mostly useless.
But graduates are very sought after because they've proven themselves able to go through a tough grind (even though they don't have a shred of practical experience after graduating)
And anyone with a degree who works in there field is a "engineer" even though the number that go for full Chartered status is a small minority - and that is more about who you know not what you know.
The BCS is a complete joke, completely irrelevant to anyone working in the field, I have no idea how it still exists or what it actually does for practitioners or for the industry.
I never bothered going that route as the BCS kept changing their minds on qualifying experience and the rules and actually don't do anything concreate for their members unlike say the BMA does for doctors.
So honest Q here if you feel so strongly why haven't you got your Ceng / E status
I don't feel strongly about it. I don't need people to call me an engineer, because I'm not an engineer and I have no intention of being one.
I'm a software developer, which is very different.
It's really not. Most of the software you use every day was developed by very average software developers. The vast majority of the work software developers do isn't terribly novel or particularly difficult. And you know what? They do just fine. Things work well. Systems stay on line. Companies make money.
I've seen too many "bad" teams produce perfectly functional software over the years to think that this gap exists anywhere but in Joel Spolsky essays.
I think comparing carpenters to software engineers does a disservice to carpenters. Software developers to me are more like dry-wallers. A really good one can do it faster, but the end result mostly always looks the same.
The difference between an average and a great carpenter, however, is immediately obvious. There is real craftsmanship there, and you can clearly see the difference.
It isn't until you get to the edges of software development that those same differences become clear. When you're dealing with very high scale, very aggressive performance metrics, or very difficult mathematical concepts you see the difference between average and great developers. But those are less than a fraction of 1% of software projects I'd wager.
* I should note: I'm a software developer.
If you have great developers, there will be a single straight wire from the switch to the lamp.
The end result for the user is the same though, until the mess of wire eventually overheats and burns down your house.
The key factor that you're probably not thinking about is time. A good dev doesn't think faster or have blazing fingers that fly across the keyboard, a good dev is constantly setting him/herself up to shave time off of future tasks, and those investments are constantly paying dividends.
Those "bad" teams can definitely produce software that runs, but they do it 10x slower than a "good" team could.
What takes a bad team of four six months to make takes a good team of two only 5 or 6 weeks.
Time is the key. Writing maintainable code is the investment a good developer makes because it pays off by increasing speed of interacting with that code.
I should note that I’m a software dev and a family member is a general contractor
While I do feel that folks grossly underestimate the skills that the best carpenters acquire over time, the software ecosystem generally is unique is that it's vastly large and complex. By any measure (switch points, surface area, person-years, etc.), the software we've all built in the past 70-ish years is the most complex thing ever implemented by human-kind.
Long ago it became far too large for any one person to "master" to the same degree an excellent carpenter might master his or her skills.
And the market agrees with the point of view, and values the economic output of the "best" software developers (at scale, not just one data point) at a far greater multiple over the "ok" ones, relative to other skills and trades.
One may not like this analysis (and my associated statement), but it is the state of the world.
For example, the people at Apple who came with blocks and Grand Central Dispatch did an amazing job, but the difference between them and the average software developer is like 100x. A great software developer can write something that other developers can build on, which makes them more productive.
Conversely, the people who came up with Java, they figured out how to turn a great programmer into an average one, and make them want to quit computers altogether, but the below average programmers are now also average.
Projects that don't fail, usually have to be rescued from some crazy doomed plans fairly regularly.
You seem to have found a calmer, saner corner of the industry.
Any profession to which someone can devote their life to (incl. carpentry or software development) is going to have gaps in skills between "ok", "good", and "great". Pejorative attitudes which measure something like software development as "harder" than carpentry are not beneficial and simply dilute the discourse.
Especially as many software devs are becoming more like "code plumbers," if anything the skill gap may be bigger in carpentry in the future.
First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration.
Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc.
The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization.
The bigger question in my mind: why firms at all? If individual PMs are so great, why bother with employees? Wouldn't you rather just contract all the elements of a technical project to other folks for a fixed fee?
I do wonder what portion of good ol' boys and good ol' fashion nepotism tend to fill out the "PM" position on big firm's org charts.
From the article, "Firms exist as an alternative system to the market-price mechanism when it is more efficient to produce in a non-market environment."
Also, programmers are (often) worth more in teams. Two programmers who work well together are far more productive as a pair than the sum of what they'd do in isolation.
This isn't to discount the hard work involved in completing the technical tasks, but commiting _to_ begin something along a certain development vector is, indeed, the hard part.
Why not just... not employ the people you need to fight in order to get anything done? How does the organisation and its shareholders benefit from employing full-time professional obstacles to progress?
talking to customers (read: networking for their next job),
going to conferences (read: shmoozing with their next potential boss),
researching competitors (read: figuring out where to apply)
researching products (read: learning on the job)
Is it any surprise they tend to make more than people stuck in the office coding?
going to conferences -> Because face-to-face contact is far faster and richer than sending emails
researching competitors -> What aren't we doing that others are doing better / what are we doing that's winning / what to change?
researching products -> well, yes, constantly learning
Listening to customer needs, staying ahead, constantly learning. Sounds like good general career advice.
A paradigm shift is necessary before we end up strangling ourselves.
Nothing wrong with learning new things, though. I can get behind that. It's important to stay intellectually honest, however.
On the other hand, research efforts on vision, language and autonomous systems are genuinely pushing boundaries.
I 100% agree that most companies don’t need nearly as much ML as think though.
I think that's what the bad PMs do. Good PMs think 'what is the smallest possible thing we could do to start truly adding value against our problem space?'
A good heuristic is that good pms are problem oriented - bad pms are often focused on solutions first
On a side node, engineers do sometimes need to function as PMs to build great product.
I'm also of the opinion that a technical manager who purposely relegates engineering to a forgotten past is one who is pragmatically posturing for premature obsolescence; an axe in denial who refuses the occasional sharpening cannot be discerned from a rusty shovel and will sooner dig his own grave.
Also, even in the 0 experience group, the selection will be tougher. This is independent of the measurable or existing differences in skill. The person that is best at convincing the employer of hiring them as a solo product manager is in a better position to negotiate than the person who managed to be among the five seemingly best developers.
This could make the difference between having to do a complete rewrite at great expense, vs a small tweak.
For startup with no customers it's probably fine to cowboy up a solution, then throw away and start again, to find market fit.
But if you've got lots of customers, and you're forced to rewrite because the previous engineers left an unmaintainable mess, it's going to be hugely complex, risky, long & expensive undertaking.