
The Bulk of Software Engineering in 2018 Is Just Plumbing - karlhughes
https://www.karllhughes.com/posts/plumbing
======
alexanderdmitri
When I was in college, one summer I worked up in Vermont putting together and
maintaining trailer park homes. Plumbing was one of the best parts of the jobs
in my opinion. I'd wager the person who wrote this article has never had to
put a plumbing system together from scratch.

There is an art and a science to it. The guy I worked with put a lot of
thought into the best way to design these things, often because he'd be the
one who would then have to maintain them. He'd also get very upset when we go
out to a trailer with a poor plumbing system. The main giveaway would be a
mess of pipes going every which way without rhyme or reason. These systems
were the ones that broke the most and were also the hardest to fix.

It might just be "plumbing" to someone that doesn't have to do it, but there
is a whole lot that goes into it and you don't want a plumber who approaches
his/her work with the catch phrase: "Well, it's just plumbing."

~~~
rgrieselhuber
We've been conditioned to believe that trades such as plumbing, carpentry,
construction, etc. are beneath high IQ people so it's easy to see how this
mentality carries through into software engineering.

Infrastructure, properly done, outlasts its own empires.

~~~
gnarcoregrizz
I've met people in the trades that are just as smart, or smarter than most
white collar folk, including software engineers. A lot of these people may
have dropped out of high school or college, but street smarts are real

Some of the successful ones that own their own businesses are able to make a
lot of money (a consistent >500k/year). The downsides are that those
businesses are difficult to scale since they are service oriented, and they
depend on good physical health. Moreover, consistent quality is mostly the
responsibility of the owners, and again, that can't always scale far either.
You also get out what you put in, in most cases it seems like the pay is a
linear function of productivity. It's definitely not easy or normal to make
that much, but it's doable as a tradesman

~~~
et-al
Agree with everything you've mentioned. One thing I'll add about tradesmen is
they're some of the wittiest people to hang out with. Makes sense considering
half the time it's a bunch of folks riffing on each other on the job.

~~~
noir_lord
Aye, You reminer me of a funny story.

Before I was a programmer I was an industrial electrician and had worked on
sites, after I was a programmer we where down on a site installing some
equipment and a plumber walked past and said “your the guys who came 200 miles
to turn on a switch right?” And I instantly replied “nah, we just heard there
was a scouser doing some work so we had to come look!” the techie I was with
looked like he was gonna shit himself but the other guys around fell about
laughing.

The thing with work sites is you can’t take any shit but you won’t find
funnier people either.

~~~
Baeocystin
Sometimes I really do miss my old welding job in the shipyards, and the fun of
the people I was working with is a big part of it.

(It was also the most meritocratic job I've ever had- your ability, or lack
thereof, was on display for everyone to see. And working on teams where
everyone is competent is a joy, virtually regardless of actual task.)

------
ken
> I believe that many in the field are overpaid relative to the difficulty of
> the work they do.

I believe that _every_ programmer is overpaid relative to the difficulty of
the work they do -- or, more accurately, that The Market doesn't pay based on
_difficulty_ of work. Software pays so well because the product scales so
well.

Successful pop musicians don't do work that's 100 times more 'difficult' than
software engineering. They do work that scales even better (copying digital
music; playing to arenas; branding on merch).

Having worked outside the software world, I absolutely do not believe that
software people are any smarter, on average, than anyone else. Ever see a
plumbing or electrical or structural system fail spectacularly? Other types of
workers absolutely need to understand interactions between multiple complex
systems, deal with obsolete and incompatible systems, and deal with changing
and conflicting requirements and regulations.

If software is any more complex to deal with than physical systems, it's only
because the architects and implementors let it get that way.

~~~
ellius
I'm reading Thomas Sowell's "Intellectuals and Society," and while I disagree
with a lot in his arguments and his view of the world, I think he nails a key
point about the philosophy behind free market economies. He says that
conservative intellectuals don't view inequality and other social tragedies as
mainly the consequence of policy, but rather as an inherent part of the human
condition. Markets may act as a mechanism that _conveys_ those inequalities,
but they are not necessarily their cause. Prices are his first example.
Commanding a higher salary doesn't mean you're a harder or even more skilled
worker, it just means that you produce something that is judged by buyers
(employers) to have a higher positive impact on their well-being.

~~~
jadedhacker
This is far too celebratory of markets. Markets pay for things that people of
means want. If I make a useless toy that delights a rich person I am paid
lavishly. If I make a useless toy that delights a poor person I get next to
nothing.

~~~
pdonis
_> Markets pay for things that people of means want._

No, markets pay for things that are worth more than it costs to make them.
Only a very small fraction of those things are useless toys that delight rich
people.

~~~
bobthepanda
This is a bit of hyperbole; desire and need are direct contributors to "worth"
because you can extract more profit for it.

That being said, profit is a notoriously crude way of measuring utility, given
that a lot of tragedy-of-the-commons type problems don't have "profitable"
solutions without some type of government intervention. (See: pollution,
public education and health, childcare, working week limits, etc.)

~~~
pdonis
_> profit is a notoriously crude way of measuring utility_

There are no non-crude ways of measuring utility among different people with
different, often incompatible, preferences.

 _> a lot of tragedy-of-the-commons type problems don't have "profitable"
solutions without some type of government intervention._

They also don't have good solutions _with_ government intervention, because
the downsides of government intervention (regulatory capture, increasing power
of the state, micromanagement of all aspects of people's lives, political
power being used to benefit some instead of all) more than outweigh any
increased benefit in a particular area.

------
laogai
To use the plumbing analogy, even if most or all of the job will be
maintenance, you still want engineers who knows the internals enough to build
their own “plumbing” if/when needed. Maybe an apt comparison is how astronauts
are trained heavily for the 1% of off script situations that might occur even
if it doesn’t happen in the span of their careers.

Companies should feel able to innovate when appropriate by having hired
engineers capable of doing so.

~~~
wmccullough
I’m not fully willing to accept the plumbing analogy myself. While I do agree
that most of our work is CRUD over REST, I think there’s another element of
skill that isn’t uncovered in the article. If we are humble plumbers, why does
it take me six months to find engineers that understand basic concepts like
cohesion and loose coupling beyond a text book level (and I live in a
metropolitan area with several software schools!)?

Perhaps a more apt analogy is that we are more like home renovation
specialists. If things go well, we can update your plumbing to get away from
lead pipes, but if we have to route the new plumbing through a joist, we are
going to have to educate you on why we should do something a certain way. If
we find a water leak and rot that will put your home (business) as risk, count
on us to tell you about the risk and let you decide if you want to address it.

I’d argue that we are simply doing plumbing when things are going really well,
perhaps even when the house is small and was well built to start with. If
plumbing was all we did, I’d argue that our field would be way more saturated
than it is.

~~~
FLUX-YOU
>why does it take me six months to find engineers that understand basic
concepts like cohesion and loose coupling beyond a text book level

If it's so basic, just hire them and take a day to teach them those things.

WTF is with this trend of waiting 6 months for a hire vs. spending a day to
fix the holes in someone's knowledge.

~~~
icebraining
Because of Brooks's law. When you're already working overtime, you don't
_have_ a day to fix those holes.

~~~
laurentl
And yet you can afford to wait 6 months to find someone who doesn’t have these
holes?

I’d wager actively recruiting for 6 months takes a lot more effort than
training someone for one day, for that matter.

As a manager, I find it more interesting to develop someone’s skills than hire
a rockstar (and deal with ego issues). It’s more work, and you can’t always
afford to do it, but at the end of the day you’ve contributed to fixing the
long-term problem by adding one more resource to the talent pool. And it’s way
more rewarding, and arguably one of the best uses of your time as a manager.

~~~
mmt
> And it’s way more rewarding, and arguably one of the best uses of your time
> as a manager.

I'd say it's also one of the best uses of the time of the more senior
engineers who will act as (additional) teachers/mentors.

Having been in the position of needing to explain a concept or even just "show
my work" has helped me not just better understand what I'm doing, but
sometimes even catch fundamental mistakes in my thinking.

It's also why I like commenting on technical matters on HN. Sometimes forcing
myself to articulate my thought to someone else will reveal the absurdity of a
long-held assumption, or something similar.

------
inertiatic
>"This isn’t a popular opinion among software engineers, but I believe that
many in the field are overpaid relative to the difficulty of the work they
do."

But no one is paid relative to the difficulty of their job, dummy!

~~~
TeMPOraL
Or actual importance.

It's always a combination of what salary you can command (function of your
soft skills) and how much money you can make the person paying you (function
of market dynamics).

~~~
petra
Good points, but it's also about how much would your replacement cost.

------
Animats
Overcomplication is sometimes an issue. Remember Soylent, the hipster's
Ensure? They were always talking about their overdesigned server
infrastructure, even though you could calculate from their sales volume that
they did only a few transactions per minute. They could have used low-end
shared hosting and CGI to handle that. But no, they had to be a "tech"
company. (It looks like they've finally accepted that they're in the food
business and are selling by the pallet to WalMart.)

Plumbing has standard ways of doing things, and standard parts that rarely
change. Web services need to settle down like that. Really, few sites are
doing anything very interesting.

~~~
overkalix
There can't be a similar process of standardization if there is no cooperation
between supply actors, and there will not be cooperation if the market is
completely open. There is a tradeoff between scalability and competitiveness.
If your product can easily penetrate a market without any more than a few
tweaks it means it is also easily replaceable, and you'll be competing through
non-functional aspects.

------
mborch
He wants to make that case that software development is a more pedestrian
affair than it might seem, without much of an argument. But it's not true.

Even a small shop needs to live up to pretty much a global standard, not least
in terms of security. In the EU you must engineer with a security perspective
in mind. The field is constantly moving. People come and go faster than most
other industries.

I think there's an analogy to getting a better bike: it doesn't get easier,
you just go faster.

~~~
qaq
Working at security related company even Fortune 500 are not really living up
to "global standard" in terms of security. Smaller shops can't afford proper
security solutions and def. can't afford a competent SOC.

------
tootie
For most projects I've worked on in the past 10 years, the energy has majorly
shifted towards front-end development. Building blocks have improved
dramatically, but expectations have been rising even faster.

Backends are getting more and more homogenized. There's still a fair amount of
work that goes into accurately implementing business rules and a much bigger
focus on quality and maintainability than ever before. But there's not a ton
of serious thinking involved. I can pretty much follow a script for
implementing microservices that will scale up to millions of users. I somehow
ended up with a high-paying enterprise architect job and when they ask how we
can build a system that will have 1000 users, the answer is that you can do
pretty much whatever the hell you want so long as the framework is less than 5
years old and it will be fine. Then I go get a coffee.

~~~
StaticRedux
Not to be snarky, but this sounds like the viewpoint of a front end developer.
One I disagree with. It would be like someone saying front end developers are
unnecessary bc anyone can spin up a WordPress site and then go get a coffee.

Can you get a backend going for a thousand users fairly quickly and easily?
Sure. But that's the WordPress equivalent of describing back end development.
If you have any kind of process based workflow with varying parameters,
integration with hardware, detailed reporting / analytics, hell, if you even
deal with money or dates/ times / timezones, which is any non-trivial
application, backend development gets real complex real fast.

It may be trivial to spin up a WordPress site, but it isn't too put together a
solid, reliable React / Redux PWA with isomorphic rendering and offline
support. It may be trivial to serve a few API endpoints straight out of the
database, but it isn't when you have workflows and processes and states and
have to deal with shit like GDPR.

> But there's not a ton of serious thinking involved.

Give a fucking break. The only way this could ever be true is if you only have
experience working on extremely trivial applications with no complex
functionality.

~~~
tootie
I'm strictly backend and I've myself becoming less and less useful. Complex
requirements are one thing, but implementations just require diligence, not
aptitude. I've built loads of apps and sites. Never needed a data structure or
algo that wasn't in a standard lib. Most complexity is in integration. Most of
my co-workers have no formal CS education and are at no obvious disadvantage.

~~~
learc83
>Never needed a data structure or algo that wasn't in the standard lib.

Either your definition of algorithm is absurdly narrow, or you are working on
some very simplistic problems.

~~~
tootie
Little bit of both. But essentially the problems are getting simpler. All
sorts of problems that used to occupy a lot of my time early in my career are
now effectively "solved" and just need a library or SaaS to plug and play.
CMS, e-comm, analytics, CRM are all pretty push-button at this point. Maybe
some API calls to be made. I remember once needing 3 people spending a month
trying to get SSL certs installed on a load balancer. Now it's a checkbox.

~~~
learc83
>But essentially the problems are getting simpler.

The same could be said about you and someone who started their career in the
late 60s. You probably didn't have to write your own OS.

If you were building software that businesses expected in the 60s in the 80s,
then your job in the 80s would have been a lot simpler than it was in the 60s.
And If you are building the software now that businesses expected 20 years
ago, then yes your job is much simpler.

That's generally not how it works though. As capabilities increase,
expectations increase, and many times expectations increase even faster than
capabilities.

~~~
tootie
Well, yeah, but my point is basically that work is being pushed up the stack,
hence all the energy going into UI development. The biggest challenge for
backend devs is usually API Design.

~~~
learc83
The biggest growth in expectations I've seen are in UI, Analytics, and
Security/Data protection. 2 of those are primarily backend, and they aren't
something you can just plug in unless you are solving very trivial problems.

And pulling requirements out of business people who don't really know what
they want, then turning them into sensible data models that can produce
reliable and actionable business intelligence is still where I spend most of
my time.

------
funfunfunction
This article has an interesting point. I think the corollary to the point made
is that web software, which is what is providing most of the plumber-type jobs
this article references, has reached a point of maturation where certain
solutions have been accepted as best practices. This lowers costs for
businesses and lowers the barrier to entry of the web software industry, which
I think of as generally good, but it also cements the positions of certain
organizations that are foundational to the infrastructure on which these best
practices are built.

The saddest part about this for me is the realization that there is little
room left for truly innovated web technologies that have real impact to
businesses. Of course there are counter-examples to this (I would count
WebRTC, WebSub, and a few others as having potential to impact some usecase
specific best practices) but the majority of new software I see being hyped is
either a new implementation of an old idea (React and Vue for FRP on the front
end) or something touting decentralizing as the next major step which so far
hasn't proved itself valuable in a commercial sense.

~~~
AznHisoka
It also doesnt help that building valuable software these days requires _lots
of data_ , especially if you are doing any machine learning.

And most startups lack any data. its all owned by the monopolies of Google,
facebook, Amazon and Apple. So the rich will just get richer, and the chances
of startups succeeding in any field is as low as ever.

------
notacoward
Makes me glad I work in a specialty (data storage) which provides the parts
and tools the plumbers need. Sure, there's a shared grab-bag of techniques
etc. that we all use, but it's far from plug and play. While the market for
people like me will never grow anywhere near as big as the front-end plumbing
stuff, demand and pay have been much more consistently good for the last
couple of decades and are likely to remain so for a while.

My advice to new software engineers is to get into one of the more "boring"
specialties. You might not be in the lottery for the really big bucks, but you
probably weren't going to win anyway. There will still be plenty of
interesting and fun challenges, with greater stability and (TBH IMO) more
mature coworkers.

~~~
jaiprabhu
Genuinely curious - can you give me some examples of these specialities where
one can go deeper? In am in this boat now where I have worked on pretty much
on all layers of the software stack and I want to now narrow down my focus.

~~~
notacoward
Some of these map to classes you might have taken in college - compilers,
databases, AI, etc. I'd divide operating systems into more than one part:

* Core kernel (schedulers, memory management, virtualization) * Drivers * Filesystems * Networking

Then there are some specialties that cut across more than one of these - high
performance computing, financial tech (not even counting blockchain stuff),
security engineering, performance engineering. Then there's a whole world of
embedded stuff, from teeny-tiny microcontrollers up to high-performance
medical imaging or genomics, plus large distributed systems to control things
like military equipment or nuclear power plants.

Really, there are so many different kinds of computing that we just never even
hear of on a site like this. It's really the web/app/mobile front-end stuff
that's a niche technology-wise. (On my less charitable days I'd call it a
ghetto.) It's a hugely profitable niche, to be sure, but to me it still seems
to get a disproportionate share of attention and funding compared to other
stuff that's just as hard, just as fun, and just as important.

------
segmondy
This is a good way to offend most developers since most like to see themselves
as rocket scientists instead of plumbers. But I have to agree, it's MOSTLY
plumbing. Thus the reason we see people argue, "Don't make me leetcode during
an interview bro! Who ever implements their own algorithm these days?"

~~~
IshKebab
I asked that during my last interview but it turns out the job actually _is_ a
lot of new algorithm development, which is fun and challenging.

~~~
Apocryphon
Lucky! What kind of role is it?

~~~
IshKebab
Helping to write a sort of compiler for a new chip architecture.

The previous job I had (I've only had two programming jobs - used to do
engineering) also wasn't CRUD - it wasn't as algorithmic but it did involve
lots of low level USB programming which was interesting.

Are non-CRUD jobs really that rare? Surely if you don't want a CRUD job just
find a company that wouldn't need to write any...

------
lgleason
Beyond the ludicrous nature of the authors sttatement, a look at his Linkedin
shows that he is a non-technical CTO. IE: A CTO who has has never really
worked as an engineer for an extended period of time and din't come from that
background. Thus, consider the source and move on...

------
js8
I think the article is quite right, and David Graeber made a similar
observation when he talks about bullshit jobs (companies would rather hire
someone to tapeduct the system rather than redesign it).

But I feel part of the problem is that the companies want to rather hire a
plumber (I mean, software engineer) than to buy an off-the-shelf solution. It
seems to me that off-the-shelf solutions are very costly and many companies
think they can do it cheaper if they build it themselves. Which to me sounds
like that the market is not functioning properly, for whatever reason.

~~~
dyeje
This is exactly the trap companies fall into. They see a solution ready to buy
and think, "Oh we can build that ourselves for cheaper." The thing is, they
can't. I worked in a niche of the medical industry and we saw this so much we
made a white paper on it. The number of times people decided to build
something themselves and came back to us in a year was amusing. They don't
realize:

1\. The amount of man hours necessary to build a reliable, effective system.

2\. The expertise and insight needed to make it actually solve the problem.

3\. The ongoing costs of maintenance, adding features, and support.

So, in my opinion, it's not that the market isn't functioning it's just that
most companies are delusional about what it costs to build a complex software
tool.

~~~
pbecotte
Well...yes and no. For example, page scraping is something I have been pricing
recently. The companies I have spoken to cost WAY more than it would cost me
to build...because I need one simple solution with limited accuracy...but
their company is built around an advanced high accuracy solution. The off the
shelf thing is rarely the simplest thing possible, because once they built
that they had to keep adding features and engineers and costs :)

------
Daishiman
"Just Plumbing" is not really an apt analogy.

Because, judging from all the people I've been working with throughout the
years, it's _amazingly_ difficult to write a clean, cohesive, maintainable,
scalable CRUD app that will actually satisfy your customers' demands.

Making things that work is easy. Making things that work _well_ , that's
another story. And it's the reason why there's so many open spots and high
salaries for senior web devs who can deliver.

~~~
stevehiehn
Installing a toilet is easy. Planning/creating the infastructure for a large
building complex is another story :)

------
sonnyblarney
"The Bulk of Software Engineering in 1XXX is Just Plumbing"

When was it not?

What were these guys thinking?

The vast majority of software is taking real world applications and dealing
with the 'kind of a mess' of situations that can arise.

Even login/credentials can be a minor pain.

Doing it clearly and succinctly is the challenge.

Though it's often more 'solving problems' than 'building something' we should
maybe think of ourselves more as construction workers than research
scientists.

Upon graduation my skills were filled with so many holes I suggest that every
CS grad takes a 3-month long 'power course' wherein the get all the basics,
best practices and common tooling down.

------
nobody271
The writing has been on the wall for at least a decade. I remember hearing
about how programmers would be more like plumbers all the way back in 2008.
Sure enough, that really has come true for a lot of developers.

But I agree that this is good. The job is just changing. For example, maybe
you wrote a custom DAL back in 2010. Now what you likely have is a mess that
you have to pay several people a senior salary for because they are the only
ones who understand how it works. Meanwhile a general purpose plug and play
solution like entity framework is ubiquitously understood among .net
developers. So now we don't need those "senior" DAL guys simply because they
understand the mess they helped create a decade ago (also, about the last time
they read any books).

I see it as more a levels of software development thing. low-level > medium-
level > high-level > plumbing.

------
amelius
Yeah, in a way it is the curse of "open source": there are so many libraries
and tools freely available that the only problem left is to tie them together.

------
matz1
I have no problem when getting paid engineer job when I'm just doing plumbing
:-) but the problem is when I go look for a job or interview, there is
literally no job ad for software plumber so I have to pretend to be engineer.

------
zelos
I constantly have this feeling: software development as a job, relative to the
amount it pays, just feels ridiculously easy most of the time. Churn out some
API that takes data from one place and exposes it in a slightly different form
over here, write some unit tests and docs, done.

Then other days I'll spend an hour doing nothing but thinking, staring out the
window, trying to figure out the best way to do solve some complex multi-
threaded issue, before writing any code, and I feel like a real engineer
again.

------
falcolas
I have to be honest - I don't see this. Maybe I'm lucky.

My current company is basically creating Microsoft Office on the web - because
for the targeted niche, Microsoft Office (Google sheets, Libre office) wasn't
cutting it. There aren't libraries for that.

My previous company was applying machine learning to the twitter firehose, and
present the results in real time. The existing libraries weren't up to the
task (or the volume).

Before that, we built tooling to allow ten people to administrate 40 plus
companies' on-premise databases. There were a few libraries and tools we used,
but nothing that we could just "plumb together".

I saw in the Reddit thread that the "plumbing" argument applies mostly to
backend developers who just put REST libraries on top of databases. OK, I can
kinda see that, but it's not like the complexity of the programs has
disappeared, it was just _moved_ to the frontend. And in a way, that's a
benefit to the company (and the employees) - the resources previously tied up
in re-creating REST libraries before REST libraries existed can now do other
things.

The value of programmers has never been in the movement of data from point A
to point B, it's been in the application of business logic to that data.

~~~
drb91
> There aren't libraries for that.

Sure there is—textarea and http. The rest is just plumbing.

I’m only half-joking. I just don’t see a huge difference in difficulty between
business logic and plumbing—they’re two sides of laying out the program, and
neither one is more difficult than the other. Both lead to complexity and
confusion if you lay out poorly.

I think that if you are good enough at googling, the vast majority of problems
businesses are solving have already been solved (or the problems are well-
mapped) and you just need to adapt the successful patterns to your domain.

------
ramoz
If devs see their jobs as "plumbing" then that is a problem with their current
culture & environment.

Most devs should be focused on delivering business/mission value. This means
custom code, unique ip, algos, GUIs, etc.

However, the current state of devops has thrown all sorts of engineering
challenges at devs that lose sight of the actual value that should be
continuously delivered. It becomes plumbing when we get caught up in redundant
tasks such as dependency management, tests, logging, containers, runtimes,
integrations, cloud provider specifics for scaling/ha/sre, and connecting them
all to create "the platform" that is supposed to enable true continuous
delivery of value. We end up more focused on this internal platform
development even though the tooling is pretty advanced. We still have to
configure/orchestrate/manage that advanced tooling, and this, in turn, creates
the plumping effect. We greatly underestimate the work involved with all of
these things.

It is not plumbing when we are focused solely on the continuous delivery of
value. We remain lean and agile, constantly testing a hypothesis and
developing our "uniqueness".

~~~
matte_black
Very few developers deliver direct business value or revenue.

For every rockstar dev who gets paid a lot of money to make unique solutions
and use his brain, there’s dozens of unsung heroes who do the grunt work
reading from docs and setting up plumbing so the rockstar can deliver his
value effectively.

These developers are not working in cool languages or solving new problems,
and many of them will never do so in their life. They are just expendable
workers swinging hammers and writing JavaScript.

~~~
ramoz
But isn't it a problem that most devs are setting up the plumbing vs already
being enabled to deliver rockstar/valuable solutions?

i.e I've worked at a large enterprise and we all (product devs & ops) became
so focused on our grand platform (continuous delivery, heavy integration with
AWS, container orch, all using OSS) that product development stalled. Though
we were doing "rockstar" things with cool tech and our resumes have a bunch of
cool words, but it's true that most of that work was plumbing. In the end that
tooling became a b __ch to manage and continually developing upon. The product
continues to stall.

Wouldn't it be better for all if we weren't so caught up in the plumbing of
these advance/cool tools? -- that's the thought that actually leads into
something like building your own platform, vs something provided. Ops then
become more like an overseer sre type of role, and most engineering shifts
back to direct value.

------
InclinedPlane
> _" This isn’t a popular opinion among software engineers, but I believe that
> many in the field are overpaid relative to the difficulty of the work they
> do."_

This is straight up classism, period. Software devs want to pretend that their
high (or even just decent) salaries are due to membership in an elite, highly
skilled class. Software development isn't some brain dead job like burger
flipping or some gross blue collar job like, ugh, plumbing, right? And there
you see the classism raise its ugly head. The denigration of entire job
fields. Forcing a separation between "true" developers and other workers also
requires denigrating those other workers, how else can "true" developers be
raised up higher without pushing lesser workers lower?

A lot of engineering is "just" plumbing. So what? A lot of real-engineering is
also "just" plumbing (or the equivalent). So what? Does that mean it's
valueless? Does that mean people who aren't in the elite should be treated
like second class citizens or that they shouldn't make a decent living for
their work? The truth is that people who do "just" plumbing work (whether in
software development, in engineering, or actual plumbers) still do valuable,
high quality work. Still do skilled work (not that that even matters or is a
meaningful term). And still deserve to be compensated well for that work.

Let's stop shitting on burger flippers and plumbers, let's stop trying to
define hard class boundaries between some hypothetical elite of "true" highly
skilled knowledge workers and everyone else. Let's stop calling for people to
be paid less for their work, almost everyone (even software devs) are being
underpaid today. It's unhelpful, it's divisive, it's prejudicial, and it's
just mean.

------
matchagaucho
Kind of a cyclical theme... but this is why the job titles of "Developer" and
"Engineer" were created.

"Engineers" design/build components that perform with repeatable results.

"Developers" work out the plumbing details between components in a system.

When hiring, don't mislead people into thinking they're interviewing for an
Engineering role when you actually need a Developer.

~~~
jxub
I think people need to stop seeing software as punching coloured characters on
a screen and start considering it as an occupation that varies widely in
skills, wages, and status and distinguish between the blue collar "plumbing"
and white collar "architecting" parts, with a smooth array of options between
both.

This, of course, is hard if you don't program and it's not really meaningful
to evangelise about, so I'll leave off this rant here.

~~~
matchagaucho
For a recent home remodeling project I paid an Architect $90 hr and the
Plumber $130.

There's nothing diminishing about being a Plumber.

"Software is eating the world" means there is more demand for _developing_
systems than designing them.

~~~
jxub
Yes, I guess developing/architecting are largely orthogonal to paycheck, it's
more about the current market forces than the position in this scale.

However, our lizard brain status system still associates directing and leading
things with more status than just requirements implementation and there is no
way around that yet. Also, less demand means that this more "abstract thought"
type of positions are more prestigious by the virtue of scarcity.

------
_bxg1
One of the greatest personal flaws your average engineer has (I can say this
because I fall in this category), is the constant desire to do more
engineering. We have a need to make newer, cleverer stuff ad infinitum,
whether or not it's needed, whether or not it's even useful. This is why there
are so many frameworks, this is why people make their systems scalable before
they need to be and bloated beyond their use case. We like shiny toys and we
like to use them to make more shiny toys. It's hard to be someone with that
mentality when most of the major problems in your field are pretty much
already solved.

------
quotemstr
The transformation of the software profession into law is progressing. As we
engineer most moderate-difficulty tasks away, what remains is the trivial or
the challenging, and these two classes of task correspond to the peaks of the
new bimodal salary distribution. We have a glut of people in the field capable
only of the former class of task, so that peak is going to rather low.

This dynamic isn't speculative. It's _exactly_ what happened to law. It's what
happens to any field that goes through a boom and attracts a certain kind of
person who's more motivated by immediate financial reward than talent or
passion.

~~~
arwhatever
What exactly did happen in the field of law? Split into lawyers / paralegals?

------
mongol
The bulk of IT is certainly data plumbing. The question is what the
engineering part is of the total. Compare with physcial logistics, you have
the lorry drivers, the mechanics, and the engineers that develop the next
lorry model. There will always be a need for tools and algorithms to work on
the data in innovative ways. That is probably where the engineering skills are
the most essential. But these jobs are likely fewer than there are people
wanting to think of themselves as software engineers.

------
arcticbull
100%, we're Dell - we grab the bits from the different manufacturers, design a
motherboard that connects it all in a way that ideally keeps them from falling
down unexpectedly. Sometimes we make a custom doohickey. Probably not though.
And if we do, maybe we shouldn't. That is, until you get to a certain scale.
As you get bigger and bigger, it makes more and more sense to roll your own of
everything to squeeze that last little bit of situational performance.

------
danharaj
If what we do is plumbing, then what we're pumping is salt water. Some of us
get to build better pipes.

Software in 2018 is better built than in 2008 for a lot of sectors. In 2028 I
expect our pipes to be more robust. I also expect a single engineer to be able
to do the plumbing 5 developers do now. At least with the right technologies.

------
analog31
After reading a number of comments...

I wonder if we in the tech sector tend to romanticize the trades. I don't
necessarily disagree, but would only offer a caution: _Look at their hands._ I
had a new screen door installed on my house, ordered it through Home Depot,
and they contracted it to a father-son business. The father was not much older
than me (mid 50s), but was broken and hobbling.

There's also a widespread view that we will always need tradespeople, e.g., to
build houses. But the trades are being replaced by automation in subtle ways
that could eat away at their livelihoods bit by bit. Examples include plumbing
fittings where you just cut a plastic tube and squeeze it onto a fitting,
rather than sweating it together with a torch. Another is the ability to
design and order a "custom" house that's built in a factory and assembled on
site.

~~~
notacoward
I think it's a form of overcompensation. We all _know_ deep down that we're
pampered and privileged and frankly not contributing as much to the world as
we're taking out of it. Thank you, supply and demand and barely-free-enough
markets! But instead of just accepting that and making our peace with it, many
of us try to make up for it with our choice of sports, hobbies, or
accoutrements. There are way more techies obsessed with martial arts or
brewing or sports cars than even other demographic factors such as income
would predict.

I think it's the same with trade and craft knowledge. Any techie with any
exposure at all to the more traditionally "manly" trades (I'd include things
like military/police work or firefighting here) tends to flaunt it as a way to
show that they're more worldly and "authentic" than all those other techies
who have never known any other life. I used to do it myself, except that my
"trade" was the simple one of actually being poor. Then I realized that it was
just another kind of virtue signaling, and it lost its appeal.

------
yakshaving_jgt
Most people who buy SUVs don't drive off-road. Most people who buy diving
watches don't dive. They do expect those things to perform if pushed to those
lengths though.

It's possible the same mentality carries through when hiring programmers. I'm
not sure it's a problem either.

------
thegabez
If software started to become as reliable as PVC piping and standardized and
regulated by a governing body like it is with plumbing, then all your problems
are solved. Until then, keep learning and make sure your engineers aren't
over-engineering the crap out of everything.

------
adim86
I don't feel the author is belittling plumbers or is saying software
engineering in 2018 is less glamorous. I think the point he was getting at is
that lots of software engineering recruiters and employees treat software like
we are making the next PageRank Algorithm or inventing the next rocket. He is
pointing out that today lots of what we do is using the already built software
to create new things. This is ok, this is still hard, this is still worth a
lot but we need to work on our perspective, both for hiring and also for job
expectation.

------
chiefalchemist
But it's not just tech / engineer hiring that has this problem. Add on a
manager that has peter principle'd out and there are plenty of opportunity to
talent mismatches.

That aside, the truth is, the vast majority of what's asked to be done has
been done before. Even the particlars are likely to be wheel reinvention.

Mind you, it hasn't happen yet, but an object driven drag & drop (or VR)
driven solution / application builder is only a matter of time.

Splash on some AI and the role of engineers not in the top percentile is going
to trend to zero

~~~
chrisco255
Visual drag and drop application builders have existed for some time. The
problem is that as soon as your app needs to go "off rails" which all apps do
at some point, you won't have the skills or knowledge to tweak your program
with some visual editor tool. And I don't see AI doing anything remotely close
to solving for that. You would need to be able to communicate with an AI on a
deep level of understanding of business requirements and the AI would also
need to understand all the layers of the software stack to be able to produce
code with AI that even rivals average individuals. That's something akin to
general AI and we are a long ways off if it's even possible.

~~~
chiefalchemist
Yes. There have been attempts.

I see AI as being the collection and resoltion tool. Imaging GitHub with a
brain. The current problem is far too much wheel reinvention. Far too much
language specific solutions when - ultimately - the language doesn't matter.

------
danesparza
So for application development, I tend to agree. Interviewers should focus
less on computer science questions and more on application development
questions.

But ... with advances in AI / Machine learning, drone tech, and self-driving
vehicles, I think there is a growing niche in software development for more
serious science / data analysis. Investor expectations are being set more and
more by this niche -- and causing regular application developers to have to
push the envelope.

------
whatshisface
Isn't this a description of what IT people and operators are supposed to do?
"A bunch of services, set up with a little glue and a lot of configuration,"
is exactly what ops is about. Customarily, programmers run the show and tell
the operators what's needed, but perhaps for a small business it would be best
for an operator to act as "software concierge," arranging for new software to
be engineered only when necessary or helpful.

~~~
mmt
The "concierge" idea is a new one that I hadn't heard before. It helps
describe a piece that I feel is missing from what it is I feel that I _do_ ,
because, so often, perhaps because the terms "operations" or "system
administrator" don't have the word "software" in them imply that I don't
understand it or can't apply engineering to it.

Although I don't feel comfortable going as far as dictating the outcome of
some version of the build vs buy decision, I agree that, when I happen to be
on staff, tremendous value is lost by not at least consulting me as to what
"buy" [1] alternatives are available. As an experienced technology (not just
software!) concierge, I often know even more than one.

Unfortunately, by the time I'm hired, it's almost always too late. Especially
at modern startups, programmers very much run the show, so the focus is very
much on writing new software.

That said, I don't think this notion is unique to ops engineers. It can be
extended to programmers, as well, in that very senior ones tend to follow
similar models, of simpicity, code reuse, libraries, and, ultimately,
eschewing actually writing (especially rewriting) new software unless
necessary.

[1] quotes because it's often free-gratis, merely not custom-written there

------
snarfybarfy
I once wrote a class called SuperMario that did some plumbing between the
database and the HTML template and people didn't think it was funny at all.

Contrary to a lot of comments, I still think the word "plumbing" fits well
with connecting specific columns in the DB with specific holes in the
template.

Like in real plumbing, you get shit when you connect the wrong things:-)

------
et-al
_" Show me your flowchart and conceal your tables, and I shall continue to be
mystified. Show me your tables, and I won't usually need your flowchart; it'll
be obvious."_ \- Fred Brooks

Dug up this old HN thread about Data First if anyone else is interested:
[https://news.ycombinator.com/item?id=10291688](https://news.ycombinator.com/item?id=10291688)

------
_bxg1
Right now anybody working in software (outside of startups) makes an extremely
comfortable living. I think we're headed for a bubble-burst as the job market
approaches saturation. The "bootcamps" are so successful because you don't
have to be a genius to do the plumbing, but you get paid as if you are one.
I've considered going back for a master's degree for this reason alone.

------
auvrw
i frequently compare workaday web-dev to general contracting (a favorable
description from my perspective). no issues with the article title.

here's my issue with the article _text_ : it describes

    
    
        I came to the Graide Network and immediately started breaking our legacy app into microservices.
    

as "overengineering," and perhaps it was ... or perhaps it was just not-great
planning. after all, the premise of the article is that the microservices are
now incurring costs that can be described more like plumbing than engineering.

here's where some of my experience partially aligns with the article's
complaints

    
    
        Investors are guilty of pushing it on us
    

this has happened to CEOs at startups i've been at: [some] investors asked for
diagrams, etc., and _we_ made decisions to build invisible fanciness instead
of product for users.

here's where the author begins upon the path to enlightenment

    
    
        we can build more with less

------
xkcdefgh
My employer used to have devs from service based companies working on
contracts. Later they started cutting the amount of vendors and hiring good
talent with attractive wages. The amount of stupid code I encountered when I
joined was ridiculous, now that we have smarter engineers over here,
everything works like a breeze.

------
tomohawk
If you want to be better at building software, pick up one or more of the
building trades. There is much to learn about building things from
electricians, plumbers, carpenters, and masons. Working with tradesmen and
seeing how teams of tradesmen work can be enlightening.

I can't believe this guy said "just plumbing".

------
simplecomplex
Bullshit. Don’t trivialize software development or plumbing. Even a blog is
not simple at all. Case in point: there’s numerous bugs on his own website!
For example, on my mobile browser his “subscribe” button text is cutoff and
misaligned from poor CSS work. But it’s “just plumbing” right?

------
jorblumesea
Engineers are paid to solve business problems, not just "code things" and
"create products". I am always perplexed as to how engineers see their own
field. You are not there to create some novel framework, unless the situation
absolutely calls for it.

------
focal-point
"Just plumbing" which requires at times detailed detective work and/or the
ability to prove that many invariants hold, all while communicating
effectively with several people of different backgrounds.

------
matz1
Funny, I actually been using plumbing analogy all this time when talking to
non tech people. Many people think too highly of what I actually did. I too
feel like using the word engineer is sugar coating it.

~~~
Apocryphon
The internet is a series of tubes meme aside, I've felt that a lot of modern
engineering, at least tying together REST APIs with web/mobile GUIs to do
CRUD, is really just data plumbing. We're just packaging and unpacking blobs
of information. That's it.

And the difficulty involved is that we're not yet smart enough to build
efficient, easily-adopted systems that use natural language and don't contain
a million dependencies and hacky workarounds.

------
plesiv
Blunt generalization incoming... it's naive to read this article (similar to
the majority of others) as an info-piece. It's a marketing piece, any
informational value is a by-product.

------
greenhatman
>other engineers insisted they overbuild things such that junior devs will
take a long time to understand it

Anywhere that I've worked this will get your PRs or architectural ideals
rejected.

------
kpwagner
These comments crack me up. Hacker News readers seem to have a lot of opinions
about plumbing, the plumbing trade, the difficulty of plumbing tasks both
intellectually and physically, pay scales of plumbers, personal experiences
with plumbers, etc.

Did you get that "plumbing" was just, like, an analogy? The article wasn't
really about plumbing.

~~~
RichardCA
It resonates for a reason. In plumbing everyone understands the difference
between what's theoretical (flow rate equations, hydrodynamics, metallurgy)
and what's practical (nearly everything else).

Computer science has always had this tension between being an actual "hard"
science and an applied engineering discipline. A lot of otherwise intelligent
people pretend to understand that distinction when they really don't and it
leads to no end of problems.

~~~
wgerard
> Computer science has always had this tension between being an actual "hard"
> science and an applied engineering discipline

I actually would go much further: I would say that the tension is between an
applied engineering discipline and a vocational degree ala, well, plumbing.

I don't think there's much question that fields like quantum computing belong
more in the "applied engineering" category (or even the "hard science"
category). There's much more question about whether professional software
development has any significant relation to the field of Computer Science
other than the bare fundamentals.

My hunch in the future is that we'll see the field bifurcate into two very
different professions, ala EE and electricians: "Computer Engineering" (or
whatever it's called) will become a much more limited field that is overseen
by the NSPE and has a licensure system, and "Software Development" will become
a vocational degree (if that, even) and represent the vast majority of
software development jobs we see today.

That obviously has problems, but I see it as an inevitable compromise that
might eventually help resolve questions like, "this set of customer personal
data was leaked because of a known but unpatched vulnerability, who signed off
on this software release?" and also relives a lot of the pressure on regular
software developers to understand these fundamental CS concepts that don't
come up much in practice.

------
booleandilemma
Be careful with the word “just”. I’ve found that people who use it often are
blissfully unaware of the details of things and the work involved.

------
stevehiehn
I find this article interesting largely because it appears to tease out class
& elitism.

------
stakhanov
...that argument has been around for as long as computing itself, and it's no
more true today than it was at any point in history.

If the microcosm you're in is CRUD systems, that argument may be valid, but I
think the author is overgeneralizing if he equates CRUD to the sum total of
everything there is to Software Engineering.

There is a constant lifecycle in this: A new idea is born or first becomes
technologically possible. At that point it's very difficult to put the idea
into reality, but the demand is so pressing that the market will pay whatever
it takes for individuals to live up to the challenge. Then, gradually, it
becomes commoditized and, eventually, everything that's left to do is
"plumbing". ...but at that point, a new idea will have been born or will have
become technologically possible for the first time ELSEWHERE, and the REAL MEN
will have moved on to that (leaving behind only the QUICHE EATERS to do the
plumbing work).

So, if you've been in the field for a long time, your career path might have
looked like:

Writing CRUD systems on IBM host systems in the 80s, when that was still an
intellectually demanding job, because you wouldn't have learned programming at
university as programming wasn't yet a thing, but you still managed to learn
what bytestreams to send down some line to make an IBM terminal show a number
in red or green, and you did it in PL/2, a programming language that hadn't
yet had a chance to learn the lessons learned from several decades of software
systems gone wrong.

Writing some of the first pieces of internet banking software in the 90s. Your
employer, the bank, wanted to make damn sure they were not going to make fools
of themselves by using this new and unproven technology in a way that would
allow funds to be misappropriated. So they paid you handsomely to learn
everything you could about telecommunication technology, security engineering,
encryption, etc because you actually had to manually code all that as there
were no frameworks or libraries for that just yet.

Doing algorithmic trading software in the 00s. Everyone else was ripping
databases out of their systems because XML was the future, and the XML was
processed by a java programme to run in a virtual machine to run in a
container to run in a virtual machine to run on god knows what. Since you
still knew how to do real programming for real machines in a way that didn't
involve converting your data to XML and back, your software executed orders to
buy and sell equity on the world's major exchanges faster than anyone else's,
and you were paid handsomely for it!

Doing data science & data engineering in the 10s. Organizations had been run
for several decades by numbers-driven people whose computing competence only
reached as far as Excel. The numbers-driven mindset did not go away. But the
data was now there in such quantities, and stored in such nonsensical ways,
that everyone who was previously an Excel-guy now needed one or two of those
data engineers at their side. Also, because it now became possible to use the
data to answer some pretty complex questions, those questions were now
seriously being asked for the first time, and everyone discovered that they
had been asleep at university during the lecture where someone explained what
a standard deviation was. So that's why you learned everything you could about
statistics, machine-learning etc etc. You applied it, and you got paid very
well.

So, what really sets you apart if you're not just a software engineer, but a
GOOD and SUCCESSFUL one, is that the stuff you're working on is always the
stuff that's NOT YET plumbing, because it's a major force right here and now
despite the fact that nobody has seen it coming.

...I don't know what the future holds, but I think I'll be well equipped to
deal with it, because THAT's what real software engineers do and there's no
framework to do THAT.

On a side-note: Stack Overflow developer survey results routinely indicate
that like half of the nation of software developers have been writing code for
a ridiculously small number of years (like two? don't remember the number
now). If you're one of them: The above argument does not include you. You're
not a software engineer yet in the sense of the argument that I'm making,
above. But don't feel bad about that. Next year, you'll probably switch to
sales or become the pointy-haired boss of some REAL software engineers.

~~~
kzzzznot
Last paragraph is inflammatory, elitist and rather 'Murica'.

Also what is that opinion based on?

~~~
stakhanov
It follows logically. The contention is "You're not a software engineer yet in
the sense of the argument that I'm making". The "argument I'm making" is that
software engineering is about adapting to change. If you've only been in the
business for two years, no sufficiently meaningful change has happened yet
that you've had to adapt to. I also don't see how it's elitist. Learning to
code (from a book, for example) and sticking with it for more than two years
is rather attainable and hardly the reserve of any elite. I'll have you know
that I have not spent any extended period of my life in America and didn't
know what "murica" even meant until looking it up just now.

------
okket
"just plumbing"

------
learc83
From looking at the Author's linkedin, it doesn't look like he's had much
experience working as a software dev.

Funny, a CTO thinks programmers are overpaid and our work isn't really that
hard. Who would have thought?

~~~
dang
Please don't get personal. Even if you're right—and the odds are you're
not—the damage it does to the container here is greater than any information
it adds.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

~~~
learc83
I don't think pointing out that there is a conflict of interest by the author
of a public marketing piece is getting personal.

If Mark Zuckerberg posts something to the effect of: "guys what we do isn't
that hard, we should probably get paid less", I'd call him out on it too and I
hope other people would as well. I certainly wouldn't consider that a personal
attack.

~~~
dang
The dude wrote a blog post. You're welcome to respond to what he said in the
post. Digging up personal details on the internet to use against him is not
cool. Please don't do it again.

~~~
learc83
I didn't dig up anything. His linkedin profile is the 3rd link at the top of
the post before the content.

There's a signup for a newsletter-- also before the content. The whole blog is
marketing for his CTO advice company.

If someome sets up a marketing page, sells themself as an expert, and then
posts a link to a resume backing up that expertise, it's fair game to click
the link to the resume and verify.

------
dilatedmind
my friend is a pipefitter in chicago, installing piping in a new 60 story
building. plumbers make more than software engineers, when they have work to
do.

