
The creeping IT apocalypse - forrestbrazeal
https://forrestbrazeal.com/2019/01/16/cloud-irregular-the-creeping-it-apocalypse/
======
munin
This kind of change has also impacted me. It shows up when I'm trying to give
students advice about starting their careers, and I realize that the first
jobs I had (system administrator, SOC worker) have been replaced by robots.
Especially in the SOC, I was a "Tier 1" analyst that would do monitoring
(watching a bank of green lights waiting for one to turn red) and first level
triage and analysis. This has been replaced by ML driven data processing
systems.

So I think the apocalypse is double-bladed: while automation kicks a bunch of
current workers out by making them immediately redundant, it also freezes out
the next generation by removing entry level jobs and not really replacing them
with anything equivalent. Meanwhile, universities and vocational ed programs
won't get this memo for another ten years so they will continue to happily
propel waves of students onto a set of closed and locked doors.

~~~
dleslie
What you're describing is _training_; and the act of training employees has
long since gone out of vogue in western markets.

~~~
org3432
I've worked in IT type jobs for over two decades, in some sense training would
help, but to be frank the code and systems IT folks have built are of such low
quality that it's better for everyone if they're handed over to more competent
teams that can cost effectively maintain them long term.

~~~
wyclif
One reason why training is out of fashion is because of poaching. Companies
don't want to invest a lot of money in training their employees only to have
their closest competitors become the beneficiaries. That is a myopic view, to
be sure, but it's a common one.

~~~
org3432
I think they do train people, it's just not formal. And the other issue is the
companies have no clue what their doing most of the time anyway, so you just
end up with IT people on the market will all sorts of nonsense opinions.

------
klodolph
This has been happening since the 1970s. Or earlier. I don’t really think of
it as an apocalypse. IT skills have never had a long shelf life. Any time you
are a technology expert at your company, in IT, the technology landscape will
shift under you. This is the Red Queen Hypothesis in action. People who ran
mainframes in the 1980s became trusted experts and then most of the jobs
evaporated. Same thing happened to people running critical VAX or Unix
systems. Your skills are only valuable as long as the related technology is.

The same thing happens to programming positions.

But I think the good news is missing from this article—IT jobs are, overall,
sticking around or increasing in number. (According to the Bureau of Labor
Statistics, the jobs are growing “faster than average for all occupations”).
You do have to keep updating your skill set, but it’s not like manufacturing,
where efficiencies eliminate jobs altogether or move them to completely
different sectors. And there is that ageism to worry about, and uncertainty.

I’m personally more worried about some of the other remaining white-collar
office jobs, like the accountants, paralegals, HR, various banking positions,
etc.

~~~
Ruxbin1986
There are hundreds of thousands of people or System Administrators that have
made life long careers out of managing networks, server farms, windows and
linux systems since the early 90s. It's not fancy as software development or
drives business value but work that needs to be done.

The Cloud greatly diminishes and in some cases completely eliminates that
work. The only thing left is actual software development.

~~~
3pt14159
If you've tried to hire a good sys admin in North America you'd be shocked at
how high their salaries are getting. Amazon is hiring them by the boatload.
Shopify is moving to all cloud because they're unable to staff.

The problem is that _junior_ sys admins aren't as useful as before to most
startups. I still think they'll figure it out, but the industry is changing.

~~~
geggam
This. Competence is reducing per capita. The market is saturated with IT
people who dont know what they are doing

~~~
dennisgorelik
The average competence is NOT reducing. What is reducing is demand for
mediocre (and low) competence.

This trend results in less competent people flooding the job market.

~~~
softawre
But if demand for low competence people goes down, those people aren't
learning on the job, therefore decreasing the average competence of the
workforce.

------
sarcasmic
I agree with the points raised in the writing, but it mixes automation,
abstraction, and industry consolidation as if they weren't separate processes.
As such, the transformation being described isn't an impending cliff, but an
ever-present pressure of economic forces that affect all business all the
time, and one is wise to watch for.

Automation replaces repetitive work with tooling and work that's more complex.
Abstraction allows one to delegate to another for details, which may include
choosing from a palette of pre-made options. Consolidation will come about as
fewer independent players can sustain themselves in the market. Some will be
out-competed by economies of scale, some will be starved by restrictions on
intellectual property and lack of access to expertise.

This process has already played out for "small business websites", yet there's
still lots and lots of web developers and web designers employed or
freelancing. The current wave of WYSIWYG website generators is actually very
good, and they have add-ons and integrations that make sense for their target
market. But plenty of clients don't want to mess around in it, so they'd
rather hire someone. This could be maker of the generator, or it could be an
outside consultant. In either case, the person brings judgement, experience,
and creativity, to tailor the deliverable to the needs of the client. These
are skills resistant to automation, but not immune to abstraction and
consolidation.

In the end, the antidote is the same as it always was: be adaptable, be
personable, be resilient, and be resourceful. These are especially important
in one is in a comfortable job shielded from most competitive pressure,
because they will be the most surprised and unprepared if their current
employment is made redundant.

~~~
kpennell
saving this comment...

------
zelon88
I keep seeing thse kinds of articles where the author has drank the Cloud
kool-aid themselves, forgotten how to function without it, and insists that
it's impossible to function without it. This guy even drags manufacturing into
the mix, and obviously has no idea about manufacturing in the United States.

Manufacturers in the USA don't make cheap coffee cups. We don't make
underwear. We don't make car fenders. We make warheads. We make gyroscopes. We
make electro-mechanical assemblies that China or Malaysia or Mexico would
screw up. We specialize in quality over quantity, and we specialize in cutting
edge tolerances and specifications. We make export controlled things for
enterprise contracts and the government. Things that require certifications to
produce, and govermnent regulatory compliance, and tight tolerances. Nobody
here is making the 100,000,000 wrenches you can buy at Wal-Mart.

We don't make 100,000 of anything either. We make 100 gyroscopes for General
Dynamics, or 5 jet engines for General Electric. We make US military grade
munitions and weapons for the government. The author obviusly doesn't realize
that the company making the wafers for Raytheon ISN'T ALLOWED TO USE THE
CLOUD. All that great automation that helps AirBNB function with no
infrastructure is meaningless when you have to protect your IP from nation
state actors. To probably >50% of American manufacturing the Cloud is useless.
It's a consolidated attack vector that WILL be compromised in the future and
lead to liability. Sure you can put a NIST 800-171 or DFARS compliant business
in the Cloud, but it costs extra and it's not worth the risk. You hear about
misconfigured buckets leaking data almost daily. Nobody doing govermnet
manufacturing work wants to deal with that headache. Infact, I've been in this
industry for 10 years and I have NEVER seen a DFARS compliant supplier with
outsourced IT infrastructure. I've visited hundreds of companies over the
years. What you're describing doesn't interest American manufacturers one bit.

~~~
munin
> The author obviusly doesn't realize that the company making the wafers for
> Raytheon ISN'T ALLOWED TO USE THE CLOUD.

This is probably going to change. People like you said the same thing about
health data, and student data. The savings were so tantalizing that the
regulators and stakeholders figured out how to make it work. What do you think
GovCloud is for? C2S and "Secret cloud"?

Our university had a 3-4 person dedicated Exchange team. When "Google Apps"
came out, people wanted us to switch to that from our old mail server stuff.
Go figure, why would you keep using pine and squirrelmail when you could use
gmail? "It can't hold student data" the IT team said, "it isn't certified for
FERPA or ITAR." Okay, true. Fast forward two years, now Google's "Apps for
Education" can deal with both. The switch was sudden and brutal and the
university no longer has a 3-4 person dedicated Exchange team or an Exchange
deployment of any kind.

~~~
jimbokun
And at some point, AWS or Azure might be considered more secure than servers
configured and administered by an organization that doesn't have that as its
core competency.

~~~
illumin8
This. Who beat every single private organization with Spectre/Meltdown
mitigation? Amazon Web Services.

There is a pervasive myth that servers run by private organizations are more
secure than those run by the public cloud providers, and the opposite is
actually true. Does your organization receive embargoed information from Intel
to mitigate side-channel 0-days before they are publicly announced?

------
pjc50
> Repetition is a sure warning sign. If you’re building the same integrations,
> patching the same servers over and over again every day, congratulations –
> you’ve already become a robot. It’s only a matter of time before a small
> shell script makes it official.

Absolutely - if something is repetitive, it's a candidate for automation. This
is true across all disciplines. Only the as-yet unautomatable human judgement,
insight and communication is safely valuable.

On the other hand, "go away or I will replace you with a very small shell
script" has been a BOFH joke since the 90s.

~~~
ReptileMan
I have actually replaced a person with 40 lines c# program

~~~
hobs
I replaced a person with a one line code change (performance tuning) - they
got her another job though.

Literally had a business call to see if they could run another instance of
$LOB_SOFTWARE because they had a person clicking a button for 40 hours a week.

~~~
hadlock
We replaced our entire overnight testing and on-call operations department
with SSD and a little bit of SQL optimization. Jobs that used to kick off at
5pm would run until close to noon the next day. When I stopped interfacing
with that unit those jobs completed by 9pm. Went from a department of 25
operators to a team of 15 or so. We weren't able to reduce further due to
vacations, on call, sick time etc.

~~~
hobs
Yeah, I see the biggest problem here when existing code which ran fine never
got the maint it needed to scale up the team, so they throw more bodies at the
issue instead of pay to fix the code.

Good work.

------
redleggedfrog
While I think this essay has some good points, it ignores the problem that I
always see with this idea - people. I really wish the business people I deal
with on a daily basis could have their business requirements met by such
automation, cause it's the least fun part of my job. But the don't because I
don't care how awesome your cloud provider tools are, they always come to me
with some weird requirement or platform and I'm back to "munging JSON in C#".

The problem isn't the technology, it's the complexity of the customer's
business requirements, and their nearly complete inability to transfer those
requirements into software without complex implementations that they could
never hope to implement themselves. I would love to see more tooling to help
with this. I have been waiting for 25 years. It gets better, but not nearly
what can be described as an apocalypse.

~~~
yourapostasy
> ...their nearly complete inability to transfer those requirements into
> software without complex implementations that they could never hope to
> implement themselves.

I wouldn't even mind if they even had "those requirements". That would be a
huge step up. Oftentimes the requirements are not written down and stuck in
tribal knowledge. And woe betide you if the tribe is an outsourcer or offshore
team. About half the time, they're unintentionally leaving knowledge in the
heads of their meat-robots, and getting the information transferred out of
those heads is painful and time-consuming, because "set of procedures for
meat-robots" is effectively what they're hired for.

If the procedure is periodically performed, I often get pretty good results
just asking for copies of the resultant emails reporting completion, and
working backwards from there. Automation at this layer of staff work is
considered exotic developer-realm, needs-a-budget-and-a-project-manager
effort, even for what most HN readers would consider relatively trivial multi-
hour or multi-day scripting work.

The abruptness and agony of automation sweeping through these layers in the
upcoming years as the tooling to discover, capture, distill, and maintain
these requirements with tight synchronization to software teams maintaining
the code behind the automation are going to be politically challenging, as a
lot of these people have zero notion what they're doing can be automated away,
even as they consume the results of ML in their daily lives.

And I'm rather glum about teaching these people how to perform the
automatation themselves. The reception I've gotten to my offers to help them
get on the track to learning programming and automation have been very
underwhelming. Even if someone doesn't "get it" about coding, just the
exposure to the thinking patterns would help me enormously cut down on
unnecessary meeting times, as there are still way too many people whose
conception of automation is closer to "can't they/you just...[magic/mind-
read]?"

------
F_J_H
A point that often gets missed with "low-code" tools, is that it's not so much
that they enable "non-coders" to build applications, but that they enable
experienced developers to go so much faster.

I've been using a fullstack low code development tool for several years now,
and when it comes to developing CRUD apps or data reporting apps (with charts,
interactive, drill down reports, etc. etc.), it's astonishing how quickly you
can stand-up a secure, fully responsive web-app, complete with authentication,
authorization schemes, report subscriptions, etc., without writing any code at
all.

And, when you bump up against the limits of the declarative/low-code aspect of
the framework, you can toggle over to java script, your own CSS, SQL, etc., so
it's not like you paint yourself into a corner.

So, I agree, if Amazon creates something like this, and it is as good as some
of the existing low-code tools out there, it's going to have a big impact over
the long term.

edit: typo

~~~
dleslie
Which tool?

~~~
F_J_H
I hesitate to say it on HN as everyone seems to hate Oracle (and for good
reason), but the tool is Oracle Application Express.
([https://apex.oracle.com/en/](https://apex.oracle.com/en/))

In my earlier consulting role, and now as a CDO, it's my go-to tool for CRUD
and data presentation apps.

(I don't work for Oracle.)

~~~
empthought
I used that for small internal LOB apps at a huge bank when it was still
called HTMLDB. It saved a huge amount of time, despite its architecture making
experienced developers wince.

I wish I had the time to spare to build a PostgreSQL/Python clone of it.

Though SQL is certainly a form of code; it's just nice restricted domain-
specific code that people without the title "engineer" or "developer" often
know.

------
adamc
As a senior employee in an organization that has migrated much functionality
to SaaS and the cloud... I'm doubtful. In my experience so far, what we do
changes, but the need for IT employees hasn't gone down. Most of what we did
was figure out how to solve business problems with IT, and that continues,
cloud or no. SaaS offering are sophisticated, but hard to use out of the box
when you have significant regulatory (and other) requirements.

If there are IT jobs developing for smaller organizations, maybe those will go
away, but... I think a lot of that disappeared already.

I'm close to retiring (from this job, anyhow), so it's not a personal issue
for me. I just haven't seen it happening as described.

------
strikelaserclaw
I think for most mid level guys the threat isn't AI doing their job. I think
its the influx of people who will undoubtedly join the "coding" workforce once
automation takes away many trivial jobs like driving trucks, cabs, or flipping
hamburgers at McDonalds. All these people will be told to retrain themselves
and go into the tech sector, and we'll get a huge influx of cheap tech people.

~~~
kypro
I don't know if you know any working class people, but a lot of them simply
don't have the intellectual capacity for tech work - which is often why they
flip burgers.

~~~
kamarg
And a lot of them also just didn't have the opportunity to learn it but are
easily smart enough to pick it up when their ability to make a living depends
on it.

Let's say 10% of these working class people just need a push to discover
they're actually decent at coding/devops/admin/etc. 3,500,000 truck drivers
means an additional 350,000 people competing for IT jobs. Now factor in all
the other people employed in these sorts of jobs. There's going to be a huge
glut of incoming "cheap" labor when these sectors become more automated. I
don't expect them to be competing with people like Linus but they're sure as
hell going to be competing with the neighbors kid that went to a public 4 year
school for a CS degree because the job prospects were good.

~~~
Latteland
There are no doubt already 300k job openings for different kinds of tech work
in the US. There's probably 30k jobs just in Seattle (endless local npr series
about jobs at [https://www.kuow.org/series/region-of-
boom](https://www.kuow.org/series/region-of-boom)). But not everyone will make
it as an programmer, we need so many jobs for different skill sets.

------
Spooky23
We've been through this before. It was called Windows NT. The democratized
tools were Access and Lotus Notes. Amazon is doing this for the same reason
Microsoft did -- it's revenue by a thousand cuts. People spent $5,000 in 1995
to give diesel mechanics PCs so they could update work orders -- they will do
the same for little apps.

The reality is, you're going to have a million monkeys hitting a million
keyboards, and very few will be producing Shakespeare. All of that crap will
be consuming lots and lots of AWS/Azure/etc bill.

You'll need way more IT people to rationalize it. There are tens of thousands
of people in the United States whose purpose for the last decade has been re-
implementing the 90s version of this in formal IT systems. You will have churn
as we purge the legacy staff, especially windows click to admin types.

------
michaelbuckbee
Automation hits white collar professions as a massive productivity
improvements for the top performers displacing everyone else working in the
field (think the top 10% of people in your position doing 100% of the work).

In web development this is most apparent (to me) in SAAS application
development, where many/most of the underlying pieces of building a CRUD
application that can scale to thousands of users, and be really functional are
now provided by other SAAS apps which provide a _better_ service than the
average developer can scrape together themselves.

Billing -> stripe.com over writing against the gateways directly

Database/Hosting -> Heroku PostGres/Redis and compute

Email -> Sendgrid, Mandrill, ActiveCampaign

Or even just SAAS frameworks like BulletTrain (Rails) or Laravel Spark which
dramatically cut down on the boilerplate and integration code you'd have to
write.

------
Ruxbin1986
As someone who's been in IT over a decade, I am concerned and so many IT folks
are going to be blindly hit.

Sure, some of them will still have positions the same or similar roles but
there will be a crunch. The large outsources will be hit overseas (WiPro,
Infosys, etc.) but it will also impact administrators at medium-large sized
businesses in typical American Cities as Forrest mentioned. The worst part out
of all of this is too many colleges and especially technical colleges still
teaching networking, linux or windows administration as if they'll be able to
have life long career. That is no longer true.

I don't want to imagine what it'll be like for those students who graduate,
get good jobs (now), a mortgage and start to raise their family only to find
themselves unemployed in the middle of their lives. I don't expect much
sympathy from the largely meritocratic tech industry or anyone else.

As for myself, I already work for one of the big three and apart of many
"cloud" migrations. I should be okay but at the same time I am somewhat
conflicted. Am I going to need to go back to school for Computer Science and
become an fully-fledged actual software developer? I mean, it's fine, there's
still enough time (I don't think we will really feel the burn for at least
another 4-6 years) but is it reasonable or realistic that everyone needs to be
rockstar developer?

~~~
ocdtrekkie
I've always kept my coding skills up a bit as a hobby, but I don't
realistically see this IT apocalypse coming. Everyone in Silicon Valley thinks
they're going to automate away everyone's problems, but nobody there has
managed to prevent people from needing the same type of support they needed
two decades ago: Why doesn't my printer work, and how do I know if this email
is real? (If you believe Gmail or Office 365 can do the last one, you're
wrong, FYI.)

I don't think we're anywhere near an "IT apocalypse". I think we're more
likely to put a ton of machine learning engineers out of work long before
companies start needing less help desk technicians and sysadmins. I think a
lot of people have moved to the cloud only to discover they needed just as
many people to help manage their cloud presence as they needed to manage their
on-prem hardware.

~~~
TarpitCarnivore
> but nobody there has managed to prevent people from needing the same type of
> support they needed two decades ago: Why doesn't my printer work, and how do
> I know if this email is real?

The author does touch on this though by highlighting that you will need less
and less people. As more services move into 'cloud' solutions it can free up
time for those and they'll step into those spaces.

~~~
ocdtrekkie
I don't see "less people" as an upcoming IT problem, as most places I look are
getting more IT people, as more parts of their business become dependent on
computers. I haven't seen any evidence of that changing, and believe it or
not, the average business still has plenty of migration to digital means left
ahead of them. (Where I work, manual punch card time clocks are still used.)

~~~
Ruxbin1986
Look at the cloudification or automation in Windows Products - Windows Updates
are pushed through Intune over the internet, Exchange, SharePoint are all
through Office 365. Entire On-Premise Datacenters are pushed to Azure.

Yes, someone still needs to manage all of this but you need a lot less people.
Or you ship these "trade jobs" to low cost areas like India.

~~~
ocdtrekkie
People keep stating "you need less people" with the cloud like its an
objective truth, when there's really no evidence of it, and I'd argue its
patently false. You end up paying both your own IT and the cloud providers'
IT, for a product that also doesn't work when your internet is slow or down.

The main thing moved to the cloud where I work leads us receiving and handling
the same number of support tickets as when it was on-prem. The difference is,
now some tickets we _can 't_ fix, and have to wait for the cloud provider.
Service is worse, and it doesn't really save us any time.

A lot of cloud solutions offer an on-prem option. The tools are the same, it's
just a matter of it running itself in the building or running itself somewhere
else. A lot of times, running something on-prem means spinning up literally
the same software you could have them host for you.

(Also: Windows Updates also aren't some crazy painful manual process that
Intune fixed. You can just tell WSUS to approve everything automatically if
you want, and just as similarly, you can manage Intune more granularly which
takes up your IT staff's time and effort.)

~~~
illumin8
You need less people with cloud is objectively true, as long as you do it
properly.

If you're simply running EC2 instances with your same off the shelf software,
you're not doing it right, but you'll still eliminate your entire datacenter
physical facilities team and server install/rack & stack/replace failed disks
team.

If you do it properly, it's incredible what you can do. I have a client with
applications running in Ireland, Frankfurt, Singapore, Tokyo, and the US,
totaling around 50 EC2 instances running containerized workloads that
automatically heal, APIs that are accessible globally and won't go down unless
6 AWS regions simultaneously fail, about 30 static websites, DNS hosting for a
dozen domains, monitoring, auditing, and log analytics for all of the above. I
set it up in about 3 months as a single engineer and manage it all with about
8 hours a week of total effort. The cost to my client is basically the same as
hiring a single senior engineer, but they're running infrastructure that would
have taken a team of 3 shifts of IT professionals without the cloud.

~~~
smadineni
Nice. I wouldn't mind joining you for a contract

------
forrestbrazeal
Author here - I'm serious when I say that I'm happy to be a sounding board if
you're at a career crossroads. Twitter DMs are open @forrestbrazeal.

~~~
finebalance
I've just started my career in ML, and I already sort of feel that a majority
of the work that people around me are doing can be done by an automated
pipeline (like Python's AutoML) and throwing enough compute power at the
problem. It's quite worrying.

~~~
disgruntledphd2
Learn statistics and how to communicate them.

That's going to be a long time being automated.

For instance, if (for example) you model insurance data in the EU, you cannot
use gender as a factor in pricing (even though it's effective).

In general, the modelling/ML pipeline is the easy bit, the hard part is the
data cleaning and figuring out how to translate a business problem into one
that can be solved by data.

tl;dr as many people have said about Comp Sci over the years, learn the
fundamentals (statistics and experimental design) and you'll be in a much
better position.

------
kamaal
Programmers have been predicted to be going out of jobs since the days of
COBOL. There is a reason that is not happening(yet), or hardware companies
would have been all shipping pluggable chips, we'd just config-connect them
and be done. The reason is in the word itself `Software`.

The real problem with these ready-made plumb-and-plug modules is sooner or
later these are either too slow, or expensive, or just a pain to
refactor/redo. Eventually you just come back and realize you need a more
granular control over things, and anything you are likely to come up with
resembles a programming language.

I had this moment of realization myself while having to change a complicated
graph in Pentaho Kettle a few months back. The graph looks bonkers hard and
brittle, changing anything requires redoing all the dependent elements of the
graph, and if you have a graph complicated enough you will be forced to
rewrite it. The real trouble there is no functional/unit testing with these
things. And then you realize, you are just better off with a full fledged ETL
language/programming language. The second problem I faced was running into
performance issues. Want to change the sort algorithm? Running into heap space
issues? Want better logging? Want a better threading model? All the best.
Nothing is possible.

This is above and beyond the need for meta-programming facilities. At that
point whatever GUI graph you draw is worse than any verbose code you will
write.

Regarding programmable tools, we already have those. Vim, Emacs, Microsoft
Excel all give you a degree of meta control over the tool and what you want to
do with it. But that's that, and it is often hard to bend this tools to your
command.

These are just a few reasons why there won't be an apocalypse soon.

~~~
droobles
I empathize with this sentiment, and will add that it may take fewer
developers/IT folks to get a product out the door (MVP), but a company that
depends on the product will have more specific business requirements and will
eventually onboard more people to deliver solutions specific to those needs.

The job market will close up a bit, but right now tech is looking like the
California gold rush, where 4-5 years ago any bootcamp grad could jump right
into a web dev job (at least in my job market in the Midwest). I think if you
continuously learn and remain marketable as the times change, then as a worker
you will be fine. I also like the comment that mentions that you may just end
up working for the cloud provider rather than the business application
company.

------
mark_l_watson
Nice blog, keep writing.

I manage a machine learning team and I also think that at least partially
automated data curation and modeling will reduce the number of people required
in my field. It might take 5 or 10 years, but I think it will happen.

I think you are spot on that IT and devops will take a hit. I look more at
Heroku’s model that AWS and GCP as the future. That said AWS and GCP will keep
getting more ‘Heroku like’.

~~~
thrav
Heroku and its parent company, Salesforce. Salesforce databases are code and
no code manipulated and should already carry most of the business and customer
data and offer several different off the shelf products for plugging other
data sources in. For most companies, it could cover the majority of what their
legacy IT department does, and for many companies in the Bay Area it handles
everything, except their product and what’s in JIRA. For others, it is woven
into the product.

It’s a complete blind spot for most engineering minded people because they
never realized how flexible the platform was, and with Bret Taylor running the
show now, it’s miles away from just being a clunky Sales CRM.

Couple of examples of recent developments:

[https://developer.salesforce.com/blogs/2018/12/introducing-l...](https://developer.salesforce.com/blogs/2018/12/introducing-
lightning-web-components.html)

[https://lightningdesignsystem.com](https://lightningdesignsystem.com)

[https://developer.salesforce.com/platform/dx](https://developer.salesforce.com/platform/dx)

~~~
666lumberjack
My experience with Salesforce (mostly with Marketing Cloud) has been that the
developer experience is truly awful. Mediocre documentation, no debugging
tools to speak of and systems that never quite do everything you want. Maybe
other parts of the ecosystem are better but I'd be very reluctant to take
another job where I had to work with it.

~~~
wrs
_Actual_ Salesforce is just expensive, complicated, cumbersome, opaque,
legacy-encumbered, and programmed in a unique language and UI framework.
Marketing Cloud is all that, plus what you said.

------
DebtDeflation
20+ year technology consultant here. I have done work for dozens of clients
across just about every major industry out there. The number of (relatively
well paid) people I've seen at clients in this neverland between Business and
IT whose jobs revolve around pulling data from one system, munging it offline,
and then loading it into another system, or similar tasks that should have
been automated with a script a decade or more ago, is absolutely staggering.

------
diego_moita
I object to the word "apocalypse", it is just business as usual.

Better automation has been "reducing the number of people required to deliver
technical solutions" for ages.

Local Area Networks replaced many mainframe computers in the 80's. Optimized C
compilers took the jobs of countless Assembly programmers. WordPress, Joomla
and better web frameworks (Django, Rails) took the jobs of many Perl/Web
developers. Python enabled a lot of people to do what FORTRAN/Java/C++
programmers were able to do before.

"Apocalypse" is just the normal state of affairs.

------
te_chris
I'm advertising for 2 roles at the moment: a Senior Backend engineer and a
Junior Frontend (in London, UK). Almost impossible to find someone for the
backend role, but I've had to turn off the advertising as I've had 71 people
apply for the junior frontend role. The mix is fascinating, a lot of ex-
bootcampers, some CS grads, some self-taught people, but all of them are
desperate for a shot to get into our industry. I've found this quite worrying
as a signal for what's happening in the wider economy.

~~~
joelbluminator
Could be a lot of things, but my guess - maybe seniors don't find your
offering compelling enough? Try avoiding "young teams" and "beers" and put
more "healthy snacks", "work life balance" in the job offer and see what
happens. Seniors generally have better options than juniors, you're gonna have
to work hard to get and retain them.

~~~
RealDinosaur
I read an article regrading this recently... With junior engineers, they are
selling themselves. With senior engineers, the company has to sell itself to
them instead.

------
vinceguidry
I'm waiting for it to happen to web dev so that companies can find something
else to fixate on so I can go do that.

Companies are always going to follow the latest trends and it's always going
to take smart people to follow them. I'm not worried about my ability to make
a living. I just can't wait to see what comes.

~~~
joelbluminator
don't hold your breath, this beast won't go to sleep soon.

------
sudofail
The author and I share nearly identical work histories. I've been in IT for
about 10 years, starting with AWS and database administration, then turned
more DevOps with a focus on CI/CD. Over the last few years, it's become very
obvious that the DevOps role is requiring more and more development skills.
Simple bash scripting is not going to cut it in the modern tech company.

A couple years ago I made the switch to full time development. I now do most
of the DevOps stuff for my teams, but from a developer role, instead of a
sysadmin/cloudops role.

I'm certain that's going to be the future. Look at Google's requirements for
SREs. They are full-fledged software engineers.

~~~
ragona
I’ve gotten to a point where if we write a bash script for something I
consider it somewhat of a failure. I want operational tooling to be easily
maintainable by a large team of software engineers, and bash scripts just
aren’t that.

Services that run services is the way to go. Lose a box from the fleet? No
problem, the operator service bounced it and got it running again. I’ll admit
that timelines often mean you don’t get to build the grand vision out of the
gate, but I absolutely agree with you that bash scripting isn’t how we should
be running services these days.

------
agentultra
Is there another side to this story? I've spoken with friends running the
engineering teams at big analytics companies and they've gone back to self-
managed, bare-metal servers because they couldn't get the cost/performance
they needed from "one size fits all" cloud solutions.

~~~
ilaksh
Well AWS is expensive.

~~~
Ygg2
And if AWS becomes one of the few games in town, it's going to get more and
more expensive.

------
CodeSheikh
Conspiracy aside, AWS is most likely profiling all of its customers code bases
and coming up with patterns as in what domain their applications are being
used for etc. In order to capture that low-code/no-code type of market they
don;t have to do any market research when the research is already sitting with
them. It is pretty genius from a business point of view but yes can become
very concerning for mid-tire programmers as the article states.

------
debacle
Most enterprise IT is change management. Migrating that database cluster takes
3-4 man hours. Planning the migration might take a half dozen meetings, vendor
calls, dry runs, calculating transfer duration, etc. Enterprise IT isn't going
anywhere. The cost of doing it right 100 times is greatly outweighed by the
potential loss in doing it wrong once.

People who are already on the cloud aren't going to ditch their service
providers so they can take time away from their actual business to manage
something they don't understand to save a few hundred dollars a quarter.

Finally, programming hasn't changed substantially in the last 15 years for
most people. I know that might sound shocking to people on HN, but most
developers are working in dingy cubicles on archaic systems without continuous
integration or release management. They're deploying to production and then
jiggling the handle until things work. The systems they produce are just good
enough to keep other business units crunching along, and their saving grace is
two-fold: they don't cost enough to warrant real scrutiny, and the potential
utility of an efficient IT department is non-obvious to most business
managers.

~~~
jpindar
Cubicles! Where do I apply?

------
hi41
I am a developer on HP/Nonstop. Can someone please explain what automation
means with respect to software. Is it bash scripts? I can see bash scripts
doing routine tasks but not complicated business logic which I think can only
be done by writing programs. I lost my job due to outsourcing not automation.

~~~
pmoriarty
_" I can see bash scripts doing routine tasks but not complicated business
logic which I think can only be done by writing programs."_

Bash scripts are programs.

~~~
hi41
Bash scripts have existed since 1970s when Unix was used. That did not result
in loss of jobs though.

------
geggam
Completely agree with this.

There is something to think about. The folks who are running things are
plentiful right now but as this churns we will run out of folks who understand
the lower layers well enough to manage it.

As I interview people I run into this for things like SRE and DevOps already

~~~
arminiusreturns
Exactly this. I've been a sysadmin since I got out of the Corps, with a recent
break and now found a really exciting startup that is automating something
else. This is the first time I've really forced myself to stop allowing
technical debt and lack of staffing to dictate how I automate, so I am working
very close with the devops/SRE team, and I am continually flabbergasted at how
much they _don 't_ know of the underside of the systems they use, or just
fundamentals of many things. (I even had one guy yesterday try to tell me I
should stop using emacs and start using pycharm because "it's a real IDE" ,
yet this is one of the guys who's going to "automate all the things"!?) I'm
having to learn to automate better, but they don't seem to be learning how to
admin better, but I also consider that part of the sysadmins job... just as
devops and SRE need to be "evangelists" in order to get C-suite buy in, I
think the sysadmins need to make sure the lower level knowledge isn't being
lost, or worse, discarded falsely as not relevant.

Having seen the inside of so many places though, it also continually surprises
me how many companies are all struggling with the exact same issues, to the
point I'm highly tempted to try to pitch a whole IT solution I've been
thinking on for years at the next YC.

One thing I've noticed on HN is that too many people tend to think of all
companies as SV software startups, when there is a huge swath of companies in-
between coasts that don't have a single dev or programmer and are just doing
their business. Much of the "devops is killing sysadmin" hysteria is overblown
due to this filter bubble.

~~~
eropple
There's definitely an issue in the devops/SRE world around understanding the
totality of their stack. But I would submit that this is endemic to tech in
general, not just devops/SRE folks. The combination of siloing and incuriosity
isn't an SRE problem, at least not exclusively.

Selfishly, it also makes things harder for me, as somebody who is a
generalist's generalist, because people legitimately do not know what to do
with somebody who's built and shipped mobile apps, can drop into a new piece
of backend software and rapidly get up to speed, will architect, implement,
and manage your cloud environment, _and_ has hard-won opinions about rack
cabling techniques. But I do OK regardless.

------
ilaksh
This is a real thing for certain areas like packaging and deploying software
or database and to a degree. But I think that since we continue to create more
and more software, there will still be plenty of demand, until AI becomes a
significant aspect of software development. That may not happen until at or
near the time that we achieve truly general artificial intelligence. This is
pure speculation but in my opinion the first "real" AGI systems are less than
two years away. Within 5 to 10 years of that happening, I expect unaugmented
human programmers to be obsolete.

------
czbond
I submit - that engineers should NOT be working on anything technical except
at the highest abstraction layers. We should NOT be coding, or scaling
servers.

Technically minded, astute engineers are great at solving problems. In an
ideal world, we'd be using our ability to solve complex real world problems.
I'd much rather the Golang engineer equivalent 20 years from now solve
critical watersupply issues to a village, than writing API's with flame
graphs. Full disclosure: CS engineer by love and training.

------
shoguning
The same thing is happening in data science, just at an earlier stage. There
are software/service providers like H2O and RapidMiner that automate the whole
model selection/optimization/productionization process.

It will take some time, but I think these providers will basically displace
advanced ML knowledge workers, especially at large fortune 500 companies. IMO
to stay ahead of the curve, data scientists need to pick up more business
knowledge and move into a business/financial analyst role.

------
dgudkov
>Repetition is a sure warning sign. If you’re building the same integrations,
patching the same servers over and over again every day, congratulations –
you’ve already become a robot.

This is absolute spot on. It's our company's bread and butter to automate
repetitive human operations. I can tell that many companies are looking for
solutions that would automate as much human work as possible. Repetitive
routines are the typical candidates to replace with software or less skilled
personnel.

------
jugg1es
This article is very prescient. This is happening now and the pace is speeding
up considerably. I work in healthcare and we hit a tipping point about a year
ago where HIE's and hospitals are now comfortable with their vendors going to
AWS instead of insisting on on-premises infrastructure. In the last year, my
company is basically turned on a dime and we no longer use virtually any of
our on-premises hardware. If you are in IT (non-developer), you should be
concerned.

~~~
hn_throwaway_99
At the same time, as a software engineering manager, I would have given my
left eye for a highly competent AWS DevOps engineer. It took months to find
someone with the skills we were looking for.

~~~
eropple
You might be an exception, but food for thought: the number one reason why my
clients, when I was running a consulting business, couldn't find in-house
DevOps folks was an unwillingness to pay. At best, most places would pay line
engineer rates--the same thing they'd pay for a Java or a Rails developer--for
a skillset that's significantly broader than most (certainly not all, but
most) line engineers at a given level of seniority and is significantly more
in-demand. In the Boston area as an example, I regularly saw companies trying
to get senior devops engineers at rates more appropriate for mid-level, and
principals for hilariously under rate. (We're talking $140K offers for a
senior devops engineer when the next place would offer $180K, and $160K for
principals who could get $220K elsewhere.)

To be fair, this is sometimes trickier because a lot of "devops engineers" are
actually mouse-driven system administrators, so the means and medians often
look odd. That somebody who does what I do, and somebody primarily doing
things hand-o-matically, would have the same job title throws a wrench into
one-to-one comparisons. But you can figure it out.

------
peterwwillis
If IT were going to become obsolete, technology wouldn't suck. But it does.
Virtually all technology that either has or uses a microprocessor sucks. And
it probably always will, as long as humans are creating technology.

In fact, the steady increase in both the use and the diversity of technology
means we'll need more people than ever in IT. The job title might change, but
there will always be a department dedicated to fixing shitty technology and
help users use it.

------
SlowRobotAhead
Apocalypse, no. A great filter, yea I think so.

Comparing to factories shutting down is hyperbolic. It think what we’re about
to see is more in line with a a job market that rewards things other than
specific application knowledge.

However, I can’t say much in defense of the mid-level IT Pro because just this
past month I went from considering a server running an OS and MySQL db
somewhere that someone would have to maintaine into using AWS DynamoDB with
API Gateway with automated calls to Lambda functions - I can completely get
rid of our server and paying a mid level IT Pro to maintain it. For our usage
it’ll be practically free now, where I feel like I would have been paying a
lot of “enterprise IT” overhead before.

That doesn’t mean that guy is going to starve, he will have to either adapt as
factory workers have or be valuable in a different way than “I’m good at
something people don’t really need anymore”. This is as old as time.

~~~
Ruxbin1986
Yes, this is old as time but it's just now we've starting eating into
positions that with a moderate education investment you could have a decent
technical career.

Now that isn't the case. The traditional system admin will need to adapt, but
adaption looks like going back to school for software development.

------
MrL567
The flipside of this is, in a hypothetical world where Amazon / Google is
trust busted, this problem is delayed somewhat because only Google / Amazon
have the necessary scale to create the IT apocalypse. I mean there is
Salesforce and Heroku but the major IT automation stuff is from the FAANGs
really.

------
jamisteven
They use to say a jack of all trades is a master at none, now these days SME's
feel like they are walking on quick sand. SME's in core competencies still
useful but more and more are becoming jack of all trades by default of how
quick the industry evolves. Survival of the agile-ist.

------
erikb
Until now the destruction of jobs through progress has never reduced the
amount of work we needed altogether. For instance think about how the
development of transportation created dozens of new jobs actually, and brought
us in many regards onto new levels that wouldn't have been even imaginable
before. I wouldn't worry too much about something that might not be just-bad
and instead try to figure out what can be done with the changes that we
already see. E.g. all this nice virtualization and automation means it's so
much easier to share code and changes and ideas, and helps us communicate over
borders, language barriers, etc.

------
sebringj
I feel this is a very late article and that it has already happened but not in
the crisis way. Meaning, this has allowed a shift in work to focus on
programming tasks rather than infrastructure as much and hence there is more
competition to get features out the door quicker which helps in more IT work
etc. for programmers. I haven't seen a shortage of work in other words, rather
more of it. In terms of code-automation for non-programmers, these will always
fail unless they become AI integrated with some agency and can figure out the
minutia of demands real world apps actually need.

------
jtchang
I am seeing this happen with game developers as well. Unity has made it super
easy to release a game which is fantastic. However a lot of newer game
developers don't necessarily know as many fundamentals.

------
shusson
> Forrest Brazeal is a cloud architect, writer, speaker, and cartoonist,
> currently based in Forest, Virginia. He is also an AWS Serverless Hero

The authors bio indicates extreme bias :p

------
wiz21c
FTA:

>> But they’re less expensive than a bunch of humans with health benefits

Whenever someone opposes health, fatigue of human against robots, he basically
makes human less human and more robot. Hence, justifying the superiority of
robots. It also makes it almost logical that human are PITA to work with and
therefore, are overrated, that is too expensive.

The only way out is to reaffirm the value of human, instead of underlying
their weakness.

~~~
czbond
Ok. While idealistic, this isn't what businesses do.

Businesses are required to sell, to pay salaries. Prices come down over time.
Competition mounts, it gets bloody. Consolidation and then suppression of
margins.

Humans ARE machines, just a different type; it doesn't "Feel right" to call us
that, but we are meat machines.

There is no way to expect the trend of automation, removal of human tasks for
a "better, faster, cheaper" environment. it won't happen.

------
seanmmasters
Give it 50 years and we'll all be "tech priests" from Warhammer 40k, praying
to the "machine god" Alexa in hopes she stops telling us to take our meds and
instead just turns the darn TV back on.

------
chiffre01
Most people working closely with any sort of computer realted technology
should already know they've singed up for continuous change. If you want to
learn a good paying skill for the next 20-30 years try these:

Plumber, Electrician, Auto mechanic, Janitor, Police officer ...

------
tjr225
It would be interesting to know the growth of employees at the cloud providers
relative to the number of IT employees at non-cloud-provider companies over
time.

I wonder how much of this is simply a consolidation of talent and resources?

------
gameface
On the other hand, as an individual working in IT I can achieve so much more
than I could before... team sizes seem like they’re a lot smaller now as a
result. That’s the upside, surely?

------
nunez
None of this is new. "Learn to code, or become irrelevant" has been a massive
talking point for at least 10 years, maybe more, and this is only becoming
more true by the day.

------
shusson
> but that they(cloud providers) are straight-up reducing the number of people
> required to deliver technical solutions.

I would like to see some evidence for this. Has anyone had such an experience?

------
preordained
I thought this was going to be about how our shoestring, bubblegum, and tape
made cloud applications become self aware and kill themselves out of shame and
disgust...

------
oarabbus_
>No, the real trend to watch here is not that the cloud providers are making
it easier for non-technical people to code (although they are), but that they
are straight-up reducing the number of people required to deliver technical
solutions.

Prehistoric era: Agriculture and domestication are straight-up reducing the
number of hunter-gatherers

1800s: Cotton mills reducing the number of manual textile manufacturers

1908: Internal combustion engine straight-up reducing the number of horse
stablehands

-Vacuum tubes reducing the number of human-computers

-Cell phone reducing the number of switchboard operators

Not seeing the problem here.

------
juddlyon
The use of the word "apocalypse" seems hyperbolic. How are these changes any
different than previous technological advances?

------
fred_is_fred
If you are a point-an-click DBA you should be worried about your job even
without this or any future products from any vendor.

------
megaman8
It's kinda tough figuring out a whole new direction if you're in IT. But, if
you're looking to get into web development, it's a pretty good time. Start off
with HTML, CSS, Javascript and AJAX. that's the foundation and from there you
can learn VUE/basic HTML/css from my tutorials:
[https://codeorc.com](https://codeorc.com)

------
driverdan
The site is broken if you have JS disabled. The top banner covers the entire
first page of content.

------
lsc
Eh, I've been a sysadmin/programmer since 1997 or so; I'd argue the slow
automation of our profession has been going on forever. it used to be that
sysadmin started as the kid who swaps the tapes and does the reboots, and went
up to the person automating, and above that, the person troubleshooting your
backtrace when the kernel panics. We were all sysadmins, just of different
levels.

But... the job, if you were a mid to senor sysadmin, was always to automate
away your job. There was always this race against yourself and the other
sysadmins to keep your skills ahead of the scripts, and there were always
people who fell behind during the crashes and couldn't get back in.

Really, I think the radical change that cloud brought us sysadmins was that
employers care so much less now about your ability to go from a kernel
backtrace to bad hardware. Knowledge of hardware in general has lost most of
it's value; that's a lot of my knowledge and experience that isn't worth much
anymore, and a _lot_ of people who will have to find new roles. in my early
20s, I paid expenses while trying to start a company being on-call datacenter
guy for a few small companies. I mean, I'd charge a minimum number of hours,
they'd call me up or depending on the customer, my systems would page me when
something needed fiddling with at the datacenter. - this job... almost doesn't
exist anymore, because most of the customers are on AWS. (I mean, the job does
exist; prgmr.com still even has co-lo customers, though I don't think they are
accepting more... but it's super rare.)

Now, it seems like we want to split the sysadmin role into different titles;

SRE - production sysadmins; you have a high bar for programming skill and
Linux internals knowledge, but a very weak requirement to have experience with
standard configuration management tools (often SRE implies you will be using
custom configuration management tools) -

then you have the devops title for smaller scale cloud sysadmins with a low
bar for Linux knowledge, a medium bar for programming knowledge and a high bar
for knowledge of standardized configuration management tools.

The sysadmin title remains for people who hit SRE standards for linux
knowledge but don't meet the programming bar of SRE or necessarily have the
experience with the standard configuration tools of the devops.

Generally, both in terms of pay and in terms of the number of machines you
manage, SRE > DevOps > Sysadmin - of course, that's modified by level,
location and company;

(Note, both SRE and DevOps involve systems design in ways that I don't
understand well enough to speak about. I mean, I can pass SRE interviews at
3rd tier companies no problem, but at the first tier, I'm not even sure why
I'm failing, which means I'm a long ways from passing. I mean, I also need to
level up my programming to be a SRE at a first-tier company, but I have a
solid idea of where that bar is and what I need to do to get there. I don't
understand the systems design questions well enough to even really know what I
have to learn.)

Also note, I personally think that SRE is probably a more solid career path
than DevOps going forward, if you are in an operations career. Programming
basics has been the thing that has changed the least during my career; the
skill that is most portable. You don't want to marry a particular tool any
more than you have to, 'cause that tool... might be out of favor next year.

~~~
lsc
The other thing I point out here is that "the cloud" is suuuper expensive, and
while that is still way cheaper than hiring a me or two to handle your
hardware while you are small, there comes a point pretty quickly on amazon
where buying hardware and hiring a few people like me becomes cheaper for base
load in the long term.

I believe that companies are staying on amazon longer than it is in their
interest to do so, and I further believe that this will change when people
start looking for savings during the next downturn.

For that matter, I believe that people will overreact and move in the opposite
direction; you will see a fad of people buying their own hardware before they
really have enough load to justify the person with the requisite hardware
abilities. That's just how these in-sourcing/outsourcing fads work.

------
BenjaminBlair
The problem extends outside IT field, as you mentioned factory workers. I
think the main problem is that there's no coordinated effort to control
inevitable shifts. Workers unions tried protecting workers rights but
capitalism clearly won and if there's no profit, you have no securities. IMHO
pieces of advice in this article are very good: do not put all eggs in one
basket. Don't be afraid to let go of outdated knowledge. And prepare for the
change that is going to happen. I'll just add that this is as much a political
issue since unemployment is and will be a huge problem, I can only hope our
public sector will become productive instead of reactive.

~~~
Ruxbin1986
Capitalism won and in return we got Trump.

That doesn't sound like winning.

------
penglish1
I really thought this would be about something else, given the title, but oh
well.

As others have pointed out, this has been going on since.. the beginning of
IT.

I got my start in the 90's, working for an employer that sold tech software
on: AIX, Digital UNIX, Ultrix, SunOS, Solaris and IRIX. SunOS and Ultrix were
on their way out, but still supported. Windows and Linux were relatively new
additions. 64-bit was just starting to become a thing, so we needed to support
both 32 and 64 bit versions.

Supporting dev, stage and prod environments for all this was a huge job, and
took a fairly high level of technical skill as well as a huge amount of domain
specific knowledge. Really - far more domain specific knowledge than technical
skill, on the balance.

It did require building a lot of software, and autoconf was a (new) blessing
for cross-platform builds.

My CS degree was helpful, but honestly there just wasn't much programming
needed, outside of shell scripts.

At the time, every few years someone would say how all of this "IT management
stuff" would be automated away. I distinctly recall Sun executives banging on
about it in the media a bunch just as I was starting to work full time after
graduation.

And I distinctly remember thinking - they are completely wrong. And they were.

Commoditization was a big shift - now all those UNIXes are gone and we're left
with just Linux.

"The stuff" that cannot be automated was shifting then, has shifted a lot
since then, and continues to shift.

At the time, lots of knowledge and skill was needed to build sendmail for
every OS, configure and install it everywhere. Now we've basically just got
Linux, and just about every package you can think of is available via apt and
yum.

Configuration is done through a DSL like Ansible, Chef or Puppet.

And now we're shifting such that we'll just use SES or some other cloud
service for sending, and we won't manage mail servers at all - or any other
commoditizable service - SQL database, noSQL, NFS, block storage, etc etc.

Or we will, we just won't be tweaking many knobs and buttons on it - and we'll
still be managing it primarily with a DSL rather than bash scripts.

And perhaps writing a fair bit more "real" code as well.

But - somebody's got to stitch all that together - as the article says, the
key is providing value that is specific to the company/product/service. It
always has been!!

Creeping - sure.

But is it an apocalypse if a bunch of DBAs and Windows Administrators have to
learn some new skills, or retire, or lose their jobs? People who basically
have had to be continually learning and adapting all along?

Was it an apocalypse when I "lost" the career value of all the skill and
knowledge I had related to IRIX, Digital UNIX, Solaris, etc?

There is a REAL creeping apocalypse, but it isn't this. It is security.
Software is eating the world, and for every line of code written, X new
security bugs are introduced. In this, I'm including social engineering bugs.

That is creeping.

And the apocalypse will be when some combination of those bugs leads to
something truly horrific. If it hasn't already - like the end of democracy.

------
expertentipp
Pop the ad bubble and the IT apocalypse as in the article will not be a viable
scenario.

------
hartjer
What click bait

------
beerlord
If this article is accurate then we should end the H1B program immediately, or
at least increase the minimum income required on it to $100,000.

------
ec2
His thing about the 'fox' at the bottom makes me hope any job he attempts is
immediately automated, what a dunce

------
robertcorey
Read my response to this article here: [https://medium.com/@robertcorey1/hey-
moron-you-better-start-...](https://medium.com/@robertcorey1/hey-moron-you-
better-start-studying-aws-or-youll-lose-your-job-and-end-up-on-the-street-
dd381c300119)

~~~
lenova
How do I downvote this comment into oblivion?

