
Managed services killed DevOps - phillipchaffee
http://techcrunch.com/2016/04/07/devops-is-dead-long-live-devops/
======
jnardiello
Software dev who just switched to devops recently. Good luck to whoever will
manage an AWS infrastructure on his spare time between developing one feature
and another. Seriously, the cloud is not shifting roles. Actually, the more
everything gets moved to AWS, GCloud, etc.. and the more the level of
abstraction will grow - the more you will need specialised devops IMHO.

Overall, disappointing article.

~~~
jt2190

        > Good luck to whoever will manage an AWS infrastructure 
        > on his spare time between developing one feature and 
        > another.
    

The point of the article is that this work will be outsourced to companies
that specialize in DevOps, that _most_ companies that develop software will
not need the specialist in-house.

(edit: Think about $MEGA_RETAIL_CORP. Their main-line of business is not
software development, but retailing. Despite being a large company with deep
pockets, they have difficulty recruiting the best and the brightest in the
DevOps world to come and work with them. When $TURNKEY_DEVOPS_CORP approaches
them and offers their devops as a service product, $META_RETAIL_CORP eagerly
signs up.)

~~~
oblio
They're just moving the turtle lower. Now we'll have horror stories about
devops outsourcing just like we have those about dev outsourcing :)

------
stickfigure
The commentary here is missing the point. The article is about _managed
services_ , not EC2, and yes, it is killing devops. My team just built a very
successful ecommerce company doing respectable traffic volume with zero ops or
devops.

We built it on Google App Engine. If you don't like GAE there are other PaaS
providers that offer something similar. Developer life on a PaaS consists of
building features day in and day out. It's not mucking with Docker, Chef,
Ansible, Salt, Puppet, BOSH, apt, yum, or any other low-level systems
management tools. This stuff doesn't even waste space in my brain[1]. Instead
of learning this stuff, my team has been been learning about the business
needs of our customers and building them out in code.

I plan never to hire a devops team. Running servers is like writing assembly
language; appropriate only for companies with very specialized needs.

[1] Not quite true - I used to work on the BOSH team. It's a great tool for
deploying CloudFoundry. But you shouldn't use BOSH, you should use CF, hosted
by someone else.

~~~
nasalgoat
I recently interviewed a PHP dev who asked me why we even have backend
developers. From his perspective, we could just use Parse and it would do all
the "magic" of the backend.

Then Parse shut down.

I think the lesson here is, you should never trust your core product to a
third party. Also, when you use cookie-cutter services, you really limit the
kind of things you can do, and vendor lock-in really, really limits your
options in the long run.

~~~
stickfigure
I have heard this repeated many times and it's nonsense for two reasons:

First, it isn't actually true: Third-party managed services (say, exotic
databases) are a mere network hop away. If you _really_ had to you could call
out to your own self-managed services in EC2 or whatnot. Even on Google App
Engine, which is one of the most restrictive systems, we use a multitude of
services running elsewhere in "the cloud". This is occasionally a mild
annoyance, but it doesn't significantly alter the value equation.

Second, it's irrelevant: Migrating our application off of GAE would be tough,
sure. You can call this "lock-in" if you want, but it ranks somewhere down a
distant thousandth after a long list of concerns about my product and how well
it is meeting the needs of my customers. Worrying about lock-in is a premature
optimization.

~~~
nasalgoat
The typical argument about the superiority of cloud services is that it's more
hands-off, but what they don't talk about is how shared resources causes all
kinds of new headaches I never had before.

I recently migrated from a dedicated DC to AWS because of business reasons
(ie. CAPEX vs OPEX).

Before, my PostgreSQL DBs were always perfectly in sync because they were on a
dedicated, private network. Now, I often get large replication delays because
the network in EC2 isn't of the same quality I had before, so I've had to
spend a lot of dev time working around the delay issue with creative caching
solutions.

My point is, there's trade-offs and mostly my calculations show that they're
not really worth it, _if you have knowledgable Ops people_. Which, once you
grow to a certain size, you should.

~~~
stickfigure
I think you're misreading the message - _managed services_ doesn't mean EC2.
Managed services are things like DynamoDB, ElasticBeanstalk, SQS (in the AWS
world) or Cloud Datastore and App Engine (in the Google world). If you're
building on IaaS like EC2 or GCE, you will need ops people. If you build at a
higher level, you don't.

~~~
nasalgoat
If anything my argument holds up because you're putting even more abstraction
between you and your systems by using the managed services, which leads to
less understanding of how things actually work.

For instance, I went with running PostgreSQL instead of RDS because I have
much more control over what's going on in the DB than I would with RDS.

But your point that the managed services don't require ops might be true in
the short term, but what happens when something breaks? Or you experience a
level of growth that expands past the capabilities of the services you've
organized? There's hard practical limits to all the of SaaS offerings - mostly
latency and throughput - that doesn't really come up in these discussions.

The analogy about it being like assembly programming might be more apt than I
originally thought, given how not doing it yourself generally results in
slower and poorly optimized solutions, but then "good enough" is often the
mantra of the day at a certain scale.

~~~
khedoros
Further on the "assembly programming" line of thought: there was a _long_ time
where compilers just couldn't match the speed of hand-coded assembly...but
that time is almost completely behind us, and probably has been for quite some
time.

As management tools mature and automatic, systemic optimizations are developed
for SaaS systems, I wonder if the analogy will continue and areas where you
need to manage it yourself to eke out performance and flexibility will get
fewer and farther between over time.

------
rubiquity
DevOps is dead? Well I guess that means I can finally delete my fat repo of
Ansible, Terraform, and Packer configs. Using AWS, GCE and others doesn't get
rid of DevOps. You're just managing computers in someone else's data center
instead of your own.

~~~
jtreminio
I agree.

I recently created a private cloud infrastructure on Digital Ocean. It would
have been easier and faster (and more polished) if I'd gone with AWS, but then
I would know how to use AWS, and would not know how to do these things myself.

Why learn about HAProxy, keepalived, tinc, etc when you've got AWS to handle
it all for you?

Answer: Because relying on a service to abstract the inner workings for you is
a path to the dark side.

~~~
toomuchtodo
Not sure why you're being downvoted. Abstraction is for ease of use, not
ignorance as to what's occurring under the hood.

~~~
davidrusu
No. Abstraction is for higher level thinking. Dev Ops is still young so we are
seeing many attempts at providing these abstractions but it is a very large
domain to abstract over and thus hard to make a non leaky abstraction.

Given a non leaky abstraction, ignorance of whats under the hood is just fine.

~~~
jtreminio
What I was trying to say was more along the lines of a 3rd-party service
abstracting the work is not a good idea in the long-term.

It is unthinkable right now, but what will happen with Amazon/AWS goes
bankrupt and no longer exists? All these AWS Certified Engineers who are not
actually devops will suddenly be lost.

On the flipside, if you learn everything from the ground up and then jump on
AWS or whatever service is available, you will be in a much better position
when the day comes to move away and to something else.

Abstractions in something like programming languages is fine because the
language won't simply disappear one day - you'll evolve as a programmer and
will be stronger for it.

~~~
moondev
> It is unthinkable right now, but what will happen with Amazon/AWS goes
> bankrupt and no longer exists? All these AWS Certified Engineers who are not
> actually devops will suddenly be lost.

Or they could just read this document and move to GCP.

[https://cloud.google.com/docs/google-cloud-platform-for-
aws-...](https://cloud.google.com/docs/google-cloud-platform-for-aws-
professionals)

~~~
toomuchtodo
It's all virtualization/containerization/object|relational storage, no matter
how you cut it.

Know the fundamentals, and you'll be fine. Everything else is rhinestone
marketing.

------
rvenkatesh25
I think the analogy with QA deserves some credit here. "Testing" as a part of
software process has not gone away, but the people who solely did QA are now
gone - either evolved into doing both dev and qa; or moved to companies where
the older processes still exist.

Testing was more closer to writing-code - so that got absorbed into the dev
side faster. Disappearing QA was a you-blink-and-you-miss revolution.

DevOps is a little farther. Several developers are not able to wrap their
heads around the deployment systems, the implications of hardware and network
etc. But they will. Eventually. Because they are smart people.

I think the message of the article is that people who solely have DevOps
expertise, should look to evolve to developing code as well. That way we have
smaller, well-knit teams with each member having an additional speciality
other than writing code.

~~~
HashHishBang
My current job seems to disagree with you and their point about QA. As far as
I've been able to see, the market explicitly for QA Automation is actually
growing. The idea of

>Developers are responsible for full test automation and making sure their
code works.

Is laughable outside of [insert new sexy startup here], if only due to the
difference in mindset.

Granted I'm totally biased as a QA Automation, but the analogy actually
removed quite a bit of credence that this article might otherwise have held
with me.

~~~
pilom
Idk, QA Automation is growing but that has rapidly replaced TONS of "follow
the procedure by hand" type QA specialists. People who knew how to write
procedures well and document executing them well are disappearing in droves to
1-2 people who write QA Automation scripts in Selenium or something.

~~~
HashHishBang
Yeah I can definitely see that side of things. The manual aspect of QA is
definitely reducing, though I would hesitate to call it entirely dead. I love
the fact that I've got experienced manuals who are able to craft well written
procedures that I can automate.

There are more and more opportunities for QA Automation with the ability to
write those procedures, which I suspect are the ones replacing the manual QA
specialists you reference.

------
alexandrerond
What I have noticed more and more often lately is people who say:

"Yeah, I'm in team X and I do the Devops part"

So we basically have an "ops" person sitting with the devs, and call the team
a "devops" team. The devs have still no idea how to deploy their code. The ops
have still no idea how the code looks like. But hey everyone feels better.

This is also true for many job offers in the form of "looking for a devops
engineer". No, what they are really looking is for an ops person that has to
sit in a dev team and do the part no one likes to do. s/He will be lucky if
s/he can actually spend 10% of his time producing application code.

~~~
osweiller
_No, what they are really looking is for an ops person that has to sit in a
dev team and do the part no one likes to do. s /He will be lucky if s/he can
actually spend 10% of his time producing application code._

So? What's the problem with this? DevOps generally means someone who uses
developer like approaches (e.g. scripting, automation, developing one-off
solutions, etc) to solve operational issues. It is a role, and someone has to
do it, and I would argue that there are people who love to do this role.

But if someone sought a job in DevOps thinking they'd be producing application
code, they only fooled themselves. If someone sees some sort of status issue
and forever tries to fight against their role, well that's just detrimental to
everyone.

~~~
alexandrerond
If you are solving operational issues, then you are doing ops (regardless of
how much is infra-coding), imho.

I always thought of the Devops Engineer as a person who cares about their full
stack, both application development and the operation of it, and understands
how one affects the other. Something a step further than just infra-coding.

------
ChrisArgyle
The title is a bit click-baity and misleading. DevOps isn't dead it's just
becoming centralized.

The rub seems to be that quality managed services are widely available and
affordable enough to obviate the need for in-house DevOps.

~~~
toomuchtodo
> The rub seems to be that quality managed services are widely available and
> affordable enough to obviate the need for in-house DevOps.

I have yet to find developers who are throwing themselves at carrying a pager.

DevOps isn't going away. Its just less relevant in smaller teams. Larger teams
are always going to have an ops team.

~~~
lisivka
DevOps does not need to carry a pager. DevOps will automate disaster recovery.

~~~
toomuchtodo
I'm a devops engineer, and I assure you, I have been on call for the last two
years in my current position, attending to application and infrastructure
alarms.

All disaster recovery cannot be automated. A sanity check is required before
your Elasticsearch or Redis clusters decide to f___ their datastore.

EDIT: Or perhaps RDS is to your liking? We've had our replicas (off the same
master) in AWS fail, at which point they would not further replicate. We
needed to fail our app over to the master entirely, reduce its workload to not
kill the master, terminate all of the replicas, relaunch all of the replicas,
and then re-apply the workload back to the new replicas.

~~~
creshal
> All disaster recovery cannot be automated.

See also: [https://github.com/blog/1261-github-availability-this-
week](https://github.com/blog/1261-github-availability-this-week)

> _The automated failover of our main production database could be described
> as the root cause of both of these downtime events. In each situation in
> which that occurred, if any member of our operations team had been asked if
> the failover should have been performed, the answer would have been a
> resounding no. There are many situations in which automated failover is an
> excellent strategy for ensuring the availability of a service. After careful
> consideration, we 've determined that ensuring the availability of our
> primary production database is not one of these situations. To this end,
> we've made changes to our Pacemaker configuration to ensure failover of the
> 'active' database role will only occur when initiated by a member of our
> operations team._

~~~
toomuchtodo
Pinterest as well uses manual intervention:

[https://engineering.pinterest.com/blog/open-sourcing-
pintere...](https://engineering.pinterest.com/blog/open-sourcing-pinterest-
mysql-management-tools)

------
matt_wulfeck
Google SRE and Netflix Devops are places where Devops is not a mindset or
culture but type of engineer that requires operation and development skills.

The reality is the field is less welcoming of specialized developers. We all
have to develop, test, and deploy our code. Call yourself whatever you want
but make sure you comfortable doing it all.

~~~
malkia
I've started reading the Google SRE book that was just released, and now I
know more about the people I have to work with when the situation arises.
Liking it so far :)

[http://www.amazon.com/Site-Reliability-Engineering-
Productio...](http://www.amazon.com/Site-Reliability-Engineering-Production-
Systems-ebook/dp/B01DCPXKZ6?ie=UTF8&ref_=k4w_ss_details_rh)

------
creullin
I love that the screenshot is HTML. DevOps is known to be very heavily HTML
based work... .

~~~
mfenniak
In a proportional font. Ew.

------
rdtsc
One funny things I noticed about DevOps culture is that it is usually
presented as developers should do more ops. I never quite saw it go the other
way, encourage existing ops to do more development. Wonder why that is.

~~~
dsr_
Once upon a time there were programmers.

Then there were systems programmers and application programmers. Systems
programmers wrote operating systems and utilities for them. App programmers
wrote apps. There was a lot of crossover.

Then there were operators, systems programmers and application programmers.
Operator was a junior position who did physical things (mount tapes, plug in
cables) and ran commands to do things on the systems. They usually moved up to
being...

Systems administrators, who did some programming in service to the systems,
but not not much. The more senior a sysadmin was, the more time they spent
programming and the less time they spent doing physical things... unless they
wanted to do that.

Sysadmins started to specialize. People who configured switches and routers
and talked to telephone companies became "network engineers". People who spent
time working on firewalls and security policies and thinking about that became
"security engineers". Junior people who read scripts to end users became the
helpdesk. And so forth.

Then we noticed that a bunch of people were doing things manually when they
should be automated. This was especially bad in places where there were no
senior sysadmins or systems programmers. But we did have the internet, and
senior sysadmins got together and started writing tools to make their lives
easier: infrastructure automation.

You probably know the story from there, but I'll wrap up with one more
important point: you know how when writing a business application you need to
have a subject matter expert who actually knows what they are doing?
Operations is exactly the same way. All the automation in the world won't help
you if you don't have someone around who knows what they are doing. Some
people can outsource this to "the cloud", but not everyone.

------
soyiuz
Devops and QA are more alive than ever. The article actually makes that point.
It is just that the boundaries between the disciplines are being blurred
consequent to the erosion of distinctions between hardware and software,
testing and deployment.

It would be more precise to say that the era of the code illiterate sysadmin
or QA engineer is coming to an end. So is the era of the developer who didn't
understand the basics of operating systems and networking.

------
Voloskaya
This is based on the premise that DevOps is a team of people, it's own silo.
And that is not what DevOps is about IMHO...

~~~
e12e
I was surprised to see that most of the comments here seem to take DevOps to
mean simply Ops. The whole _point_ , the _only_ point, of devops is to not
have "write feature, throw it to ops"-processes. I think the latest generation
of PaaS can been seen two ways: on one hand, it's like having a great ops-
teamt that will handle everything you throw at them (and have set up clear
borders for what is "their" responsibility), the other is to say that it's
great for devops: with the low level of management needed for cloud
infrastructure (compared with setting up your own hardware, choosing and
tuning a network storage solution, manage fail-over for databases, making sure
to patch ssh/TLS/libc/libz/libtiff/libpng...) means that we get better DevOps:
you might prototype in a local docker, but as part of the commit cycle, every
dev will run tests, and push out to production, with rolling updates.

I don't know when "DevOps" became so watered down -- I guess it's the same old
story as with Lean/Agile: the real win is in changing the business process,
and has nothing to do with tooling and consultants: changing processes is
actually hard, so no-one does it. But when you've paid out the nose for
"educating" your "team" to do agile/devops, you start to claim that your
waterfall was built by Toyata. Because you paid for the privilege.

~~~
illumin8
To me, containers seem to be a perfect way to have a reasonable contract
between Dev and Ops. The Ops team just has to maintain a secure, stable, and
performant infrastructure that can run containers, and Devs can put whatever
they want in the container, and not have to worry about anything below it.

~~~
e12e
Sure, but now dev has the responsibility for upgrading all those libraries
again. Just because your container worked fine 5 years ago, doesn't mean it's
a good idea to run the same container in production today. Maybe it parses png
images, or handles ssl termination.

Or you could say that ops now has the responsibility to scan all containers
for old binary code/bad behaviour, and throw the container back over the fence
if they find an issue.

I do agree that containers are a good start on having such a "contract" \--
I'm not sure it's enough.

------
johnm1019
QA is dead because Agile killed it? (quoth the article) I work at a Fortune 10
company and this is just wrong.

Furthermore, leveraging QA in an Agile process is critical to continuously
improving our software quality.

~~~
organsnyder
Exactly. QA's role has definitely shifted (or should shift) toward being
involved earlier in the process and helping to define requirements, but it is
by no means dead.

------
tomphoolery
Ops teams are more useful when they're analyzing data and improving
architecture based on the results of their analyses. The ideal ops role is
beginning to phase out the "overqualified support engineer" framework and
usher in the new idea of being an "infrastructure architect".

To me, this article is more about the architectural/business decision of
outsourcing as much as possible to 3rd-party services instead of reinventing
the wheel in-house. How many Chef repos have the same stupid passenger
cookbook vendored in? How many times have you had to fuck with that cookbook
because it broke or something got updated? My websites for
[http://psychedeli.ca](http://psychedeli.ca),
[http://brother.ly](http://brother.ly), and
[http://www.waxpoeticrecords.com](http://www.waxpoeticrecords.com) are hosted
on Heroku and, apart from the occasional Redis issue, they've been chugging
along quite smoothly. I've had little to no issues with my development and
continuous deployment process, which give my apps the same safeguards that I
get with work (even more) with no one needed to "watch over" the
project...other than me of course.

------
mountaineer
DevOps has never been more popular in the whoishiring thread. I run monthly
trends on the postings, and DevOps just entered the top 10 of terms used[1].

[1] [http://www.ryan-williams.net/hacker-news-hiring-
trends/2016/...](http://www.ryan-williams.net/hacker-news-hiring-
trends/2016/april.html?compare1=Chef&compare2=Puppet&compare3=Ansible&compare4=DevOps)

------
peterwwillis
_" why not simply teach all developers how to utilize the infrastructure tools
in the cloud?"_

You need someone to liaise with the Cloud company, set everything up, test it,
organize changes, and update the system regularly with new functionality. Even
if a Dev had 10 years experience managing infrastructure, they still shouldn't
be doing it because _they should be developing products_. If only there were
someone whose job it was to manage _Operational_ changes, and liaise with the
_Developers_... Hmm.

And by the way, not everything is moving to the Cloud. We've had managed
services in one form or another since the early 90's, and the result is always
the same: the service provider is blamed for things working shittily and the
company brings services back in-house.

And his continual refrain that "QA is dead" \- does this guy understand
product development lifecycle? Whether you're designing a brand new product or
just bringing out a new version, you _have_ to have QA. Can you imagine a
medical device or medical web service being developed without QA, or QC?

Thanks for the troll, TechCrunch.

------
gcarre
Devops is dying. And the secret weapon is not mentioned in the article
(Docker). With Docker and a scheduler (swarm, kubernetes, etc), what are you
going to use Chef or Ansible for? Just to install Docker on a Linux instance?
Docker is just getting started, and we can already see that they are going to
make a lot of tools and companies obsolete.

~~~
Chico75
Even with Docker, I don't think that you would prefer to configure the
environment yourself rather than use dedicated tools like Chef and Ansible,
assuming your environment needs are complex enough.

------
signal
So, DevOps as you (mis)understood it is dead. Long live DevOps as it was meant
to be understood.

------
fapjacks
Look, I appreciate what people are trying to say about this stuff, but there's
one universal truth that managed services _absolutely cannot_ get around:
Cost. As someone that's bootstrapped a couple of businesses from zero, I have
had to lean heavily on my ability to work with bare metal. It is an incredibly
valuable set of skills that translates literally to more money. Managed
services will not kill off DevOps because there will _always always always_ be
companies that want to be profitable sooner, or don't have a ton of money to
throw at AWS.

Second, this is a really clickbaity article.

------
slowmovintarget
DevOps != Ops.

You don't have a DevOps guy, or a DevOps team. You have a team that does both
development and operations. The less work they have to do (like the reduction
of effort with managed services) the better, but it's still work we need to
do.

If you call out DevOps as a specific role, they you're not doing DevOps. Just
like those practicing "Agile" are likely lacking in actual agility.

------
mkane848
As someone with the label "Application Developer" who's been having to develop
and deploy and learn my company's entire backend without much help other than
"oh, here's the username/password for that, here ya go" from my superiors in
IT, I don't know what to think of this.

The servers and databases are all managed by my one boss, but I'm basically
now responsible for every aspect of my application's lifecycle. While I'm a
1-man team now, I know down the line we're looking to expand and I'm trying to
set things up right for when that day comes - proper version control, and an
automated way to push updates to my Django app.

I was looking at Docker and the likes, but we're unfortunately running
Microsoft SQL Servers and Windows Servers which apparently aren't going to see
much support for virtualization/containers til the SQL Server 2016 version
comes out. My company is an engineering firm, not tech, so paying for an AWS
bucket or me making a case to get the fuck away from Microsoft backend stuff
(the headaches I've had with MS SQL already...), etc etc probably isn't high
on their to-do list.

Maybe due to the sort of company I work for this article just doesn't apply to
me anyway, but it seems like the author assumes any company rolling out
software or a service is doing so at the cutting edge and as its primary
focus.

I'm not sure my struggle to find a sane way to automate our deployment and
updates would be helped too much by literally any of the cloud based services
anyway. Even if they gave me a green light to go all-out on whatever service I
wanted to use, at the end of the day I'd also have to manage all of those
services and their connections to the app as well. I'm trading off some work
for another kind, and a lot of these cloud solutions are hardly painless.

------
lazyant
If you automate sysadmin then you can make the case that sysadmin is dead, not
the people who write the automation.

------
armabiz
Just one thing taken from the article comments:

> HTML used as the screenshot is as horrible as article.

+1

------
FussyZeus
_looks around my devops office_

I guess our company didn't get the memo?

------
jackcosgrove
In my experience developers have had to set up continuous deployment
environments. If you design a set of cloud resources for your application in a
dev environment, you are the best person to script that deployment into
another environment. So the developer of the application ends up having to
script the deployment as well. DevOps always seemed like too much
communication between team members for too little benefit.

