
The End of Microservices - reimertz
http://lightstep.com/blog/the-end-of-microservices/
======
iamleppert
I'll tell you the real reason behind microservices: developer fiefdoms. "Faux-
Specialization". It allows developers to feel like they have control over
certain pieces of infrastructure and run the gambit on their strategy for
getting ever more increasing pieces of the pie.

It has nothing to do with building reliable software. You could just as easily
build and deploy a single networked application (so called "monolith"), that
is composed of many different libraries that have well defined interfaces
which can be tested in isolation. In fact, that's how most non-web software is
still written and done.

The real reason is that by having these microservices, it allows single
developers or teams to own or control parts of the codebase and enforce their
control via separate repo's, and when speaking runtime, via authentication:
Sally can't see the code to Joe's service and Joe can't make requests to
Sally's production instance of her service that gives a guess at how long the
car has to arrive to pick poor end user Bob up.

I've seen this same thing play out countless times at large tech companies and
startups alike. It has nothing to do with building scalable, or more
maintainable, or more cleverly designed applications. If anything, it adds
more complexity because now we need to do all kinds of data marshaling, error
checking, monitoring, have more infrastructure for something that should have
been done in shared memory/in-process to begin with. Not to mention all the
issues and headaches caused by fan out of tons of API requests, complicated
caching scenarios, etc. I've seen the horror of microservices architecture
where no one person is responsible for the actual app, only their "service".

There are a few exceptions where its useful to scale out parts of a
distributed application, but in 99% of my experience the services aren't a
real distributed system anyway and are vaguely organized by function,
developer interest, and yes, control.

~~~
hibikir
You say it as if it's a bad thing... Compared to the alternative, fiefdoms
really are a wonderful thing!

I once work for a giant fortune 500 corporation. Our department had a good 300
developers, and had been writing code for years. They had built the whose
system using the classic, box-box-cylinder architecture. There were hundreds
of little top tier 'services', but in practice, they all shared the same
database, had to be deployed pretty much at the same time, and version
upgrades had to go in lockstep. Every so often, database changes were
necessary, and the world would grind to a halt for months to make the system
still work after the migration: It was awful.

On top of this, having everyone using one stack really meant that a council of
elders got to make tech decisions, and everyone else just did what they were
told. This built a giant hierarchy. People near the top would never leave,
because nowhere else would give them that much power. Talented developers
would take years to gain influence, so they often just left. What remained was
developers with no ambitions other than getting a paycheck... it was horrible.

The alternative was to let people maintain their own stacks, as long as they
provided interfaces that other people could call. By limiting how much code
could talk to a database, you didn't need negotiations to change something:
Teams made any change they wanted as long as they remained backwards
compatible, and then had to lobby users to upgrade to newest API versions if
they wanted to remove the backwards compatibility mess. It was awesome in
comparison.

A gigantic place won't have those problems, because they can invest money on
making whatever tech decisions they made tenable: PHP is slow? Let's build a
new runtime and compiler, says Facebook! If you are tiny, you don't need any
of this BS, because if your team of 8 engineers can't agree, your company will
fail regardless. But when you have 200 engineers, it's giving people more
control over a piece of the pie or bleeding talent.

The one thing you still need to do is make sure teams have the right size, and
people have enough accountability, that the product still works. You also need
silly amounts of data marshaling and error checking compared to the monolith.
But the terror of a company that can't hire engineers because the only way to
have any control over your daily life is to have been there for 5 years is
just hard to compare. When people say they don't want to work big corporate
gigs, what they really mean is that monoliths become soul sucking.

So yes, I give thumbs up to fiefdoms, just like I'd rather have inefficiency
in a republic vs a theoretically efficient dictatorship.

~~~
r2dnb
I really value your contribution, thanks for sharing.

I give the proponents/consultants credit for proposing a practical and
incremental solution to these people instead of just insulting them or saying
"you're doing it wrong".

That being said, the issue I have with microservices is the fact that the
below is repeatedly used as a core technical argument :

> They had built the whose system using the classic, box-box-cylinder
> architecture. There were hundreds of little top tier 'services', but in
> practice, they all shared the same database, had to be deployed pretty much
> at the same time, and version upgrades had to go in lockstep. Every so
> often, database changes were necessary, and the world would grind to a halt
> for months to make the system still work after the migration: It was awful.

This is not a feature of "monoliths", this is a feature of shortsighted design
(at the beginning) and cowboy design (at the end).

Let's be clear : apart for load balancing for performance critical areas
(which only exist in 20% of the applications, and only cover 1% of a typical
such application - or 60% if you are Google, Facebook, or Amazon or the rest
of the top 10), there's nothing that microservices do better than in-memory
libraries. Also, centralizing everything as opposed to having processes run on
various servers creates the issue of the single point of a failure. Whole new
classes of similar issues appear. It basically is buying troubles.

Adding microservices will never be a neutral operation, and incurs a real
cost. This cost is the cost of distributed objects which has been known for
over 20 years and has been well-documented by our fathers.

Once the above has been dismantled, it then comes to : "yes but teams can
develop in different languages", "yes but dependency management becomes
easier". But once again, when it comes these interoperability considerations,
a solution also exists and has been created about 15 years ago. This solution
is called .NET (30+ languages compiling to a single Intermediate Language, COM
components, first-class library support, etc...). Even the "yes but I need to
communicate with non .NET in-memory applications" is very maturely addressed
by the framework. (it was one of the first-use cases addressed)

I also read in this thread : "the ability of teams to control their own
destiny and not be blocked by other teams". With libraries teams can choose
the version that works best for them, and a well-design in-house library would
provide the extension points (Dependency Injection, Dynamic Binding...) that
would allow the clients (the other teams) to specialize the behavior.

So in a gist, my main reproach for this, is that if people put the same effort
they put in learning microservices to learning system and component design,
they wouldn't even be a need to talk about the former. My regret is that
people will always choose the hype and the new. But hey, that's how the game
works.

~~~
d0mine
The discussion about monoliths vs. microservices reminds me about Knuth's
10+pages of Pascal code vs. McIlroy's 6-command shell pipeline
[http://www.leancrew.com/all-this/2011/12/more-shell-less-
egg...](http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/)

------
lobster_johnson
I see several people criticize microservices here. We've been doing it for
about 6 years and are extremely happy with it.

A core principle which a lot of people and articles ignore, though, is
_reusability_. I bring this up on HN every time there's a discussion about
microservices, yet I've never seen any discussion about it.

Essentially, you build out the backend to act as a library for your front end.
So we have login, storage, analytics, reporting, logging, data integrations,
various forms of messaging, business-structural stuff, etc. etc. all bundled
up as separate services. The front ends just use these services to build a
coherent product. The front end is the monolith: The microservices are the
cloud.

For example, let's say I wanted to create a new product called "Hacker News".
I'd use our storage service to store links and comments. I'd use the login
service to let users log in. I'd use our messaging service to send
notifications about things like verifying your email or send password resets.
I'd use our analytics backend to emit events for reporting. And so on. I could
easily build the whole thing without writing a single line of backend code,
and without setting up a new cluster, because every backend service has been
designed with multitenancy from the start.

This ability to piggyback on a platform of services where I think the real
utility of microservices lies. Everything else — fine-grained scalability,
smaller surface for tests, language-independence, swappable implementations,
etc. etc. — are secondary to that.

~~~
carterehsmith
You are not getting more reusability from a bunch of microservices, compared
to one big service, I don't see where you got that?

~~~
vog
I also don't see how microservices are more reusable.

However, the _replacability_ is usually much better due to being forced to
define sharp boundaries with clear interfaces.

~~~
pjmlp
But that is the thing.

It just means that instead of using the language features and creating clean
interfaces across modules, they were witting spaghetti code.

With microservices they get to write spaghetti REST calls with the added fun
of distributed computing failures.

------
dasil003
Equating "Microservices" with "Information Superhighway" really shows the tech
bubble that this article is written in. "Information Superhighway" was a
vacuous but mainstream term used by politicians and public figures.
"Microservices" is a tech hype train led by expensive consultants and pickaxe
companies thriving off the current tech boom.

Don't get me wrong, a service-oriented architecture is the only thing that
scales to large companies. Once you get to dozens of engineers and millions of
lines of code you will inevitably need to have an SOA because Conway's law.
Also, there is a learning curve to building microservices which improved
tooling really helps with.

However the thing that really grates at me is how these articles say things
like:

> _Services are now an everyday, every-developer way of thinking_

With nary a mention of the overhead. There is no way around it, distributed
systems have an irreducible complexity no matter how good your tooling and
boilerplate is. You have to put in extra work to decouple everything and
handle failure in a way that actually reap the benefits of the distributed
system. And in the end, what these articles always gloss over is the interface
between these systems. If you can easily define an interface between systems
that stays relatively stable as the service evolves, then congratulations, you
have a good candidate for a service with minimal overhead. But for most
applications, those interfaces are shifting all the time, and there is no
better tooling than multiple logical services running within one binary and
build system where _integration testing and validation is cheap_. This is a
real fucking problem people, it's not going to go away because there's a
couple billion dollars worth of venture-backed startups ready to blow their
cash on you in the vain and most likely misplaced hope that they are actually
going to _have to_ scale to dozens of engineers. Premature scalability is one
of the worst siren songs for young engineers and we're seeing it in spades
right now.

~~~
sp527
Speaking as a young engineer, I can tell you many are already looking past
microservices towards things like AWS Lambda, AMQs, and BaaS, which make a
great deal more sense. Why? Because it helps reduce dev effort down to purely
the logic you'd have to write no matter what, with better guarantees about
reliability and scalability, and less maintenance. I hesitate to say
'serverless' because that still feels somewhat out of reach, but that's the
direction things are trending towards.

I also get the sense that a lot of purists moulded in the ways of yore are
alarmed at the waning relevance of their skillset. This to me seems like a
bigger problem than the evolution of software paradigms.

~~~
dasil003
Oh yeah, I don't think it's bad research going into this. I mean I've been
writing software professionally for almost 20 years now and I am super excited
about (for example) containerization and other trendy areas. I think it's
_great_ that people are pushing the boundaries of SOAs, and I think
microservice is a good term describing _how_ the boundaries are being pushed.
We will all benefit from the improved tooling and experience gained.

My only complaint is this ostrich mentality of declaring ubiquity and ignoring
the downsides and unsuitable cases. I suppose drinking the koolaid might be
necessary to really buy in and push the paradigm forward, but sometimes it
feels like willful ignorance.

~~~
sp527
I agree completely. I may simply have had the luxury of so far working in
environments wherein software fads aren't hailed as Swiss-Army panaceas.
Here's to hoping my track record doesn't ever trend Dilbertesque :)

~~~
dasil003
Me too. At this point in my career I've worked very closely with more than 100
programmers for long periods of time, and only one of them would I classify as
being overly interested in fads. To the contrary, I give a ton of credit to
the vast majority of programmers as always being pragmatic about the problem
at hand.

I think the problem lies more in the PR, advocacy and conference circuits
where there's an incentive and agenda to present things as silver bullets. It
mirrors a greater problem in public discourse that everything has become so
polarized that nuanced debate is drowned out by noisy, confident blowhards.

~~~
sp527
"The trouble with the world is that the stupid are cocksure and the
intelligent are full of doubt." (Bertrand Russell)

Back when I was still an employee, I strove to take a more intermediate tact
that let me be more vocal about technical direction. What's surprising is how
willingly people with authority will listen to what you have to say, once they
realize you know what you're doing. I've also been fortunate to have had roles
that enabled that in the first place. And now I'm trying my hand at
entrepreneurship, which feels practically utopian :)

------
msoad
One thing I don't like about SOA is that an error does not have a full stack
trace. I know Zipkin exists but it's nowhere close to what we had in a
monolithic app where you could just put a breakpoint and trace back to where
exactly an error is thrown.

If we can find a way of running a giant monolithic app in development and
production environment without vertically scaling our machines, I would rather
have that.

Every bug I'm working on is like a mystery that I have to hop to many services
to find what's going.

I also think HTTP is the worst protocol for apps to talk to each other.

~~~
uluyol
What's wrong with HTTP? If you're going to build an RPC protocol, you are
going to need encryption, stream multiplexing, a way to distinguish RPC
methods, support for extensions (e.g. auth), and compression.

Add the fact that high performance HTTP servers and clients exist in most
languages, and building an RPC protocol on top of HTTP sounds pretty
attractive. No wonder gRPC did exactly that.

~~~
devonkim
Tons of round trip calls potentially in most RPC-style systems are one start,
which Cap'n Proto addresses with its unique style of avoiding round-trips for
many network transactions which can drastically reduce network chatter.
Security is addressed simultaneously (but not necessarily sufficiently for
many people or regulatory bodies) in Cap'n Proto at least via use of
capabilities instead of building it into transport layers.

------
sinzone
When this year I came back from Dockercon I immediately wanted to write
something very similar to what this article describes. I wanted to imagine a
world where Containers and Microservices were part of the past already, and so
I wrote "DockerCon 2020" [1] and how it will look like.

[1]
[https://medium.com/@sinzone/dockercon-2020-a513ed04eefb#.rbz...](https://medium.com/@sinzone/dockercon-2020-a513ed04eefb#.rbz3pruf4)

------
danblick
I'm sorry -- I think the author completely misses the point about why
microservices were controversial at all?

Distributed systems are not the same as centralized ones, and you cannot paper
over the differences between the two. It is wrong to think that distributed
microservices will completely replace centralized services in some future
paradise. The difference is not a tech fad; it's more like a law of nature.
Distributed systems should plan for network failures, yet nobody wants to get
a "503" from their CPU.

~~~
AstralStorm
You will get 503 from your kernel if you try hard enough. Handling this is
technically required in any application, but often ignored, especially when
allocating memory. (Because there out is a deferred failure.)

------
ianamartin
I'm not totally convinced that allowing developers to build faster is really
all that great of an idea. At least not the sacred ideal that seems to be
accepted without any question.

Most of what I see when people are moving fast is building things as fast as
they can think of them based on the first idea that comes to mind that sounds
like it might get things done.

But the reality is that the first way that you think of implementing something
isn't always the best. It's often just about the worst. Giving people the
ability to take any whim of a design and run with it all the way to production
isn't the best thing overall for software quality.

Perhaps I'm alone here, but I'd like for developers to slow down and put some
thought into what they are building, and how it's supposed to work, and if
it's going to be able to do what it needs to do. I see a lot of "close enough"
in my line of work.

I know it's different in a startup, where testing the idea _now_ is important,
and I'm not slamming that. But the vast majority of developers don't work in
startups where getting a product to market before a competitor is the
difference between making billions and going home broke.

We temper our desire for perfection by reminding ourselves that good enough is
okay for now. I'd like us to temper our desire for speed by remembering that
there is such a thing as soon enough.

~~~
nkassis
What you are describing is definitively an issue I face every day but I don't
think speed of development is the issue. Higher productivity is good but often
what is called higher productivity is just due to carelessly ignoring design
and planning in favor of get things done now mentality.

Often the worse offenders are rewarded for being highly productive and the
people who end up having to cleanup, refactor and get things actually working
are not acknowledge for the vital effort they put in. In my view this is the
result of bad management incentives and failure to properly asses results and
contributions.

------
TickleSteve
"Microservices" are a new name for a very old concept.

This is just low-coupling, high-cohesion by another name.

Small, composable, decoupled, highly cohesive components are what "good"
software has been about for decades, but it now has a new name in the server
s/w world; "Microservices".

Only the name is new & hyped. The concepts have been true forever.

~~~
alayne
There is software and concepts around managing microservices that didn't exist
before, and so on. Nothing can be new when you use such a vague way of
comparing things. There were electric cars in the 1800s, why talk about Tesla?

~~~
TickleSteve
...so what is a new concept regarding microservices?

I don't see anything new... new tools, granted. Then again, there's new tools
being produced every day for every architectural style.

------
ris
Another good article on this subject: [https://m.signalvnoise.com/the-
majestic-monolith-29166d02222...](https://m.signalvnoise.com/the-majestic-
monolith-29166d022228)

My experience with microservices has been pretty painful. My analogy of
microservices is it's a bit like building a car factory on two sides of the
Danube. And there's no phone line in between. You've got a factory building
cars up to a certain point, but then they have to stop work and pack it all up
onto a barge, figure out how to fit everything on the barge and send it away
across the river for the other side to spend time unpacking & figure out how
it all fits together...

As a django guy, I've tended to follow the pattern of spending time making my
models nice and rich, with useful traits which will be helpful at all levels
of the app down to the views. To then have to pack this all up and deliver
some "dumb" json to the other side feels like a massive waste of time. With
microservices I spend my life marshalling data around.

And the number of times I've realized I've just spent an hour discussing the
philosophical implications of how a particular bit of the rest interface
should be designed, all for an interface that we're the only consumers of and
doesn't need to exist in the first place... I've found depressing.

The ramifications on testing are a further story. Do you have to test all
kinds of ways you can receive your rest requests malformed if you're the only
consumer and know exactly how you're going to use it? Good use of developer
time?

------
rbosinger
Ok. I'll be the guy to bring up the Elixir/Erlang ideology that has gained
such popularity here. Although I don't have a ton of experience with it yet it
seems like the possibility of having the idea of "services" built into the
language/framework design is very realistic. That's exciting for me. Although
true SOA can be a mix of many technologies I personally find that scary. What
happens when your whole platform ends up as a web of services on different
technologies and you lose various talent? Now you have to recruit all kinds of
different expertise or hope that certain services keep ticking without that
knowledge in house.

------
drdaeman
I don't know about the microservices and stuff, but I've got one cumbersome
monolith to deal with, and it had started to rot (you know, rely on outdated
dependencies that one can't upgrade without significant effort etc etc).
Splitting it to a few isolated different systems looked like the only sane
choice.

Luckily, I've had to redo one logical part of the monolith anyway, because of
some changing business requirements. So I made it a separate independent
project, that had used all the modern currently-stable tech (rather than few-
years-old one + accumulated baggage of the past architectural mistakes) and it
all went quite nicely.

It took me 1.5 weeks (quite busy ones, but meh) to extract all the old code
pieces that I've needed, clean them up, update with the new logic, and get the
freshly-minted project ready, tested, packaged up and running in production.
The only thing I've lost is ability to run cross-db queries (we just have a
folder of read-only SQL query snippets to fetch some useful statistics once a
week or so), because I put the data in a separate database. I hope,
postgres_fdw would work when I'll need it.

Would I've tried to update the whole monolith, it would've taken me months.

So, the next time I'll work on some large-enough part, I'll probably extract
it into a fresh project as well. As I see it, I'll end up with a remains of
legacy project surrounded by a few small(er) monoliths. And then the legacy
piece would be small enough to get cleaned up.

(I don't know about micro- scale and putting every tiny thing into a different
microservice, though. I have an impression it requires a lot of extra unwanted
cognitive load to manage so seems like an overkill to me.)

So, my point is: software (code) rots over time. Multiple projects (services)
allow to update pieces at different pace, which is less stressful on
developers.

------
tomc1985
It is weird to read people write about microservices (or some other tech fad)
as if it is this otherworldly thing that requires instruction and training. So
many words dedicated to describing the supposedly bad old days!

All this stuff is just another aspect in the life of a practitioner of
computing. A proper expert should see these things not as a fad, but as a
collection of techniques that can be added or subtracted to at will depending
on the prevailing need. It's silly to declare any of these fads dead or alive,
they're just simply techniques that ...people... have bundled together under a
common label

~~~
xxs
...but you can write (and sell) books about, talk at conferences too

------
mdgrech23
The title is link bait and does reflect the arguments put fourth by the
author.

~~~
romanovcode
Not to mention it's a plug to the product he's selling.

------
stevehiehn
Maybe a more appropriate title could be 'Microservices are the Norm'

~~~
brianwawok
I don't think they are. I would be surprised if 10% of new code right now was
coded in microservices.

~~~
stevehiehn
Ya, you're probably right. Its likely just my bias. It feels like its the only
thing i encounter at my day job.

------
justinhj
I joined a company where the proof of concept had, inevitably, become the
monolithic application we would work on for the next two years. Everyone on
the team agreed that the monolith would be a liability so we started to share
knowledge on microservices and plan for that in the future. To do this we
stuck to a handful of rules. Systems should do one thing and do it well, with
a well defined api and protocol. Whilst all the data may be in the same redis
and MySQL instance we made the data store configurable as well its location,
and made sure systems did not read or write each other's data. We wrote
generic systems as libraries with no dependencies on the rest of the monolith.
The results of this work, which was a lot of refactoring, is that when we
decided to farm some work out to a contractor we could do so as a
microservice. They worked in their favourite language with their own tools,
implementing the api we wanted over a specified protocol. At any point it
would be possible to split out services to scale them horizontally, but we
didn't have to until we need to, because every split increases the operational
costs and complexities a little.

------
kevinr
I feel like maybe the Big Idea of microservices is that web APIs provide
better isolation guarantees than library calls, and now with the move to SaaS
either the scale of our applications is large enough or (more likely) with
virtualization the intra-server network latency is small enough that we can
afford the extra overhead of web APIs relative to library calls in exchange
for that isolation.

------
gedrap
When it comes to services, I think it's worth to talk about one common use
case which comes with different motivations and problems: adding new features
to old, probably poorly engineered, monolithic application. Features that are
not tiny yet-another-crud-on-a-new-table but completely different than most of
the existing functionality.

In this case, they really pay off if they are separated well which sometimes
is hard. But executed well, it allows to keep moving quickly as the
requirements grow. Of course, this is not an excuse to avoid refactoring
monolithic application, improving testing, etc.

I've worked in such a setting in companies, and both times it was a win and
helped to build important to the business features really quickly and
reliably.

But is it worth to write an application from scratch in a service oriented
architecture? Probably not, most of the time. Especially if 'product to market
time', 'MVP' and similar concepts are very important for you.

------
k__
I had the feeling that microservices would add too much of complexity, but
with FaaS this is canceled out by the fact that almost all server management
complexity is handled by a different company.

------
tuananh
They have the word "microservice" on their homepage :D

------
EGreg
John Titor is back!!

~~~
stephengillie
Did he bring back the IBM 5100 that he promised?

[https://en.wikipedia.org/wiki/John_Titor](https://en.wikipedia.org/wiki/John_Titor)

------
daxfohl
Every generation's microlith is the next generation's monolith.

For this generation, the end of microservices will be when we can look at a
cluster as one big unit, and deploy a microlithic monolith on it.

For the next generation, who knows how they will slice it up.

------
andyidsinga
re: "it was never the size of the services that mattered: it was the
connections and the relationships between them"

re relationships I would say these are better thought of through the lens of
_separation of concerns_.

imo, connections, a la protocols like http will fade into the background and
be a focus of ops. A piece of code knows to write/read data to/from _________,
for which it has been authorized access by address/hashname/entity.

------
hhsnopek
Not to seem naive but I work with a lot with microservices and I've never
heard of lightstep...

... follow up, what's everyones opinion on this?

~~~
kornish
Ben Sigelman, one of the founders, is a former Googler who's got tons of
domain knowledge about profiling distributed systems of services and about how
to structure calls and dependencies within those systems. He gave a fun talk
at the SF Golang meetup in 2014. I, personally, am very excited to see what
he's up to and will be keeping an eye out for more Lightstep stuff - I guess
it's his newest thing.

(if you're reading this, hi Ben! Sorry we never got together last year)

------
orasis
The Gartner Hype Cycle appears to be accelerating...

------
vegabook
nitpick: nice graph on the "information superhighway" but a label about
exactly what is being measured (0.000055%, _of what_ ) would be great. Is this
a word count? Or its rate of change? There's a reason our (stuffy)
professors/sub-editors always insist on sources and accurate labeling. I'm
only interested because the most recent data points, despite large decline,
still suggest a non-negligible use of a term which I haven't heard in years
other than in a small amount of anecdotes/jokes.

~~~
djspoons
It's just % of bigrams that appear in Google's sample of books. [1] Not
especially meaningful, I admit!

[1]
[https://books.google.com/ngrams/info](https://books.google.com/ngrams/info)

