
Untangling microservices, or balancing complexity in distributed systems - ablekh
https://vladikk.com/2020/04/09/untangling-microservices
======
hn_throwaway_99
Some other commenters have discussed it, but one thing to make very clear that
I think a lot of people are missing:

1\. I fervently believe, after having lots of experience with both
microservices and monoliths, that microservices do not provide _any_ technical
advantage over monoliths. If anything, they can make a lot of technical
details more difficult.

2\. Microservices, however, _can_ solve some particularly thorny
_organization_ problems for large teams with respect to how these large teams
build complex software systems.

Thus, in my experience microservices can serve a useful purpose, but they will
only be as successful as your organizational-wide communication is.

Furthermore, for the developers replying "I don't see why I'd ever use
microservices for project xyz", well, if you're a team of 1, there is hardly
ever a need to use microservices.

~~~
mfDjB
I wonder if there is a difference in terminology between the way I use
microservices and the industry? I have worked on a number of systems, both
monoliths and microservices, and when a service gets big enough (think Spotify
or Uber) monoliths simply don't scale. I will agree that doing microservices
before you have to is unnecessary complexity and what I would consider
overengineering, however if the consensus is they provide no technical
advantage I have to disagree, here are some that come to mind:

1) A monolith will have to consume a large amount of resources per request due
to having all the models available at runtime. Lets say I have a system that
handles 100 RPS but only 1 RPS uses that model, and that model is GB's of RAM
and requires GPU, that means I will have to have this memory and GPU allocated
for every request, making scaling much more costly. (Note: your mileage will
vary depending on your complexity)

2) A monolith is a central point of failure, if I make a mistake in that
service it can affect all of my requests, rather than just the request that
calls out to that microservice, this is a difference between my entire service
going down and just a feature.

These are just 2 I thought of off the top of my head, but I wonder why I seem
to be out of step with HN consensus here.

~~~
winrid
Counter points, based on experience:

1\. Yes, if you have an application that works this way you will have to
manage your deployment to compensate. But you can use configuration as feature
flags to keep a monolith but when you deploy it to X servers you enable one
feature set and to Y servers you enable another. So you get the benefits of
developing in a monolith but the isolation of microservices.

2\. See #1, but also use proper error handling. It's been a while since I've
seen a single request take down the whole app.

~~~
mfDjB
I wonder if this is why my issue is with the terminology, what you have
described in 1 to me is a microservice architecture. I currently have a
monorepo deploying microservices in order to get the benefits of developing in
a monolith but with the isolation of microservices, but I do consider it a
microservice architecture on the whole.

~~~
closeparen
There are definitely ways each approach can approximate the benefits of the
other:

\- Microservices can be deployed as processes on the same machine and
communicate over the loopback interface or domain sockets without much
network-related risk. This is still more complicated than function calls, and
I wouldn't quite call it a monolith.

\- Monoliths can have traffic for different endpoints partitioned over
different deployment groups. So the behavior of one endpoint can be changed
independent of the others. But a module deep in the stack, potentially
consumed by many endpoints, cannot be deployed on its own in a way that
affects all its consumers. So I wouldn't quite call this microservices.

------
0xDEEPFAC
Relevant comedy:
[https://www.youtube.com/watch?v=y8OnoxKotPQ](https://www.youtube.com/watch?v=y8OnoxKotPQ)

~~~
ablekh
Hilarious video (and most likely very true). Thank you for sharing.

~~~
hinkley
Do you think he ever found love?

~~~
ablekh
I was referring to technology aspect only. Regarding love, you'd have to ask
him. :-)

------
js8
I find the discussion of monolith vs microservices to be very unhelpful. It is
a discussion about how to split the code.

But what you really want to understand, when building a distributed system
(i.e. system to process large amounts of data), is how to partition the data,
so it could be processed in parallel.

Essentially, there are two options, vectorization (single instruction multiple
data) and pipelining (multiple instruction single data). They both have
different uses in different scenarios, based on the critical path dependencies
in the data processing.

Monoliths are easier to vectorize, microservices are easier to pipeline. But
to choose which one you need before you understand the nature of data
processing you need is a wrong way to do it.

~~~
closeparen
The "vectorizing" and "pipelining" here seem to work when describing
changes/deployments made to the system, but that seems orthogonal to the data
processed by the system.

If one part of your workload is suited to pipelining, and another part of your
workload is suited to vectorizing, then that might be a reason to split the
workload into different processes running on different clusters. Few but beefy
nodes for the vectorized part. Many smaller nodes for the pipelined part.

~~~
js8
> The "vectorizing" and "pipelining" here seem to work when describing
> changes/deployments made to the system

That's not what I mean.

But you're correct, what you want to do with the data informs your decision of
what should be separate processes, and you once you know, you might decide to
split (modularize) the processing code accordingly.

Doing that other way around (i.e. to design the modules before you understand
the data flow) is just going to cause more trouble.

~~~
closeparen
Monolithic architecture implies the company is only willing to manage one
production deployable. Someone solving a specific problem cannot introduce new
processes / network boundaries even if warranted based on the characteristics
of their problem.

------
jennyyang
Uber is not reorganizing their microservices into "macroservices". The
original tweeter refuted it, but it's still being propagated as fact. Granted
the original tweet was poorly written but in the thread itself, he refutes it
later on.

~~~
antoncohen
Thank you. This is the core of it[1], IMO:

> Back in the day, we’d spin up a microservice that did one, small thing just
> like that. We had a bunch of small services built and maintained by one
> person. This was great for autonomy, iteration speed, learning and making
> devops a no-brainer. You could spin up a service anytime: but you’d be
> oncall for it.

> Now, as my area is maturing and it’s easier to look ahead, as we create new
> platforms, we’re doing far more thoughtful planning on new services. These
> services don’t just do one thing: they serve one business function. They are
> built and maintained by a team (5-10 engineers). They are more resilient and
> get far more investment development and maintenance-wise than some of those
> early microservceis.

[1]
[https://lobste.rs/s/mc3k1c/at_uber_we_re_moving_many_our](https://lobste.rs/s/mc3k1c/at_uber_we_re_moving_many_our)

~~~
kevindong
> We had a bunch of small services built and maintained by one person.

That just seems like it'd be a disaster at any non-trivial scale.

------
FpUser
I've never bought into this idea of microservices. Always wrote high
performance native servers and have yet to encounter the situation when well
designed monolith running on multicore monster with gobbles of RAM failed to
satisfy client. Granted it will not work for Google but what does the world
care. Most of the businesses (read potential clients) will never come anywhere
close to those scales.

Long time ago when computers were slow I did design some distributed
applications myself - reliable IP multicast server, business process servers,
map reduce type solutions like bill generation etc where it did make sense.
That was long time ago and now most of those at their old scale could run on a
single reasonable priced server.

~~~
brodie
Microservices can be useful in medium/large companies where you have many
teams of devs and it becomes cumbersome to have them all work on one monolith.
As long as there's good communication and understanding of needs between
teams, each team can develop their services how they want, in whatever
language/platform, with their own review and QA and deploy processes.

For smaller teams or small projects it isn't really super useful. Also, my
personal experience is that when you start making microservices, the monolith
doesn't go away. It also isn't so cumbersome if each team is in charge of one
or a few services and you only split things if there's a clear benefit.

I'm probably saying things you already know. My point is that I think for
multiple cooperating teams, monoliths and microservices are complementary.

~~~
FpUser
I myself directed teams of developer up to 35. Do not know beyond that scale.
Teams simply worked on modules and main designers made sure they complied to a
shared interfaces / state machines in more complex cases. Not a big deal. The
law was: I trust developer so here is your module interface and constrains.
Now go away and come back with implementation (which would still compile to a
single monolith). As I was lucky to work with very good programmers bar couple
of bad apples I've never had any real problems with the delivery and its
quality

~~~
jzoch
Its clear that you could _afford_ to not use microservices (which is great!
You didnt burden yourself with the cool approach since it wasnt necessary).
Not everyone has those requirements though and may NEED them

~~~
FpUser
I completely fail to see what is so "cool" about increasing system complexity
beyond of what is actually enough for _most of the real life cases_. To me:
reliably working product is what is "cool" and that is what I've always
pursued.

------
tutfbhuf
> You cannot build a system out of independent components!

I disagree, the Unix philosophy proves otherwise. You can build very powerful
applications, just by combining grep, sed, ls, ... and the like.

------
jka
Opinions are welcome on whether this kind of approach could help move forward
from the microservice/monolith debate:

\- Teams commit to individual repositories so that team membership, changeset
review, and issue management are kept close to the relevant code, and checkout
sizes are minimal

\- Strong language support for inter-repository dependency management is
introduced to enable effective rollback and rollforward of commits that affect
other repositories

\- The _function call_ becomes the unit of deployment -- not a microservice,
and not a monolith. This allows for low bandwidth deployments and for small
amounts of highly-utilized code to migrate to the devices where they are
needed for computation

\- Function calls and return values are signed cryptographically so that
inter-function calls have integrity, regardless whether within a single device
or across the network

\- Functions that perform I/O calls are tagged with the virtual storage and/or
network interfaces that they require access to

\- Kubernetes-like infrastructure manages the deployment, scaling, and
resource allocation of functions to devices

This seems to me like it would provide the organizational benefits of
microservices while reducing developer hand-wringing over where to define
service boundaries. It just becomes a question of 'should this be a
function?'.

Good tooling around binary-dependency-or-source-code retrieval could make it
easy for developers to work on large applications built like this and inspect
/ debug / modify code as-needed without having to check out or download all
the source code and binaries at once.

------
m0llusk
This oversimplifies the measure of complexity by focusing on the difference
between distributed and monolithic systems. A potentially interesting example
I recently encountered was a monolithic system that used internal interfaces.
Transitioning these internal interfaces from a synchronized design to an
unsynchronized design effectively introduced much of the complexity of a
distributed system into monolithic design.

Ultimately everything about data flow, typing, abstraction, and so on has some
effect on performance, encapsulation, complexity, organization. In a way all
aspects of development from onboarding to builds and tests to release
distribution need to be measured the same way we take into account of resource
usage and basic performance criteria. If a piece of software masterfully meets
all needs, but has internal interfaces that are simply too complex and brittle
to be worth learning then that software will quickly decay and need to be
replaced as the dynamic world alters the constraints of usage.

------
wdb
The backlog example in the article is crazy! Do they really do services like
that in the big world? Backlog service which encapsulates all the services I
can follow (partly) but all these separate ones

