As I’ve grown older I have developed a heuristic, which is that people who talk/write like this, i.e. being completely convinced that one way of doing things is always wrong and one way of doing things is always right, and that people doing things the other way are just ignorant/stupid/wrong, usually have no idea what they are talking about.
The best advice I got early in my career on this topic was: “Nobody serious is going to take you seriously if you give one option as the ‘way it has to be.’ Everything has trade-offs, you will be much more effective as a communicator if you list a few options, along with their trade-offs (and be honest about these). You can even have your preferred / recommended option but it also must come with trade-offs.”
One of the problems I often see is that when trying to explain one of the possible options, people don't understand it. When that happens, you try to explain it to them harder. Your explanation is then mistaken for pushing for that option. It's important to carefully navigate around this perception. People should understand that first you just want to be on the same page what your options are, and then discuss the best one.
My approach to that has always been to make sure I'm calling my assumptions with a possible option, usually that's either the use cases we care about or the scaling/infrastructure concerns.
I learned pretty early on that its really easy for that context to be lost and that miscommunication go missed. Its so much easier to catch miscommunication early when a few extra seconds or minutes are spent calling out the context and assumptions that make one option best.
This is good sometimes, other times it risks losing the audience. Extra info works better when they're ready (i.e. asked for it). This is why it's such a difficult balancing act. You don't want to preemptively answer questions that people didn't realize they should ask yet.
For sure, I was mainly thinking about later stage discussions where decisions like tech stack, architecture, or infrastructure should really be made.
Early on its often all about use cases and users you're trying to reach. If microservices versus monoliths becomes much of a debate that early engineering is probably getting too far ahead of the product IMO.
Well, you mileage can vary. I see trade-off almost everywhere but many managers are not exited to hear about trade-offs and want to get a perfect solution when you don’t need to sacrifice anything. Though it is not as bad as a manager who already made a decision ignoring trade-offs of different options.
That's how I give advice and it can be a huge struggle when working with people out of their element, or in a job they frankly didn't earn.
It's really the only way a productive design conversation can go, but those who can't contribute from their side of the fence will get frustrated. Saying things like "you just need to tell me what to do".
On the flip side, those who think they know better will have pretty much already made up their mind. Which makes things even worse if they don't know better, but for the competent ones this is the easiest and most effective setup of them all.
I agree with you. And at the same time, I often feel like it is more difficult being heard when being nuanced. It seems like what gets discussed most are strong opinions.
Yep. It’s easier for a simple message to be carried by the wind.
But the process of growing up is one of increasing capacity for discernment. Ie, you learn more subtlety discerning when thing A or B is a better idea in any given moment. Will a hard or soft approach work better? Use my old tools or learn this new framework? Make a long term or short term decision here?
It’s hard to communicate because this kind of learning takes a lifetime to accumulate.
could you give an example? my approach is usually something like "I've come up with three options here, I think the first two are equally good, I'm mentioning the third for completeness, but I don't think we should do it because...."
You have a point, but that's also where you come off as indecisive. Since the question was explicitly about that that, presenting 3 options, one of which you have reasons against, when we're all busy and meeting time is constrained, is, in the abstract, a waste of everybody's time. If later on, someone comes up with objections; options A won't work because problem X, option B has issues Y and Z, then sure, bring up option C, which addresses X and Y but has other issues, for further debate, but unless that happens, that's time wasted. imo.
This does hinge on you knowing what you're talking about, and rejecting option C for unbiased reasonable logical reasons you're sure about.
My approach is to have strong opinions (weakly held), and ask if people have objections to the tradeoffs. That tends to keep the focus on specific reasons to choose a given path rather than you and someone else just having different preferences. Doesn't always work, but it's a lot easier than fighting over whether option A or option B is just universally better.
I've definitely seen that happen, and in my opinion its just a sign of bad culture or bad leadership. That's not to say its toxic or widespread, maybe its just a poorly run meeting, but nuance should be a focus of any important discussion rather than the voice that goes ignored.
Depends on your audience. Maybe you need to dumb it down for some people sometimes. That's life. I just try to stay in situations where the audience appreciates nuance if I can
Yeah, so it presents a real conundrum. If you read the article, then he still presents arguments in favor of microservices.
> I often feel like it is more difficult being heard when being nuanced. It seems like what gets discussed most are strong opinions.
I really resent this phenomenon. It traps us in poor local maxima because our systems optimize for engagement over actual development of complex, nuanced opinions. It feels like the dopamine-addled end up indirectly pulling the levers on how we talk even if they're less interested in the actual craft.
I feel the same way about people who are "Everything needs to be containerized in K8s". Sure, it's useful in some cases, but it isn't some universal panacea.
Understand the problem space, understand the solution space, and try to find a fit, don't just grab your favorite hammer and start whacking away.
The k8s evangelists are operating under the assumption that if your product or service is successful, you're probably going to want to scale it, and if you believe in that vision, then you're probably going to want to think about k8s early in your project, not when you've grown it to millions of users worldwide.
That assumption that you will want to scale does not always hold;
The assumption that you want to scale into millions of users do not always hold;
The assumption that you need kubernetes to scale into millions of users (or that you will even want it) does not always hold;
The assumption that solving that problem from the start leads to a better outcome than waiting that the problem appears has "winning the lottery" odds of not being the opposite of the truth.
The amount of assumptions involved in all these really suggests it's mostly rationalization doesn't it?
It's absolutely perfectly fine to scale to millions of users with yesterday's proven simple tech using the benefits of time passed. You got everything that was hard before in managed services, why start complicating the simplest part (stateless app servers) now that we've got everything served on a platter?
I reckon 99% of the apps HN talks about that needs to "scale" can do it with VMs behind a load balancer, a cache, and a relational database. All of these are now available as managed services. Not to mention that you now got tailor made languages for it like Go that compiles to a simple single binary for your VMs to just kick off. Use the progress instead of trying to be clever. I mean, if actually producing a product is the goal, rather than a vehicle to unnecessarily squeeze in the latest tech into. There's no need to make a big up-front investment in building a platform on top of k8s etc. Spend the time on the product instead. And if you turn out to be in the 1% that's a good problem to have, and no, it won't be over night, so you're fine.
Surviving the initial scaling is an horrible experience, full of technical gotchas. People that have seen that will naturally want to avoid it. The problem is that it's an irrational desire, because what they do to avoid it destroys most of their chances to even get there.
It looks like a manifestation of the second system syndrome.
Could be indeed. I've also found it interesting how rarely I hear about actual proper load testing. Like, it doesn't need to be a horrible experience, and it can be done in a controlled setting before even launching if one wants to. But this doesn't really seem to be that common, just by judging how rarely it's brought up in the pop-software dev sphere.
I know you are advocating that position. But, I have seen lots of failed projects because of that thinking. They failed to get to customer #1 because they were building something to handle customer #1,000,000.
Unless you are building something that can kill people if it goes wrong just get it out the door and be as disciplined as you can along the way. If it is successful then you have the money to split it up into a SoA when you need it.
There are some none obvious ways to kill people with bad API design even if you don’t work in a medical industry and don’t deal with heavy machinery. Corners being cut here and there, context gets lost, then it gets reused in a (non)obviously inappropriate and suboptimal way and somebody indeed dies.
Please take effort and do quality stuff as if your life depends on it.
There's a balance. Scope creep and solving engineering problems you don't have are obviously really bad, but too many people see the bell curve meme for tech stacks and lean into stupid. When building something, you want to take a step back and consider some of the most likely places that thing will evolve to, then architect the software to make those transitions/evolutions easy. That way even if you build the wrong thing, you don't have to start from scratch, instead you've created a ratchet of progress that makes each pivot faster and easier.
That is why I put in their “be as disciplined as you can be along the way”. If it never ships what’s the point; but if you ship something that has to be rewritten right away it’s almost as bad.
> Understand the problem space, understand the solution space, and try to find a fit, don't just grab your favorite hammer and start whacking away.
People who understand the problem space will eventually notice that kubernetes isn't a hammer but whole toolbox. So you can reach for it when you need a hammer, a different hammer, or a screwdriver, or when you're not sure what you need, or when you're 90% sure what you need but also willing to admit you might be wrong.
Not a universal panacea, but if we're going to get into this "right tool for the job" discussion, it's a complete misunderstanding to characterize it as a single tool.
It's not about everything fits, it's just much easier for orgs to have a consistent environment for developers to use. Google does this to great success, getting out of the box logging, metrics, a universal way to define a system, access secrets. A whole class of decisions stop being made all the time. Anyone in the org can look in a repo and understand what and where everything is deployed. It's simple stuff like this that make it compelling.
That's really where "it depends" is important though. First the decisions need to be what matters and what to prioritize. With that context its much easier to decide on the best technical implementation for your needs.
The problem is conversations often end up being "we're starting this shiny new project, what tech stack should we use?"
What is really needed is a shortlist of priorities, from scaling concerns to types of users and how frequently content/data may change. Without that its just a grab bag of tools people are familiar with and the latest hype trend.
Same. Basically an inverse relationship between certainty and knowledge. Sort of like those stories told by entrepreneurs about why their startups were successful.
It got upvoted because microservices are currently sliding into the trough of disillusionment. It's trendy right now to hate on them, and the author is fully on the mindless hate bandwagon right along with a lot of HN readers.
In a few years we'll hopefully be out onto the slope of enlightenment, with microservices applied where they're useful and not applied where they're not. If we don't get there, then we'll just run the whole hype cycle over again with yet another rebrand of the same concept.
I dunno man, I worked at FB well before the micro services hype and saw a bunch of problems with them, particularly in debugging.
And in general, putting a network boundary between function calls is gonna add a whole bunch of complexity.
That being said, splitting services so that teams could deploy independently definitely also had a lot of benefits at FB, but I could never understand why so many much smaller companies took the micro services approach.
I'm not into the microservices hype either, I'm just opposed to the reactionary claims in places like TFA that you should basically never split out code into a new service. Both extremes are wrong.
My opinion is that the default should be to keep things in one service and only split them out if there's a very good technical or organizational case to be made.
I've worked on quite a few defining products with a friend, business partner, and all-round designer-engineer genius. About a decade ago, we decided to answer, especially in the early stages, that anything that needed our opinion was - "It depends."
There isn't one silver bullet for most problems or situations; it all depends on several factors.
My only real question when I have interviewed anyone for a senior level role or similar is to find out whether they'll start with "it depends" or similar.
I'll lay out what sounds like a pretty clear algorithm or infrastructure scenario but skip some key assumptions that are needed. What I want to see is follow-up questions related to users, scaling, data retention, etc. Its a big red flag for me if a candidate jumps right into a solution.
Your comment reads like you did not read beyond the title. In the article they provide analysis and contexts in which microservices are usable. Article does not fit in your description of "ignorant" or "completely convinced of one way".
> Let’s do a 1:1 comparison of microservices and modules. Spoiler alert: the argument favors modules, as there is little to be said in support of microservices when pitted against modules!
This is the introduction to their argument and it definitely accurately captures the tone of the rest of the article. As for ignorance, you get frequent sections like this:
> With a monolithic architecture, your system is either UP or DOWN, with no in-between. With microservices, you need to monitor every service. All the services need to be UP, and they need to be able to communicate with each other, all for you to be able to say the system is UP. If even one out of your 888 services is down, the system can no longer be called UP!
The author manages to take one of the most compelling uses of microservices and turn it into a negative thing. Somehow in their mind switching from a model where you're either UP or DOWN to a model where you can be partially UP is worse, which suggests to me that they don't have very much experience operating either type of architecture. Managing an incident during a partial outage is infinitely less stressful than managing a complete outage. Whether you can officially label the entire system as UP is immaterial in a real ops context if the impact of the isolated service that is DOWN is minimal.
In smaller/earlier projects, there may not be such a thing as partially up or partially down. Every module/service can be on the critical path. And if that's the case, it's easier to not be dealing with a distributed system (or to have a minimally distributed one).
I mean, yeah, don't use microservices if you're in a small or early project. That's a totally valid argument that I 100% agree with. Microservices solve a specific category of problems that you don't have until you reach a certain scale (either organizationally or operationally).
But again, this is their thesis statement:
> Let’s do a 1:1 comparison of microservices and modules. Spoiler alert: the argument favors modules, as there is little to be said in support of microservices when pitted against modules!
Their essay leaves very little room for legitimate technical reasons for a service to be split out from the monolith. Even their section titled "When Should You Consider Using Microservices?" basically boils down to "if you already have microservices, if you absolutely must use a different language, or if you're an irresponsible idiot".
Oh yeah I'm not gonna defend the article. I was only really responding to this line.
> Somehow in their mind switching from a model where you're either UP or DOWN to a model where you can be partially UP is worse, which suggests to me that they don't have very much experience operating either type of architecture.
Ah, got it. Yeah, I can imagine someone coming to their conclusion if the only system they've ever worked with doesn't have any major components that aren't on the critical path.
The only right answer when asked "should we use (X)" is "it depends."
I really dislike using react and can easily say off-hand that people just shouldn't use it, but that's because I also dislike working on exactly the types of projects that react is a good fit for. There's a time and a place for everything, if not then why would anyone have bothered to build and maintain the thing?
I can agree halfway, thing is many times "it depends" is such a non answer. like it always stops with "it depends", not "in this case you do x, in this case you prioritize other things and you do Y"
> being completely convinced that one way of doing things is always wrong and one way of doing things is always right, and that people doing things the other way are just ignorant/stupid/wrong
> people who talk/write like this usually have no idea what they are talking about
According to your heuristic, you have no idea what you're talking about?
In all seriousness though, your comment comes across as unnecessarily personal. I disagree with the OP's conclusion, but he raises some interesting arguments on an interesting topic. I came to the comments to see a technical discussion around the pros/cons of microservices. It's off-putting to see that the top comment is an ad-hominem attack
I think you might have misread OP. The first quote you extracted isn't their description of the author, it's their description of what the author is conveying in TFA, and that's honestly a pretty accurate way to capture the piece. TFA's section giving the reasons why you might consider a microservice architecture basically boils down to "if you already have microservices, if you absolutely must use a different language, or if you're an irresponsible idiot". That's not the attitude of someone who is open to the idea that someone reasonable might make a different choice than they would.
What OP actually has to say about the author themselves is that people who write like that "usually have no idea what they are talking about", which is also absolutely true in this case. There are many tells throughout the essay that give away that the author has very little experience operating any type of system, microservices or otherwise.
Yep, I used to be that guy as well. It is easier to see things as black and white because then you don’t need to dive deep and it is easier to sell to management. The really strong engineers/architects I have known can explain the trade offs of the various approaches.
For small teams or early stage development I love a well architected modular monolith. Grow past a couple of teams and maybe add some dedicated ops people and service oriented architectures start making more sense. Get real big and have the capacity to handle the monitoring and orchestration challenges and microservices enable that.
It’s all trade offs and recognizing that allows you to evolve with your operational and organizational needs.
I much prefer "service oriented architecture" terminology to "microservice" because services need not be the same size. Have a daemon listening to a queue to send emails? Sure, microservice! Have 10 REST endpoints that are backed by the same interconnected data? Sure you could have 10 lambdas or something but.. why?
Yeah I try to use “service oriented architecture” in discussions because it implies as small as necessary instead of as small as possible. I find it reframes the discussion a bit and encourages a better set of initial domain boundaries.
While I work on things that are large and need to scale well I don’t work at the hyper scaler level which is where I think the “as small as possible” type services are more commonly needed. But, that is just speculation on my part from reading lessons learned whitepapers from those companies
Yeah, I am also suspicious about people with radical opinions. Truth is USUALLY somewhere not at the end of scale.
BUT, have you noticed yours is also a radical opinion?
The reality is that sometimes, a radical opinion is actually a correct one.
Which is to say heuristics are useful but they still do not replace critical thinking.
The problem with Microservices is actually that IMO most developers simply have no time, willingness, experience or mental capacity to think critically about all that stuff. People frequently need to make decisions efficiently. The theory of efficient decisionmaking I have is that frequently enough it is more important to make a decision than to make a perfect decision.
And it kind of makes sense because I also suspect majority of the population (and that includes developers) are simply unable to think critically or retrospect about their own performance.
And what you do when you have little experience in the field and can't yet think critically about things? You use training wheels.
In case of IT (and business in general), a powerful training wheel is imitation.
So what happens is that somebody at some company will publish a paper about how they solved a problem and suddenly a bunch of people will try to jump on that bandwagon ("you can't go wrong if they succeeded with it") but without the hassle of having to actually think through it, understand what were the circumstances at that other company, how those circumstances are different from own use case, etc. They will imitate what others have done before but without realising a lot of important things about the problem. Which is how we got to this whole microservices mess.
Anyway, I am personally rolling back microservices implementations pretty much every project I join. People do not realise how much time they spend solving problems that are simply due to the fact they have partitioned large application into hundreds of small services which need to each be maintained separately. We have dedicated teams to do stuff that is simply unnecessary (they usually call them "devops", but really they are just ops because more frequently than not they are not actively developing the functionality). All the performance problems, all that inefficiency usually vanishes when you roll all that functionality into a single application and just scale that one application instance over multiple servers.
My current team is even more funny. The microservices were originally meant to allow teams to work independently, but at my current team they work hard to bind all of the development process into a single stream of work. So there is some 80 people in 10 different teams all working on same set of environments, applications, with the same monthly release process, coordinating their work everywhere. But there is about 1 service maintained for each developer which means people spend half the time dealing with complex configuration. And the other half of the time figuring out how to improve performance of an application which copies all of its data from service to service.
There’s a Persian proverb, which is a shorter form of a poem by Saib Tabrizi, which says "don't microservice". We don’t live forever, do we? Laying the first brick correctly is the first step to avoid building a crooked wall.
