Honestly I would 100% do a monorepo every single time if there was good tooling for incrementally building and testing libraries. Having to rebuild every image from scratch for every single change scales miserably. Things like Bazel exist, but you basically have to have a team dedicated to operating it (maybe the difficulty varies by language, but it was a major pain when I tried to use it to build some relatively simple Python projects a few years ago).
This isn't really true anymore, in my experience. I've used Bazel with teams of 30-50 and no full-time maintainer, let alone a team.
Once the migration is done, all you need is a few people that do some Bazel gardening every few weeks, and it's certainly not a full time job. This can be someone that does operations (CI, deployments, etc) or a product/infrastructure engineer, or one of each. Github / Gitlab scale to all but the largest projects, and even then, you can just split into two or three "monorepos" and kick the can down the road. With things like BuildBuddy, it's even easier.
As the article states, there are a lot of little of hidden costs and paper cuts when using a many-repo layout. The one that I've seen that's most prevalent is that it obscures copy/paste behavior, since it's much more difficult to detect in a many-repo setup.
Going to Bazel or equivalent is a bit of a mind adjustment, and some languages are better supported than others, but it really starts to pay off in larger projects. Especially if there's more than a few languages in use.
I have personally run converted build systems to Bazel, and use it for personal projects as well.
Bazel 1.0 was released in October 2019. If you were using it "a few years ago", I'm guessing you were using a pre-1.0 version. There's not some cutoff where Bazel magically got easy to use, and I still wouldn't describe it as "easy", but the problem it solves is hard to solve well, and the community support for Bazel has gotten a lot better over the past years.
The difficulty and complexity of using Bazel is highly variable. I've seen some projects where using Bazel is just super simple and easy, and some projects where using Bazel required a massive effort (custom toolchains and the like).
My understanding is that the Python ecosystem is notoriously difficult to integrate w/ Bazel. Javascript is another ecosystem with a lot of fast and loose stuff going on during installs. Golang integration is way better. At work, we use wrappers over bazel (e.g. gazelle) mostly to handle things like auto-generation of BUILD files by parsing source code import declarations and the like. This takes most of the friction away, to the point that many folks don't actually need to understand Bazel to any significant degree.
I think Go works so well with Bazel, because Bazel's concept of versions and modular dependency trees is very similar to Go's.
Python and JS have come a long way in this regard, but depending on the libraries you're using, these languages are still way behind Go/Bazel's standards.
If you're using python, and are scared to jump straight to bazel, poetry is a good in-between tool. More forgiving than Bazel, but strict enough to make a future Bazel migration much less painful.
Is there, say, IntelliJ support for Bazel? Do you need a central server?
I've heard bazel is a bear...
But... all mature build systems are, because they become essentially enterprise workflow engines, process execution engines, internal systems integration hubs, and schedulers. Why? Because that's what an enterprise/mature build system is, it only differs from other software packages with the same capabilities in that it concentrates on build / deploy / CI "business tasks".
My current employer uses Jenkins (which has workflows/pipelines, daemons) and then feeds into Spinnaker (which has a full DAG workflow engine and interface) and likely this is pretty close to a "standard" or "best of breed" cloud build CI system. Of course there is a dedicated team.
Oh and of course the gradle code build in github has its own pipelines and general Turing machine to do whatever you want.
Yeah, in my experience Bazel is painful, but I've prototyped my own build system before. There's no fundamental reason it has to be a pain, I think Bazel just made really strange decisions and didn't pay much thought to helping people find the happy path. When I looked into it at least, the documentation seemed to assume you've used Bazel or something like it before.
Jenkins isn't a solution because it doesn't understand the dependency graph and can't help you with things like incremental rebuilds. It's just a task runner component, which a distributed build tool would probably offer out of the box.
I would be interested in understanding what you think Bazel's strange decisions are. Prior to Bazel, I had also used some of my own custom build systems--I was multiple rewrites into it--and I had independently come to some of the same conclusions as Bazel, such as the need for target syntax that separates the package name from the target name within the package (like how Bazel specifies targets as //dir/abc:def).
There are a number of other Bazel decisions that seemed strange until I tried to figure out how I would implement some particular feature.
My conclusion is that there are lots of small reasons why build systems are a pain in practice, and that the problem is a lot more complex than most people give it credit for.
there is, but it heavily depends on language. It also has issues/makes choices with transitive resolves for things like java/kotlin, so you might have something that builds but the depencency is not resolved in intellij for autocomplete.
Make is not very well suited to this problem for a lot of reasons. Perhaps the biggest and least controversial is that it's not hermetic. Makefiles often make assumptions about the build environment, so a build that succeeds for one contributor will fail for another. I'm sure others have done a much better analysis of make vs bazel than I could do here.
