I maintain the web monorepo at Uber, so I think I can give some context.
Monorepos allows us to centralize important dependency upgrades. E.g. fixing log4j vulns is a lot easier when you can patch everything simultaneously. Same for tzdata (2022g gave very little heads up) Auditing for npm supply chain attacks was a lot simpler in monorepo than microrepos. Etc.
Monolithic version control doesn't have to mean monolithic everything.
Our web projects can be deployed independently of each other, and we leverage tool like yarn workspace focus and bazel for granular installs and builds/tests/etc.
It doesn't have to mean monoversions either. We support multiple version of libraries, though we prefer coalescing them as much as possible to facilitate effort centralization. Finding out that your library change will break downstreams before you land the change is a feature.
We had microrepos before and the main problem is that to this day I still get some random team coming to me for help w/ some rediscovered 7 year old repo that doesn't even build anymore cus lockfiles weren't a thing back then.
At a large enough org, you'll inevitably see the full spectrum of team quality, from the really good teams to the one intern/contractor getting thrown into the deep end of some unloved ancient thing. You want a common denominator that lets you do things like patch vulns in unstaffed projects.
I've done fairly large migrations both to and from monorepos. Each has pros and cons. For us and companies like Google, monorepos work well with our organization model. For others it may not.
> E.g. fixing log4j vulns is a lot easier when you can patch everything simultaneously.
This is true, though the refactoring action done can also be distributed and carried out via automated Pull Request creation to multiple repos. You still will deploy the changes over a period of time with a degree of parallelism.
> We had microrepos before and the main problem is that to this day I still get some random team coming to me for help w/ some rediscovered 7 year old repo that doesn't even build anymore cus lockfiles weren't a thing back then.
You can solve this by forcing a CI build for every repo to be run periodically.
> For us and companies like Google, monorepos work well with our organization model.
What are the concrete aspects of the organization model that make both orgs favor monorepos?
Upgrades come in a spectrum. Some are able to automerge via dependabot or similar tools, some need minor codemodding and can be managed by tools like soucegraph's new offering. Some take months of dedicated effort and digging through the ripple effects across a web of ecosystem libraries (React comes to mind).
The problem monorepos attempt to solve isn't a technology problem, it's more of a people problem. For example, say you cron CI job fails. Then what? Someone needs to look at it.
It's easier for someone to fix things they currently have context for (e.g. if I upgrade Python or update some security-related config and something breaks, I can reason it was my change that broke it), vs an unsuspecting contractor getting around to some backlog task 6 months after the fact with no context.
Organizationally, we can shard tasks to match areas of expertise. We only need one Node.js expert, one tzdata expert, one JRE expert, etc, to upgrade each of these, instead of everyone needing to obtain above average familiarity w/ obscure FFI bullshit or whatever in each technology.
How does CI work in practice? how do you avoid rebuilding and retesting the whole repo at every change? That would be an insane waste of resource and time.
You use a build system. There are a number of open source ones, we use Bazel. In a nutshell, it keeps track of dependency graphs, so you can test only affected projects.
It can also cache execution of unchanged transitive steps, so you can skip builds/tests that were already run previously (e.g. you could skip most of a large 2nd CI run if all you did in a code review was edit one file.
You can also parallelize execution across cloud nodes.
in 7 years someone will discovery a random subdirectory that doesn't build because wormholes weren't a thing back then ;)
monorepos solve none of the problems you listed. you are just basking in the short lived light after a big refactoring.
only thing monorepo does is make it easier to update some shared code and have all code which uses it run tests beforr pushing the new lib version. with many repos (micro repos is false speach to justify monorepos) you first publish the shared lib and then find out downstream failures.
anyone using monorepo in a way you can't build a small pieceocally is doing it very wrong.
Monorepos allows us to centralize important dependency upgrades. E.g. fixing log4j vulns is a lot easier when you can patch everything simultaneously. Same for tzdata (2022g gave very little heads up) Auditing for npm supply chain attacks was a lot simpler in monorepo than microrepos. Etc.
Monolithic version control doesn't have to mean monolithic everything.
Our web projects can be deployed independently of each other, and we leverage tool like yarn workspace focus and bazel for granular installs and builds/tests/etc.
It doesn't have to mean monoversions either. We support multiple version of libraries, though we prefer coalescing them as much as possible to facilitate effort centralization. Finding out that your library change will break downstreams before you land the change is a feature.
We had microrepos before and the main problem is that to this day I still get some random team coming to me for help w/ some rediscovered 7 year old repo that doesn't even build anymore cus lockfiles weren't a thing back then.
At a large enough org, you'll inevitably see the full spectrum of team quality, from the really good teams to the one intern/contractor getting thrown into the deep end of some unloved ancient thing. You want a common denominator that lets you do things like patch vulns in unstaffed projects.
I've done fairly large migrations both to and from monorepos. Each has pros and cons. For us and companies like Google, monorepos work well with our organization model. For others it may not.