I wish more people were taking a look at Bazel, and the Bazel-family, of build systems. I've looked at Meson/Ninja/Make and it works but it's truely very complicated and very difficult to get hermetic & reproducible builds from these systems. In my opinion, Bazel will be the future once the currently ongoing external dependency management design doc is implemented. I've migrated quite a few builds to bazel at work and I've seen a dramatic reduction in compile times - not even from compiling source code! Doing things like packaging zips, generating code, generating docs (swagger), etc all adds up in CI time and automating, and caching, absolutely all of it is astonishingly useful.
Bazel is a decent idea but poor execution. I think Nix is the future because it already has the "external dependency" part on lock and it's just a matter of making more fine-grained stuff.
Meson + Nix skipping the Ninja part I would really like to see. I prefer it when the "big dumb builder" and DSLs are developed a bit separately, too. Agreed Ninja's lack of purity is a real bummer.
I'm not so sure about Nix since it seems much more complex in presentation. The meat and potatoes of Bazel is the BUILD langauge. There may be some warts right now but the community is attempting to address these warts. Difficulty around specifying versions of packages, semantics for building containers, etc.
Nix users do not seem to think it's build language is complex. I think Nix might be the worlds best attempt at something like this but it's the same reason why Haskell hasn't gotten adoption: "eeh it looks complicated"
> There may be some warts right now but the community is attempting to address these warts.
Some of Bazel's design problems do feel like the outcome of a poorly executed requirements gathering process. Last time I checked, it assumed for some reason that C/C++ interface headers shared a common root folder with libraries, and if that root folder is not the immediate parent folder then it ends up including all files within that tree branch in its scope.
I also recall that the process of including system libraries that don't support Bazel is needlessly convoluted, and involved adding libraries and interface headers in separate targets just to be able to control visibility. This design oversight was specially disheartening because it made it quite clear that the use case of supporting external packages benefitted from zero attention.
Monorepos do have advantages, but Google fundamentally uses them as an excuse to fail ot understand modularity and composition. That's a fatal mistake.
I think you are missing something about both tools: the BUILD language is still quite high level, and closer to CMake/Meson than Ninja. The Nix language is also the higher level, though not that high level.
The actual "meat and potatoes" of Nix is the derivation graph. I think/hope Bazel has an equivalent, but I'm not sure how exposed it is (advertise your layering, people!). This derivation graph is simple simple as hell---each derivation is a path to an exe, arvgv, environ, etc., and derivations output paths and can depend on the outputs of derivation. This is what Meson/CMake/etc. should learn to output.
In the general category of Bazel-alike build systems, I'd humbly submit that Pants v2 is worth taking a look at. It's a generic, high performance build system for large projects, implemented in Rust with a Python plugin API. The project had a large announcement today in fact! https://news.ycombinator.com/item?id=24911148
To be fair, gyp is quite awful. I mean, a low-level language to generate project files for a few IDEs from a definition file where IDE-specific settings had to be added anyway? While the source of truth is left out of the equation, rendering those project files read-only and forcing developers to reconfigure their IDEs each and every single time they updated the project? This concept seems to have been thought out by masochists.
Do you know whether/how it can deal with database tables dependent on source .csv files?
My job involves lots of ETL-type work, where the extracted source file(s) for a particular table might have names that include the extract date or version, and a table might depend on other tables. I've been looking at make for dealing with the process, but automatic dependency management seems hard, and dealing with the variable source file names even harder (possibly because I'm not more than a make novice).
Out of curiosity, how difficult did you find the migration process? Especially in terms of having to restructure the project.
I have no direct experience, but from what I've heard Bazel is very opinionated about structure. Which makes it particularly challenging to migrate pre-existing projects that may not conform to its view. But again this all hearsay, so I'd like to hear from someone with direct experience.
It was pretty easy for the software I was migrating because it was already in a monorepo and we were already using some very similar conventions. Anyone writing Java code that uses Gradle or Maven shouldn't have much pain migrating. I'm acutally currently supporting both Gradle and Bazel on the same backend code base to give another team (who depends on my code) a chance to gracefully migrate.
The "hardest" thing to migrate has been C and Python code. For these the package for something is based on an absolute file path from the root of your repo. So `company/foobar/thing.py` is `from company.foobar import thing`.
I find this very useful personally because it's now standardized through the codebase (no one just plops a setup.py somewhere and does whatever they want) but it makes the migration difficult because you need to change the paths.
There's some work being done in rules_python and in cc_library/cc_binary that lets you control this behavior and hack it to align with your current company standards though which is really nice to use for a gradual migration.
Zim was inspired by all of these and has drastically cut build times for our team. We use it to build hundreds of Go, Python, and Node artifacts for use in AWS. https://github.com/fugue/zim
