There are build systems where this guidance isn't terribly valuable.
With Bazel for example, you typically have build files in each directory, and a test target for each test class in that build. But bazel allows you to select and run targets in a collection of directories, select targets recursively, or by a variety of other criteria. Your CI systems runs targets selected implicitly, while each build expresses the test targets explicitly.
Some (not all) still applies. I just reworked the Bazel CI at $DAYJOB to simplify the entry points and just call `bazel build //...` And `bazel test //...` erc rather than a bunch of more specific targets. We've got caching etc effective enough that just testing everything is a viable default even in a tight dev loop.
Context: I tried and failed at convincing management in moving my work monorepo to bazel. We have around 5 core languages plus a bunch of dsls.
If you don't mind sharing, what languages do you build in your bazel workspace? Were you able to use hermetic toolchains for these languages without any major hiccups? Finally, do you use engflow or buildbuddy atop bazel?
Not the parent, but at my previous job we used lots of JVM languages, plus some python and a tiny bit of JS. We were not particularly successful in getting truly hermetic builds with it - a big part of was a lack of headcount actually working on it.
My main takeaway is this: if you're the only one in the room asking for bazel, it's probably a bad choice. Bazel requires a lot of investment to stay nice to use, particularly in a polyglot environment. I've experienced endless friction getting things to work "in bazel" that would otherwise be simple using language-specific tooling.
In my opinion, the benefits (caching, build speed, a single build tool) did not outweigh the costs (nonstandard tool means you are always operating in hard-mode compared to using language-specific build tools, it's a ton of work to make actually hermetic, devs often aren't familiar with it and don't like writing BUILD files, editor support is extremely lacking outside of IntelliJ).
$DAYJOB codebase is mostly C++ and Python with a bit of custom configuration DSL for code generation.
Getting hermetic Python was pretty straight forward, hermetic C++ for cross-compilation to our target hardware was a bit more challenging but not too bad.
We also have VHDL which is built outside of Bazel for now. That's a job for 2024 and, I expect, will be a much bigger challenge.
We don't use BuildBuddy or EngFlow as we have some special requirements that made it not an option. I have a couple of personal projects using Buildbuddy though and it works well.
FWIW, I have had more luck with Pantsbuild rather than Blaze.
The fact that it does dependency inference has made adoption far easier. I've found the community pretty active on Slack too.
Using it to build Python/Java/Kotlin/Go/Docker; generally pretty happy with it, though the Python support is the best and the Go support is the weakest.
Dunno, I really appreciate a modular makefile for various tests because it enables you to easily reorder them.
When you run cargo watch or nodemon with “debug”, you can comment out your “debug-command”, write a new one which immediately tests the part you’re cranking on, then the outer “make debug” test runner interface works exactly the same but runs a different order of the same tests so the new code fails quicker if there is an issue.
Definitely agree CI bloat control is crucial, just saying, if you go too “coarse grained” you’re unable to reshuffle the test chunks quite so easily for faster runs. Matters a lot less in Rust since the tests are quick, but in languages like Python, test order can make a huge difference for developer experience.
The three musketeers concept summarizes this well: https://github.com/flemay/3musketeers, I.e. your CI should call entry points in your build file and nothing else
Some of those are specific to the historical way C projects work, and others I disagree with, but a lot are still useful and weren't mentioned in this link:
build family:
all - build everything, but don't run it (this is an important thing). GNU says this should be the default, but sometimes I have `make default` only compile the main program, whereas `make all` additionally compiles some maintainer-only helper programs and such (which might take longer).
all-$FOO - for example, all-objects verifies that everything compiles without running the expensive linking step
gen - build just generated files. Might have subtargets for non-distributed, distributed-but-not-committed, and committed files.
docs - GNU names specific formats, but info is largely ignored by everyone else. Generating `man` pages may depend on building the program first (but beware cross builds ... should that mean such docs are committed!); other forms of documentation are generally unrelated to the main build but may rely on other tools (which must be built or even installed separately - beware versions generating different output!).
dist - build a source tarball; note that this often includes generated files not committed to the repo. Generally we defer binary tarballs to the package manager these days, but historically a lot of C makefiles *not* from the GNU project also built those under some name.
clean family:
(each of these depends on the previous)
mostlyclean - delete all but a few slow-to-build core libraries. This is probably mostly an artifact of poor dependency tracking, but not altogether useless in some form
clean - delete all normally-built files. This concept remains useful because we don't like junk lying around. Compared to `distclean` this does not delete configuration; like distclean it isn't supposed to delete generated files if they are distributed.
distclean - revert everything to exactly the state of the source distribution tarball (which, again, includes some generated files). People have kind of given up on this as a stated goal in favor of from-scratch builds but that doesn't mean the concept is useless.
maintainer-clean - delete all generated files, even ones that were in the dist tarball. Many non-GNU projects don't distribute generated files (since nowadays we assume it is reasonable to have bison etc. installed), and thus don't distinguish between distclean and maintainer-clean.
(using `git clean` is useful for the end user but is dangerous in scripts)
introspection family:
help - give a basic overview of what build targets are available
list-$FOO - for things like "list what programs and libraries will be built", "list what tests are available" etc.
Now, certainly some of these could be merged into fewer entry points, but keep in mind the verbosity.
I’m not sure about this. It’s convenient, sure, but in my experience always entails more complicated build systems that are harder to understand when they go wrong.
With Bazel for example, you typically have build files in each directory, and a test target for each test class in that build. But bazel allows you to select and run targets in a collection of directories, select targets recursively, or by a variety of other criteria. Your CI systems runs targets selected implicitly, while each build expresses the test targets explicitly.