The author mentions it, and I highly recommend anybody interested in build systems read the seminal paper Build Systems A La Carte [1], which breaks down various build systems (including Excel!) into their component concepts, and along the way finds a new feature-space that hadn't been explored.
Instead of relying on modification-time comparisons between dependency and target output, use hashes of the relevant "ingredients" of building a target (cmdline, arguments, meta-data about input files etc.).
I’m a little confused by some of this (mtime vs manifest) because Ninja does rebuild things if things like the command line changes, doesn’t it? Or have I misunderstood how this tool works forever…
Aside: the make job server is very simple and UNIXy. There's a pipe shared between `make` and its subprocesses. `make` writes `N-1` bytes to the pipe. When a job wants to do parallel work, it reads a byte from the pipe. When it's done, it writes a byte back to the pipe.
(Both rustc and cargo support the make job server)
It's really an elegant design, which is nice, other than the way that information is passed into the child process (roughly: MAKEFLAGS=" -j --job-server=3,4"), which requires you to argparse out the fd numbers.
It would be nice if a new, less cumbersome, environment protocol could be adopted.
I dunno if they'll accept it but I might actually submit a PR to add this to n2, because I also would very much like "ninja with jobserver" and it being written in rust it should be fairly easy to add since the code cargo uses for jobserver support is broken out into a very simple library.
Both. But server is more important to me usually (ninja running cargo, in particular -- but also sometimes ninja running make or ninja running ninja). Technically you can obviously kinda trick client-only support into being server support by wrapping ninja in a jobserver like make, and letting that jobserver cascade down, but that's pretty gross.
So while client support is nice, it's not really sufficient. I actually don't think I have much use for client-without-server at all, the main configuration I need client for at all is ninja->ninja.
There's also that kitware's is a one-purpose fork that's just kind of a pain to install in CI because lack of packages, but it's pretty easy to `cargo install n2` on a machine where I'm building rust anyways, and cache the CARGO_HOME so it doesn't build every time.
To be honest, with the fact that compilers are no longer single-threaded almost as a rule, something like jobserver seems like an essential feature in a build system now. It's basically impossible to meaningfully limit concurrency without some level of coordination or fuzzy imperfect indicators like load average. I hope other compilers follow rust's lead here and implement client support eventually.
(and then hopefully a more robust common protocol can evolve out of it)
It's not explicitly stated anywhere, but this appears to be by the original author and one of the later maintainers of the original ninja, which probably puts it ahead of some of the other rewrites out there in terms of its likelihood of adoption.
What are the benefits of the C implementation? I noticed the adherence to POSIX command-line conventions, but that doesn't sound like a good enough reason for a reimplementation in a different language.
I guess the ability to work in really restricted/minimal environments where you only have a simple C compiler like tcc, and nothing like LLVM. Or paranoid environments where you can only use CompCert. Something like that.
I'm not talking about embedded, I'm talking about e.g. intentionally minimalist systems that have to be self-building, or systems that have to be built with a formally verified compiler (which does not exist for C++), or anything else like that.
I love build systems and have tried many of them. Redo is fun to play with. It is really easy to understand.
There is some messy reality that causes redo the struggle. For instance it doesn't natively support a single command that produces multiple build targets. You end up needing workarounds. I also find that redo starts to feel like Rails where your logic is spread around is many little files. Each file is easy to understand, but it takes a while to build the full picture.
I also created a minimal Ninja implementation in Rust some time ago. My goals were to implement it in terms of the Build Systems a la carte paper. Of course, hard to compete with the original ninja authors who obviously understand it much better. For example I used a separate lexer and environments, which got a little annoying and is something explicitly called out in their design.
(n2 author here) This is really cool! I am sorry if weird ninja decisions (like not being lexer-friendly or phony being underspecified) caused you grief, the whole thing was really built as a quick hack before I understood the problem space.
None of these have a historical ETA estimator, which has been requested in ninja maybe a decade ago. I hacked a few lines to show something, but would prefer the correct solution.
(n2 author here) ETA is pretty hard in the presence of parallelization unless you make big assumptions like "all pending build tasks will be able to run in parallel". If you had any good ideas here I'd love to hear them.
[1] https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
(an understanding of Haskell is recommended to understand the implementations, but not required to understand the concepts)