
Build System Rules and Algorithms (2009) [pdf] - Jtsummers
http://gittup.org/tup/build_system_rules_and_algorithms.pdf
======
paulddraper
So many build systems have regressed in that they don't have incremental
process built into their core functionality, like Make did. Ant, Maven, Grunt,
SBT, Gulp, Gradle...basically everything sucks at this compared to Make.
Awful.

But anyone interested in a next generation scalable build tool, check out
Google Bazel. [https://bazel.io/](https://bazel.io/) (Ex-Googlers built
similar build tools elsewhere -- Buck and Pants -- but I'm most optimistic
about Bazel.)

Like Make, Bazel uses incremental tasks at its core. But where Make uses
timestamps, Make hashes inputs, allowing for shared caches. It's also been
optimized to work with Google-scale build definitions. In beta now, 1.0
planned for end of 2017.

~~~
chubot
What do you mean that Ant/Maven don't have "incremental process"? They don't
do incremental builds at all?

There is an additional distinction between "top down" and "bottom up"
incremental builds as mentioned in this article about tup.

NEITHER Make or Bazel do bottom up builds. They are top down. That means that
incremental builds requires O(n) stat() or hashes. You have to check if ANY
input is out of date.

In contrast, tup does O(1) work for incremental builds by using inotify.

In practice, I'm not sure how much this matters, but it is interesting -- and
it is the central point of the linked article. Ninja has a reputation for
being fast, and it also does top-down builds. As I recall the Linux kernel is
quite good at caching inodes for stat() calls, so you can basically stat
millions of files in under a second. (Windows may be a different story.)

But if you are using hashes rather than timestamps, the work to check if any
of a million files are out of date is much larger.

~~~
Jtsummers
inotify is only available on linux so `tup monitor` doesn't work on all OSes.
But other platforms have similar libraries so it's possible to port it.

~~~
gavinpc
Are there such libraries for Windows? I can't find my notes on this, and all I
remember is that `strace` looked the most promising (for a non C programer).

I actually switched from Windows to Linux because of Tup—although to their
credit, they have fixed the issues that I filed.

I still use Tup on Windows for production at work, and in place of `tup
monitor` I use a service based on .NET's file system watcher, and just tail
monitor the log.[0] The config looks like this:

    
    
        <job at="%HOME%\some\project">
          <watch>
            <path>script/*.js</path>
            <path>stylus/*</path>
            <path>src/**</path>
            <path>tools/**.cs</path>
          </watch>
          <do>tup --no-environ-check</do>
        </job>
    

It's not foolproof, since I have to maintain that list, but it's better than
the alternatives (whatever they are).

[0]
[https://bitbucket.org/gavinpc/reactor/](https://bitbucket.org/gavinpc/reactor/)

~~~
Jtsummers
[https://msdn.microsoft.com/en-
us/library/windows/desktop/aa3...](https://msdn.microsoft.com/en-
us/library/windows/desktop/aa365465\(v=vs.85\).aspx)

Best I can find. I was referring to kqueue and FSEvent though for free bad and
macOS.

Sorry traveling and phone only.

I wonder if the Linux on Windows stuffs will make it easy to use inotify on
Windows? I've only heard of it. Haven't researched how that works.

------
fizixer
I guess the only gripe I have with make is that it has a rigid syntax (that
includes tabs). Since what a Makefile eventually is, is a graph, I'd be happy
if that graph is specified as a JSON file, which automatically gets converted
into a valid Makefile.

Has anyone attempted that approach? simply put a JSON wrapper around make?

~~~
wtetzner
I agree that Make's syntax isn't ideal, but JSON doesn't seem like a move in
the right direction. If you're going to give Make a new syntax, why not pick a
more pleasant syntax?

~~~
hood_syntax
Curious what you mean by pleasant. JSON is not really "pretty" but it is
simple, and powerful imo. Is it the boilerplate that puts you off?

~~~
padator
You can use mk, the successor to make, which does not have the TAB requirement
(you can use TAB or spaces interchangeably).

Regarding JSON, it is more verbose than Make. You need to use quote around
strings, commands, everything. Compare:

foo.o: foo.c gcc -c foo.c

with

{ 'foo.o' => ['foo.c'; 'gcc -c foo.c'] }

~~~
klodolph
{ "outputs": ["foo.o"], "inputs": ["foo.c"], "command": ["gcc", "-c", "foo.c"]
}

