
Recursive Make Considered Harmful (1997) [pdf] - Tomte
http://aegis.sourceforge.net/auug97.pdf
======
ajross
Yup. But imake is here now and we can all rest easy.

OK, maybe not. But this here autotools cleanly handles all those crazy
portability edge cases and gives you a nice structure to boot. It's the Right
Thing.

Sorry about that. But check out scons. That's clearly the way things should
have been done in the first place.

Late breaking news: supporters of "Ant" in the Java world were just deposed by
something called "Gradle"! We'll have experts on shortly to explain why either
of these were needed in the first place given that javac handled dependencies
automatically in its very first version.

But yeah, maybe python wasn't as clean as we hoped. Check out cmake! Here's
the new hotness for sure. Pay no attention to the really awful programming
language it embeds, it does windows!

\----

I mean, yeah, recursive make, specifically, isn't a great idea. And for large
projects, make implementations have performance issues.

But make is a tool that does two things and does them simple and well: it
tracks dependencies between files, and it lets you write simple command
sequences in an obvious language to fulfill those dependencies.

And until the authors of all these other tools get their heads out of the
clouds and stop trying to fix the OTHER problems with make while refusing to
preserve these attributes, make isn't going anywhere.

~~~
jjnoakes
> But make is a tool that does two things and does them simple and well: it
> tracks dependencies between files, and [...]

Last I checked, make only handled dependencies with multiple inputs and a
single output.

If your build step's command produced more than one output and you wanted to
model that in your DAG you had to resort to some questionable incantations
that never really worked perfectly (or rely on non portable constructs).

~~~
cmm
firstly, the upcoming version of GNU Make will indeed support multiple outputs
in non-pattern rules (pattern rules always supported this, but pattern rules,
uhh, have their own problems), using &: instead of :, resolving a bug from
2004. don't you just love the solid pace of those old GNU projects.

secondly, working around this was never very hard, albeit ugly: pretend your
rule builds just one of the targets and have the other products depend on that
one and simply touch themselves up to its timestamp.

~~~
jjnoakes
I disagree with the characterization that this is "not very hard" (and my
initial assessment was "questionable incantations" which I think is also
accurate):

[https://www.gnu.org/software/automake/manual/html_node/Multi...](https://www.gnu.org/software/automake/manual/html_node/Multiple-
Outputs.html)

Definitely not handled "simple and well" as posited by the OP.

~~~
cmm
the horrible contortions listed on the linked page are required for the
pathological "some of the targets that are built together got deleted, but the
primary one did not" case, which in my experience Does Not Happen (or when it
does, it is a symptom of deeply suspect practices -- like shitting where you
ea^W^W^W^Wkeeping build artifacts and sources in the same directory, or
relying on manually deleting artifacts to force a rebuild instead of getting
the bloody dependencies right in the first place, etc.)

------
tonyg
Relevant:
[https://apenwarr.ca/log/20101214](https://apenwarr.ca/log/20101214). Code:
[https://github.com/apenwarr/redo](https://github.com/apenwarr/redo). It's
excellent!

~~~
gonzus
And, sadly, it also doesn't handle correctly / easily the case of a rule
building two or more outputs (think yacc producing a .h and a .c file): more
contortions are required.

------
hedora
Many people miss the key point of this article:

Break your make build up by recursively including make files instead of
recursively invoking them.

~~~
qcoh
I tried this once with multiple components (each being built into a static
library), components "exporting" public includes (with correct dependencies
across component boundaries) and tests for every component (as well as
integration tests) and my conclusion: cmake it is.

------
pcwalton
One problem with non-recursive make for very large projects is that make can
take a long time to parse and interpret the uber-Makefile. Even though this
paper provides some advice for reducing the parse time, it isn't enough for
really large projects.

Android suffered from this a lot in the early days. I always wondered whether
the initial Android team had read this paper and got carried away...

~~~
cmm
it's not the parsing per se that takes time on modern computers, it's opening
and reading in all the scattered included makefiles. also what specifically
Android build system is doing behind the scenes is, it runs a python script,
which in turn runs find (or maybe several, I don't remember), to gather all
the makefiles to include.

but parsing, once all the text is in memory? peanuts.

~~~
evmar
Android's build system now generates Ninja files, which are semantically very
close to non-recursive Makefiles, so it's a real-world example you can examine
and measure.

Android's build files currently add up to 1.5gb of text(!!!), which even with
a fast parser take some time to load (~10-20s).

It's discussed further in this thread (in which among other things I suggest
that maybe something is going wrong when you have 1.5gb of text):

[https://groups.google.com/d/topic/ninja-
build/htlu9akobq0/di...](https://groups.google.com/d/topic/ninja-
build/htlu9akobq0/discussion)

[disclaimer: I am the author of Ninja]

------
bjourne
People who defend make (or its evil companion autotools) are suffering from
the Blub Paradox.
[http://wiki.c2.com/?BlubParadox](http://wiki.c2.com/?BlubParadox) They likely
have never used any better build/configuration system so they think make is
the best there is. Discussing build systems is like discussing programming
languages. If one party has only ever used Java, then they will never get it.

Personally, I switched to WAF a loong while ago and have never looked back.
Perhaps it is not the best build system there is, but it is leagues ahead of
autotools/make.

------
zxcvbn4038
I think these “<whatever I don’t agree with> considered harmful” articles get
a bit tideous after a while. For any of them you can just as easily write a
counterpoint “<whatever you prefer instead> considered harmful” with just as
much passion and merit. Mentally I tend to rewrite these as “<bob or jane> are
mad at <goto, recursion, forth, puppies, kittens, etc>” and that is probably
closer to the truth. They are opinion pieces that allow people to vent to a
crowd.

If this comment had a title it could be “Considered Harmful articles
considered harmful” or “Bob is mad at considered harmful articles”. Someone
else could come along and write “Lack of considered harmful articles
considered harmful” with equal merit I’m sure. ;)

I’ve made a point the last few years to put at least one goto in all my code
just to mess with the junior devs, though in practice the junior devs aren’t
familiar with the original material so its more the mid level devs that will
explode into green flame or curl into a ball when they find my goto. Senior
devs couldn’t care less if they find a goto.

~~~
braythwayt
For those who don't get your reference, here is the canonical “Considered
Harmful” Essays Considered Harmful:

[https://meyerweb.com/eric/comment/chech.html](https://meyerweb.com/eric/comment/chech.html)

~~~
zxcvbn4038
That is a really great link, thanks for posting it! Something I had not
considered were these occurring outside comp-sci circles. Makes me cringe.
“Sit down sir! Your theory of of spatial relativity is so wrong it’s
considered harmful! Hurmph! Hurmph!”

------
tomohawk
One thing I love about go is that it builds all the things quickly and in one
go.

For other languages, once I found rake, I never looked back. It's so powerful,
and unlike make, you don't have to drop down to other languages to do things.

~~~
akkartik
In fact, slow builds were the #1 pain point Go was designed to address:
[https://talks.golang.org/2012/splash.article#TOC_4](https://talks.golang.org/2012/splash.article#TOC_4)

------
dboreham
Developers of yarn must have read this.

------
bleair
30 years on and I've used make, imake, autotools, cmake, ant, gradle,
boost.jam (because every C++ library really should have its own individual
custom build tool-chain), scons, ninja, and maybe the least bad was a very
nice system internal to ILM (though it was purpose written for building C++ &
python targets). These days I'm using bazel the most, and while it can be
forced into working, it's pretty miserable (and requires support of about 5
people on a dedicated build team). It does not bring software developers joy.

A group/language/team decides "that's it, I hate build tool X, we're going to
solve this problem", and they go build their thing, but they care about
different details and make some things easy, handle some things automatically,
and leave other things as "just don't care" with the resulting tool ends being
just like all the previous ones.

I'd be willing to wager lots of money that in future years there will be many
more build systems, and they too will perfectly match their predecessors.

[https://www.xkcd.com/927/](https://www.xkcd.com/927/)

~~~
humanrebar
Boost seems to be switching to CMake for what it's worth.

~~~
bleair
Interesting.. I did not know that. My googling shows that there has been an
announcement with an intention to adopt cmake (~ 2 years ago)

[http://boost.2283326.n4.nabble.com/CMake-Announcement-
from-B...](http://boost.2283326.n4.nabble.com/CMake-Announcement-from-Boost-
Steering-Committee-tt4696935.html)

It appears further discussions are underway, but I don't know what that means
in terms of how quickly they will switch.
[https://groups.google.com/forum/#!topic/boost-
steering/5ifzu...](https://groups.google.com/forum/#!topic/boost-
steering/5ifzupxKINI)

In terms of bjam vs. cmake... I get why the boost library has needed a custom
build tool chain (if you're curious just look at the complexity in headers in
trying to work around various template bugs in different compiler versions, or
the many many many ways to request naming of the generated libraries). I
should have been less snarky in my first post. If boost does migrate to cmake
I predict it will be quite a gnarly set of cmake logic.

------
jeffrallen
It still is.

~~~
mistrial9
offhand, graph traversal is a lot more common now than twenty years ago

