
Makefiles – Best Practices - lelf
https://danyspin97.org/blog/makefiles-best-practices/
======
girzel
I recently decided it was time to get a better understanding of how makefiles
work, and after reading a few tutorials, ended up just reading the manual.
It's long, but it's very, very well written (a good example of one of the Gnu
projects biggest strengths), to the point where just starting at the top and
reading gives an almost tutorial-like effect. Just read the manual!

~~~
chubot
FWIW I also read the GNU Make manual, and based some code for automatic deps
off a profoundly ugly example it had. Then later people on HN showed me a
better/simpler way to do it.

[https://news.ycombinator.com/item?id=15060149](https://news.ycombinator.com/item?id=15060149)

[https://www.gnu.org/software/make/manual/html_node/Automatic...](https://www.gnu.org/software/make/manual/html_node/Automatic-
Prerequisites.html)

After reading the manual and writing 3 substantial Makefiles from scratch, I
still think Make is ugly and, by modern standards, not very useful.

In my mind, the biggest problem is that it offers you virtually no help in
writing correct parallel and incremental builds. I care about build speed
because I want my collaborators to be productive, and those two properties are
how you get fast builds. GNU Make makes it easy to write a makefile that works
well for a clean serial build, but has bugs once you add -j or when your
repository is in some intermediate state.

~~~
klodolph
> In my mind, the biggest problem is that it offers you virtually no help in
> writing correct parallel and incremental builds.

That’s interesting, because in my mind, parallel and incremental builds are
the main features of Make, the features that it is best at, and all other
features are secondary.

It sounds like your problem is with the correctness part. Make gives you no
tools to enforce that your build rules are actually correct. Very few build
systems provide any help here. What you are looking for is hermeticity. Bazel
does this by sandboxing execution of all the build rules, and only the
specified rule inputs are included in the sandbox. I recommend Bazel.

Otherwise, it is up to you to get your build rules correct, and switching
build systems won’t help in general (although they may help in specific
cases).

I find it surprising that you talk about using Make for a clean serial build,
because if you want a clean serial build, you might as well use a shell
script. Make’s only real purposes are to give you tools for incremental and
parallel builds. Nearly any other tool you replace Make with will either have
you sacrifice incremental/parallel builds or will give you the same
hermeticity problems you would encounter with Make. Replacing Make, the main
paths I see are towards improved versions of the same thing (e.g. Ninja, tup),
completely redesigned versions of the same thing (Ant, SCons), systems and
languages which generate makefiles (e.g. autotools, CMake), and the new wave
of build systems which provide hermeticity (Bazel, Buck, Pants, Please). This
last group is a very recent addition.

Mind you, Make is old and not especially well-designed, and it has plenty of
limitations, but it’s good enough at incremental/parallel builds that it has
stuck around for so many decades. Make is good enough at what it provides that
replacements like Ant, SCons, Tup, Ninja, etc. don’t seem like much of an
improvement.

~~~
jschwartzi
As I recall, the guy who developed Make wrote it in a weekend.

~~~
klodolph
And never expected it to be so successful, and realized that he couldn't fix
some of its problems because people were already relying on it.

------
nrclark
GNU Make is an awesome task-runner. I use it all the time, for all sorts of
things. It's my shell-script replacement, and I write one-off Makefiles
frequently. It's got a nice macro-processor, some helpful string manipulation
tools, and expresses task dependencies perfectly. The manual is also very well
written.

But if you're writing a software package, consider using CMake, Meson,
Autotools, or something similar. Unless you're a superhuman, any of them will
handle the corner cases better than you can. Especially the cross-compilation
corner-cases. It's extra work, but the people who need to build and package
your software down the road will thank you. Your artisan Makefile might be an
elegant work of art, but what does it do when somebody wants to build from a
different sysroot? Does it handle DESTDIR correctly?

~~~
scottlocklin
I haven't found a reason to not just use Make yet when starting from green
fields, including some very big projects. In fact, I have often been annoyed
at some projects picking something like CMake for no reason beyond it's "more
advanced," but which ends up just being an extra dependency I have to fetch
and install.

If I were to pick one of the above for building large projects on linux
variants, which one would you pick?

~~~
nrclark
For me, I mostly work in embedded Linux - that's Linux for routers, custom
hardware, sometimes a Docker image, etc. That means lots of packaging work.
Sometimes it means build machines that can't even run the compiled binary. I
compile libraries for platforms that the original author would never have
imagined.

In that space, there are lots and lots of details that need to be "just so".
Lots of specifics about the cflags and linker flags. Requirements on where
'make install' puts files. Sometimes restrictions where an output binary wants
to search for config files (spoiler: use sysconfdir).

Out of all the libraries/programs that I've had to package, autotoolized
projects are the easiest to handle by far, followed by CMake projects.

Hand-written Makefiles are usually painful to a packager in one way or
another. The most common ones being hard-coded CC/CXX/CFLAGS/CXXFLAGS, non-
standard target names, and non-standard/incomplete usage of directory
variables.

I know that Autotools is crusty in a lot of ways, and that the learning curve
is steep. It's a nasty mix of Perl and m4, and runs lots of checks that don't
matter at all. It's also what I use for all of my own libraries and programs,
because the result is worth the pain (for me).

So for any program that I expect more than two humans to ever compile, I'd
recommend Autotools if you're a perfectionist and CMake if you're not. Within
Autotools, I'd recommend Automake and Autoconf if the language is C/C++, and
just Autoconf/Make otherwise. (But be sure to follow the Makefile conventions:
[https://www.gnu.org/software/make/manual/html_node/Makefile-...](https://www.gnu.org/software/make/manual/html_node/Makefile-
Conventions.html)).

I wouldn't recommend a hand-rolled Makefile unless you're the only one using
it, the build is really weird for some reason, or unless it's a wrapper around
some other build tool.

~~~
scottlocklin
Thank you for the detailed response!

------
SamWhited
I really wish blog posts like this would at least mention that they are going
to use GNUMake extensions. There's some good stuff in here, but just calling
it "make" and leaving it at that is misleading. It's not going to work on my
minimal (non-GNU) Linux boxes that mostly run NetBSD make (bmake in many
distros) or busybox style tools. I know a very small minority of people do
something like that (or maybe not, what does Alpine run in containers?), but
it seems worth at least a quick footnote to say that most things in here won't
be portable.

~~~
rcfox
I doubt most people even know that there are different versions. (I didn't!)

~~~
dmix
It's pretty common knowledge that there are differences from the GNU package
set and linux package sets.

I've been exposed to this multiple times: from downloading GNU packages using
Homebrew on OSX, downloading different packages on Android, the variety of
options available to ArchLinux users, etc.

Anyone with basic Linux knowledge should be well aware of this. From a minimum
of following tutorials and having the basic command flags not work on common
terminal programs.

~~~
rcfox
I don't agree that "basic Linux knowledge" requires knowing which commands
have different versions. You can be intimately familiar with the Linux kernel
and never step outside of the GNU environment.

I've never used OSX, and have no desire to compile things on my phone.

And I know that there are non-GNU versions of some tools, like grep, but
whenever I've attempted to use them, I find them lacking some key feature that
I always use. I can't be bothered to learn the entire set of GNU-improved
tools, since I'm always just going to be using the GNU version anyway.

------
jedimastert
For those interested, there's also this writeup of making portable makefiles
on nullprogram[1]

[1]:
[https://nullprogram.com/blog/2017/08/20/](https://nullprogram.com/blog/2017/08/20/)

------
swills
Please add to this list "If you use GNU Make specific syntax, use the name
GNUmakefile".

[https://www.gnu.org/software/make/manual/html_node/Makefile-...](https://www.gnu.org/software/make/manual/html_node/Makefile-
Names.html)

------
vbernat
> Using the assignment operator = will instead override CC and LDD values from
> the environment; it means that we choose the default compiler and it cannot
> be changed without editing the Makefile.

People can use `make CC=clang` to override assignments. That's a pretty common
use.

~~~
BeeOnRope
Came here to say this. The "strength" of user specified variables depends on
where they come from: the environment is weaker than those specified as
command line arguments - which can lead to confusion.

------
tedmiston
There are so many gotcha with makefiles yet we still use them regularly. I'm
considering moving on to something like Taskfile [1], though I haven't tried
it yet.

[1]: [https://taskfile.org/](https://taskfile.org/)

~~~
igor47
I've been using invoke [1] which has allowed me to give my projects nice UIs
while remaining in the primary language. If I were working on Ruby I would
stick with Rake, even though I think invoke is better.

[1] [http://www.pyinvoke.org/](http://www.pyinvoke.org/)

~~~
tedmiston
I really like Invoke / Fabric for Python projects, though it feels weird to
use in non-Python projects when it comes to dependencies. [Now I have to
explain to non-Python people things like virtual envs, pipenv vs pip vs pipsi,
pyenv, etc.]

Just curious if you install Invoke individually for each project or keep one
global install?

I suppose I could always git exclude the tasks.py Invoke files.

~~~
learned
I usually keep Invoke globally installed that way I can use it for setting up
and interacting with pipenvs externally. Most of my calls beside the initial
setup make use of 'pipenv run <command>' that way I never actually have to
navigate inside of the virtualenv for most cases.

------
DblPlusUngood
A particularly tricky task with GNU make is automatically adding target
dependencies on header files and handling updates to them (gcc's -M and -MMD
switches). It would be great if the article explained those best practices,
too.

~~~
jgrahamc
[https://www.cmcrossroads.com/article/tips-and-tricks-
automat...](https://www.cmcrossroads.com/article/tips-and-tricks-automatic-
dependency-generation-masters)

~~~
AceJohnny2
Thanks John, your book is the most-referred to on my office shelf ;)

Edit: [https://nostarch.com/gnumake](https://nostarch.com/gnumake) (via
[https://blog.jgc.org/2015/04/the-gnu-make-book-probably-
more...](https://blog.jgc.org/2015/04/the-gnu-make-book-probably-more-
than.html))

~~~
mturmon
I also recommend this book. The series on “meta programming make” is also
quite good: [http://make.mad-
scientist.net/category/metaprogramming/](http://make.mad-
scientist.net/category/metaprogramming/)

~~~
jgrahamc
Ooh. That's great. I had not seen that!

------
kstenerud
I've been using makefiles for decades, and finally decided to try something
more modern.

It turns out that CMake has come a LOOOONG way, and is almost nice to use,
aside from the fact that it's horribly complicated. But with some basic
templates to work from, it's actually pretty easy to get a project started.

[https://github.com/kstenerud/modern-cmake-
templates](https://github.com/kstenerud/modern-cmake-templates)

~~~
shrimp_emoji
Or just use Rust. :3

Cargo handles makes with very simple TOML scripts, and it's the more modern
language, to boot.

~~~
kstenerud
I've been planning to, but unfortunately it still lacks 128 bit float and
decimal types, which I need.

~~~
burk96
You may be interested in this crate which adds decimal types
[https://crates.io/crates/rust_decimal](https://crates.io/crates/rust_decimal).
Here is another that I have not used but appears to add a f128 type
[https://crates.io/crates/f128](https://crates.io/crates/f128).

------
solatic
Make is a very underappreciated tool outside C/C++ circles.

I'm currently using it in an environment which uses Concourse as the CI
tooling - Concourse takes care of version and dependency management at its
level, and Make fills in the gaps for fetching the latest versions for local
development environments. Because Make won't re-download files for already-
made targets, local build cycles are fast after initial setup. Concourse can
then re-use these Makefiles by symlinking the resources it manages before
calling make, and Make won't try to re-download the resources. If you're not
concerned with portability, you can even use make to fetch (and clean) Docker
images since they live in predictable directories on Linux (but not OS X).

More people ought to approach the tool with an open mind.

------
aaaaaaaaaaab

        CFLAGS := ${CFLAGS}
        CFLAGS += -ansi -std=99
    

What's the point of the first line? Why not just:

    
    
        CFLAGS += -ansi -std=99

~~~
dewhelmed
Because CFLAGS may not be set in the environment, resulting in concatenation
to an undefined variable error.

~~~
jgrahamc
No such error occurs.

    
    
        $ cat Makefile
        FOO += foo
    
        all: ; @echo $(FOO)
        $ make
        foo

~~~
dewhelmed
My bad, I felt like I've seen that error pop up before in my Makefiles, but I
should have double-checked before commenting.

------
jjeaff
I have never worked much with compiled languages but I recently started
creating gnu makefiles in all my projects. I read someone's article, maybe
from here, that gave me the idea.

I have been putting everything in it. From the terminal commands needed to
spin up the cloud infrastructure to all the docker build commands and of
course all the build tasks like gulp and composer.

Even my docker compose commands have been replaced with "make up" and "make
down". It is really handy and has really streamlined a lot of processes when
it comes to local dev environment issues with different developers.

I basically just use it as an easy way to organize shell scripts.

The next step is to convert my ci/cd pipeline to just use specific makefile
commands from the same makefile so that everything is more portable and I can
easily execute the exact same process locally when desired.

------
corndoge
Next up: Automake, best practices

Publication date 2025

5000 pages

~~~
jandrese
Or a single page with a single word:

No

------
hzhou321
Best practice is to keep it simple. For example, if the Makefile is written in
such a way that lazy set or immediate set doesn't matter, then it is not
complex at all. But these facilities are there for achieve logic. Complex
logics are complex, especially when they are entangled. For example, we often
use build tools to generate Makefile. Some of the logic is implemented in the
build tool; some of the logic is implemented using Makefile -- of course the
result is complex. There is no rule of thumb on how to organize complex logic,
that is what programmers are paid (so much) to do. Respect your job and treat
complex logic carefully, then you won't make too much a mess -- at least do
not complain about it after making one.

------
heyjudy
If you're using GNU make, as the author appears to be doing, makefiles should
be named _GNUmakefile_ instead of _Makefile_ so as to not confuse the user or
the tools by conflating GNU make with BSD make or other make dialects.

------
ufo
Does the "?=" operator behave like ":=" or like "=" ?

~~~
jgrahamc
Both

    
    
        $ cat Makefile
        $(info FOO $(flavor FOO))
        FOO ?= foo
        $(info FOO $(flavor FOO))
    
        BAR :=
        $(info BAR $(flavor BAR))
        BAR ?= bar
        $(info BAR $(flavor BAR))
    
        BAZ =
        $(info BAZ $(flavor BAZ))
        BAZ ?= baz
        $(info BAZ $(flavor BAZ))
    
        $ make
        FOO undefined
        FOO recursive
        BAR simple
        BAR simple
        BAZ recursive
        BAZ recursive
    

If undefined then becomes recursive, otherwise the flavour is preserved.

~~~
wahern
Make variable expansions are always recursive.

For what you're describing I think the proper term as used by the GNU Manual
is deferred. In other languages the phrase lazy evaluation is common.

The one thing I've always had trouble remembering with GNU Make is precedence.
Variables defined as command-line arguments (make FOO=bar) override
assignments, but environment variables (FOO=bar make) only override ?=
assignments. != is both immediate (the shell expansion) and deferred (the
result of the shell expansion).

I _feel_ like there's some inconsistency when variables are inherited across
recursive invocations, but in a simple test with make 3.81 (macOS) I couldn't
find any. Maybe my suspicion is just a byproduct of my weird inability to
remember the precedence rules.

I normally try to stick to portable make these days, anyhow. Between the
ancient version of GNU Make on macOS, OpenBSD Make, and NetBSD Make, you're
mostly stuck with POSIX syntax. If you add Solaris' default make, and
_especially_ if you add AIX' make, you can't rely on any extensions at all.

For me sticking with portable make is easier than installing and maintaining
GNU Make on every flavor of operating system I test on, and definitely easier
than installing autotools or installing and maintaining the most recent
version of CMake.

~~~
jgrahamc
The GNU Make manual defines two variable types: recursively expanded and
simple (see:
[https://www.gnu.org/software/make/manual/html_node/Flavors.h...](https://www.gnu.org/software/make/manual/html_node/Flavors.html)).

------
dilawar
This is pretty cool summary. Thanks.

------
brookhaven_dude
What about using Python (with system calls) to build the project? Much more
understandable than Makefiles.

~~~
dbcurtis
OK, so I am as much of a hard core Pythonista as you will find. But I don't
think it makes any sense to build C projects (or many other languages) with
Python instead of makefiles. C programmers know make, the makefile idiom has
been evolving for 40+ years. A seasoned C programmer is going to look at an
idiomatic makefile and find it much more readable than some rando doing some
one-off Python script to control a build. And getting build dependencies
resolved correctly so that you can only build what is need is not trivial.

What I truly hate is all the IDE's that think they can do a better job than
make. This is the curse of the embedded world. Building an embedded project
requires calling particular tools, with peculiar flags, and I hate with a
passion IDE's that obscure all of that in some crappy XML file that is git-
unfriendly. A well-structured makefile is your friend in many ways.

make is a powertool. Learn it and your life will be better.

~~~
htfy96
As a C developer I have to say there exists no idiomatic makefile -
configuring header file dependencies with -MMD/auto rebuild targets affected
by FLAGS change already makes your Makefile look like magic. As the project
grows, eventually you will need some kind of flexibility that makefile cannot
do well. In this case, explicit Python scripts become more useful that a bunch
of possibly implicit makefile rules.

~~~
kingosticks
Perhaps we need some kind of make8 tool to check for best practices and
prevent us writing these magic make files.

------
Orphis
Makefiles, Best Practices: Don't write Makefiles, use a higher level language
to describe your goal and some tool to execute it (either directly or through
generating a Ninja file).

The Make language is the assembly language of build systems. Do you really
enjoy writing assembly all the time?

~~~
RHSeeger
I have yet to find a build system I prefer over Make. I use Maven at work and
use a Makefile to run mvn. It lets me easily run a wide variety of commands
locally as part of my build.

~~~
Orphis
Sure, if you're running commands, but you're not building Java directly with
it. That seems reasonable.

------
umvi
"Makefile, Best Practices: Use CMake to generate them for you"

~~~
nomel
You joke, but I've personally never seen a hand made make build systems, in
any company I've worked at, that could handle "fuzzing" by deleting or
modifying random files in the build system and having the expected output.

They all required a "clean" to get back in order.

For those that maintain handmade make based build system of reasonable
complexity, I challenge you to "fuzz" it before each build. It will not be a
comfortable experience.

On the other hand, I've seen many generated makefiles that behaved correctly
that are totally unreadable.

