
Time for Makefiles to Make a Comeback - mpweiher
https://medium.com/@jolson88/its-time-for-makefiles-to-make-a-comeback-36cbc358bb0a
======
bsenftner
I've been using make files constantly since '76 when I first started
programming in (US) 5th grade. It has been one of my career amusements
watching the build systems come and go. I gave up talking about Make years
ago. There is a huge population of us that smile and get things done, while
others screw around with new, complicated, never-learned-the-past tools.

I use one of the earliest Make versions that barely does anything beyond
recognize rules, rule actions, and simplistic macros. I've seen this version
of Make used to build a feature animated film. Witnessing the versatility of
that feature film's multitude of software and renders and composites all built
through Make taught me Make is the only tool needed for any building of
anything.

I remember when MS introduced a Make with extensions, and many developers ran
over and fucked up their build environments, starting them on the path to the
complex manual required build mess most people have today.

I write my Make films by hand, still. It is that easy. And you really should
have a build environment that is that easy. When you need it easy, as in some
major deadline and crap is broken, you will thank yourself.

~~~
humanrebar
> There is a huge population of us that smile and get things done, while
> others screw around with new, complicated, never-learned-the-past tools.

They get the things done they've been doing all along. But build metadata is
very important and very hard to extract from Makefiles, especially as builds
increase in complexity.

You need things like search paths accessible to get IDEs, static analyzers,
software packaging systems, and other third party tools working with your code
base.

So make works great if the only things you're interested in doing are the top-
level make commands you define, but it doesn't do you any favors when it comes
to other things.

~~~
Walf
I don't understand your reasoning here. Make can call any program, and contain
as many build targets as you would ever need. If some task can't be
accomplished in Make itself or existing CLI programs, you write something that
does that one task well, then ensure Make runs it whenever needed, even if
that's every build.

~~~
humanrebar
You could write make rules to do about anything, sure. But you don't want to
have to rewrite those rules for every project. You want make modules that you
can reuse from project to project. Those are notoriously hard to get right.
And when they are done correctly, they are hard to share across organizations,
or even within large organizations.

In particular, I write a lot of C++. I want targets for gcov, g++, gtest,
dpkg, rpm, clang-format, clang-tidy, cppcheck, clang, CLion, Eclipse, ctags,
and include-what-you-use. Am I supposed to maintain each of those targets on
each of my repos?

It's simpler to have a standard place with all the needed source files,
binaries, search paths, definitions, etc., and just wire that data up to each
extra target.

~~~
Asooka
I also write C++ and am not aware how to comfortably automate all that with a
single build tool. Is it some sort of cmake extensions? Can you please share
your setup?

~~~
humanrebar
Some of those are built-in to cmake you can google "cmake clang-tidy" to find
some examples, for instance. The rest can be added as custom targets and/or
custom configuration options.

The point is that you can write reusable cmake modules you can ship with your
package manager or even git and a tweaked cmake search path. You can get there
with make, but it's more work and a bit of a pain to replicate and maintain.
In fact I often see teams with those setups put too many things in their repos
to avoid build tool pain. Or, perhaps more often, they just give up on ever
getting clang builds to work (for instance).

------
jgrahamc
I have a theory that the people who love make are the same people who love
macros and the people who hate make don't love them.

I wrote a book on make and I fully recognize that there are people who look at
what I've written there and think I'm completely mad:
[https://www.nostarch.com/gnumake](https://www.nostarch.com/gnumake)

Make's greatest irony is that it's a system for building files based on
dependencies but has no way of actually discovering dependencies. I wrote
about this for Dr Dobbs a while back: [http://www.drdobbs.com/architecture-
and-design/dependency-ma...](http://www.drdobbs.com/architecture-and-
design/dependency-management-the-irony-of-make/228701768)

And make has deficiencies that are a nightmare to work around and get right
(e.g. spaces in filenames, recursion, ...)

If anyone else out there is a make hacker like myself they might enjoy the GNU
Make Standard Library project:
[http://gmsl.sourceforge.net/](http://gmsl.sourceforge.net/)

~~~
llimllib
(disclaimer: I like make but not macros)

> And make has deficiencies that are a nightmare to work around and get right
> (e.g. spaces in filenames, recursion, ...)

Yes! we seem to be stuck in a local maximum with make. I wrote my own make
replacement, as have many before me, and learned the folly of my own ways.

Nowadays I use (and love) make despite its terrible handicaps, but I really
wish there were something fundamentally similar but a) less C-focused and b)
containing many fewer footguns.

~~~
lederhosen
Shake is similar to Make with some improvements (it is a Haskell DSL), I have
not used it, but it looks good.

[http://shakebuild.com/](http://shakebuild.com/)

------
Iv
If you compare Makefiles to the JavaScript ecosystem it will do so favorably,
but many things will.

Make is a good build system but it is a shitty deployment system. Yes, you can
do everything you want if you put enough effort in it as it is a complete
scripting system, something many alternatives are not.

It does not mean that it is a good idea to do so.

Recently a client made me begrudgingly try Maven. 'Yet another make system' I
thought. Yup, pretty standard one: 'mvn install' ,I'll compile everything and
will also do something make has never done for me: install dependencies.

Sure, it is doable to do this with a makefile. With a lot of efforts
multiplied by the number of platforms you target. In Maven you just add repos
and dependencies to your pom file.

So yes, there recent build systems that are worse than Make but it does not
mean that make is the perfect system with all the features we need.

It is minimalistic. It works. But it should not be your only option.

~~~
mikegerwitz
Make isn't intended to be used for dependency management. With traditional
`./configure && make && make install`, dependencies are detected by the
configure script, or manually provided by the user if they have custom library
paths. Dependencies are handled by your package manager or something else (be
it submodules in git, custom scripts, etc). There's a separation of concerns.

For generating those configure files, Autoconf is extremely powerful (and
really not that complicated once you have a basic understanding). If you use
Automake, you get all the standard targets (including `make dist`, which will
build your distribution archive, as is being done manually in this article)
generated for you.

~~~
Iv
Well ./configure does not install dependencies either. Even worse: it fails at
the first missing dependency instead of listing them. Making an automake
project to compile is actually a long and manual process.

Just today I had to compile an older version of gimp, just a 2 years old one.
Configure. Fails. Apt-get this. Configure. Fails. Apt-get that. Oh, no,
configure doesn't want obscure-package4, it wants obscure-package2, that has
been removed from your distribution's repos. Arrrrrr! Truth be told, I just
rebooted on windows and dowloaded a binary from an archive.

With maven, I can just tell it which release I want and it just fetches it and
all its dependencies, compiles them and installs them. Full auto. If I add a
lib in my project, it adds it automatically and the intellisense works, the
source navigation works.

Make is years behind.

~~~
mikegerwitz
> Even worse: it fails at the first missing dependency instead of listing
> them.

Yes, this is a fair criticism. Fortunately, many projects state their
dependencies. But not all do. If I'm already on a Debian system that has
another version of the package, `apt-get build-dep' is very helpful to get
most of the way there. Of course this is completely outside Autoconf.

Whether it fails immediately or not depends on how the author writes
`configure.ac'. Usually `AC_MSG_ERROR' is used right away, because it's
convenient. Instead, some authors choose to set a flag and fail at the end.

------
dmytroi
Make provides awesome UX: in most cases you just type "make" and stuff just
works, but there is a tradeoff: it's nearly impossible to write a
crossplatform make file. Why you ask? Try copying a file (or directory) from
one location to another, sure "cp", but it's -R is different on Linux and
macOS! And then comes Windows - there is no good built-in alternative to "cp"
as it's hard to make "xcopy" to ignore failures if target file/directory
already exist (it tends to ask if you want to overwrite it or not), WSL might
fix that in the future, but currently it's quite slow on IO side of things.
So, sure given enough time and kung-fu we can write crossplatform make files
... but even if copying one silly file is such PITA .. it's hard to advocate
for make in 2017. Probably the wisest solution is just using make to run
something more comfortable, for C/C++ it can be cmake, premake, tup, etc. But
then again: if I need to add something, like another build step, do I add it
to the tool of choice or make? It's a very hard balance to sustain.

PS. Copying files specifically is such huge PITA that cmake implements
crossplatform copy for you, just run "cmake -E copy $in $out" (can be added as
add_custom_command in CMakeLists.txt)

~~~
jlebrech
maybe projects should have their own ./bin folder for make to use and have
those command refer to the right native command.

mixing make with a modern scripting language would make this possible.

~~~
lederhosen
GNU Make has built in support for scheme

------
Blackthorn
Having put together a build system together for building a modular os using
make: no, it's really not time to make a comeback.

Make is simultaneously powerful enough to seem pretty good for use, but not
powerful enough to do anything outside its very narrow problem domain without
really ugly hacks.

~~~
abritinthebay
Agree. God forbid you have to maintain cross-platform compatible makefiles
either.

------
jph
For a better "make" try "do" (also known as "DJB redo").

It's designed by Dan Bernstein of crypto fame and implemented by Avery
Pennarun now at Google.

[http://apenwarr.ca/log/?m=201012#14](http://apenwarr.ca/log/?m=201012#14)

I will contribute $100 to any Rust leader who wants to start coding "do" in
Rust.

~~~
mitch9
This seems like a prime moment to ask for a review of
[https://github.com/sagebind/rote](https://github.com/sagebind/rote)

~~~
jph
Rote is a great accomplishment. IMHO it's superior to Make, Rake, Cake, etc.
Thank you for writing this!

If you're curious, take a look at Do. It's like all of the above, yet flipped
over on their heads. The value of Do is in the design, which is essentially
the _opposite_ of Make. Do leverages the composability of items, files,
scripts, artifacts, and typical Unix pipes.

I would be very interested in your opinions about Do, and if you want the $100
to jumpstart working on Do in Rust, let me know how to donate to you.

------
Aardwolf
I wish there was something as standard, ubiquitous, fast&light, dependency-
less, and adopted by the whole Unix ecosystem and beyond like make is now, yet
better (= less archaic, more "write once works everywhere", uniform single way
of depending on other projects, focused on the actual code/bins/libs and their
dependencies instead of specific packaging for specific distros, ...).

Not some "hype of the day" build system of course as that by definition makes
it a short lived maintenance burden rather than universal.

While make is universal, with makefiles now it's so bad, that even if you
write pure C90 code with zero dependencies (which should work everywhere),
users will still find reasons why you should add all kinds of platform
dependent stuff or things for a particular distro to your makefile...

------
overgard
Nonononono. There's a reason makefiles have been reinvented 400 times: they're
a nightmare. I don't love the alternatives, but they are pretty much all
universally better.

~~~
pippy
Dependency hell is one of the main reasons why developers are flocking to
bloated solutions such as election: a single monolithic dependency with
guaranteed cross functionalist behavior.

Most build tools are stuck in the 70-80's where a single library weighing
hundreds of kilobytes was a big thing. In comparison to saving a few kilobytes
vs spending hours of a developers time hunting down old versions of libraries
is a no-brainier. Especially when you take security into account. I've seen
.Net developers just give up in frustration and download dll files from those
dodgy download sites and put them into production distributions.

I personally loathe make. I've never had a good experience when using it.

------
arca_vorago
I just got done watching a video called "make for reproducibility in science",
which made a lot of sense. The impressio I get is people do weird things to
make and then hit edge cases and complain.

~~~
jswrenn
> The impression I get is people do weird things to make and then hit edge
> cases and complain.

Like not handling file paths that have spaces? [0] That's one hell of an edge
case.

[0] [http://savannah.gnu.org/bugs/?712](http://savannah.gnu.org/bugs/?712)

~~~
arca_vorago
I'm speechless at that link. I don't understand why something so basic isn't
fixed yet in such an important package. I guess I can see why qmake/cmake have
been eating the market share now.

For what it's worth, I am pretty sure one of my first BSD systems forced me
into using snakecase so that would explain why I never hit that limitation.
That said, make, being designed originally for C, and C being also known for
snakecase, and early linux also known for discouraging spaces in names, I can
see how this happened in the first place. What I don't understand is the lack
of an implimented fix.

------
whipoodle
Discover make, realize what a bunch of dumb hipsters everyone is who created
build systems after it, try to use it, see how deficient it is for the
problems you're trying to solve, realize why those dumb hipsters made all the
stuff they made, repeat. If Chesterton's Gate were a build system.

------
AndrewCHM
I'm confused

So its agreeable that the building pipeline of NPM calling some packer does
replace any-other-language calling any-other-packer, but what is the point in
replacing the role of `npm build`, with a makefile that just calls what npm
would of called anyways?

replacing make with npm with make-calling-npm?

~~~
ealexhudson
I wrote a bit about an approach I'm using, A Touch Of Make (ATOM), which helps
provide better developer UX across teams - particularly ones working on
microservices.

[https://www.alexhudson.com/2017/04/26/articulating-atom-
appr...](https://www.alexhudson.com/2017/04/26/articulating-atom-approach/)

You're right, replacing "npm build" with "make build" doesn't win you
anything. But that's only true on the small scale. In a service world, there
are lots of projects, each with different requirements. Some will have front-
end, some won't. Others will require a totally different build process. There
will probably be different languages involved.

Using make, you can standardise a lot of this. If you set up a coding standard
that after you clone a repo, "make dep" should grab anything the project
requires, then developers don't initially need to know whether that's calling
out to npm or composer or pip or whatever - it's just "working".

This is a much bigger win when you have a number of projects. The process is
standardised, so developers know it's only a two or three step build (or
whatever you've setup). They know how to do it, and when they need to look
under the covers, they can see how it works, and this knowledge is
transferable from one project to another because Make is universal.

I don't advocate doing big Makefiles - that's why I call this approach ATOM;
only use a _touch_ of Make. But used judiciously, it smooths out projects very
nicely. Not everyone needs that, though.

~~~
meddlepal
I tried this approach. It never works in the long run. The Python developers
wanted Make for the few commands they needed in their projects. The JVM folks
had Gradle. The Python folks refused to learn Gradle (because of some argument
like: rah rah rah JVM tools hard; Make easy! Make all you need. If Make isn't
enough your language is garbage).

So the JVM folks created a crappy Makefile that accomplished some of the basic
stuff to appease the Pythonistas. Meanwhile the JVM folks continued to use
Gradle directly and the Python folks who had to work on the JVM projects got a
shittier experience through the Makefile. As new tasks were added to build
process the JVM folks would just write custom Gradle tasks to accomplish the
job which would drive the Python folks nuts because they wanted shell scripts
or make targets invoking shell scripts or whatever.

My only take away from this experience was developer tooling sucks and there
is no one size fits all approach to build systems. Try to stick to a
convention within an ecosystem (e.g. all JVM projects should use one of
Gradle/Maven/Sbt), but don't try and get cute with making stuff more common
than it has to be.

~~~
Chris2048
You say the Pythonistas refused to use Gradle, but it also seems the JVM peeps
refused to let go of it?

Isn't that the problem? Isn't Make more general than Gradle? If the project
was primarily JVM, why couldn't the Pythonistas be forced to use whatever
build system was mandated?

~~~
meddlepal
You'd have to reinvent a lot of wheels to replicate what Gradle (or Maven)
does. For example dependency management, env setup and packaging. It would be
similar to asking a Python developer to give up pip, setup/disttools and maybe
virtualenv. In the end you want productive devs. Stripping away the tools they
are productive with is counterproductive.

~~~
Chris2048
But surely these are targets for make?

e.g a pip target that builds a python dep folder. You can do the same for Java
with Apache Ivy.

Is Gradle not a make like system with a lot of JVM-specific functionality
built in?

------
senatorobama
What do people think of 'tup'?

~~~
leafo
It's my favorite build system. A big advantage is automatic dependency
tracking. It picks up unspecified inputs and automatically rebuilds anything
further down the dependency tree. It also knows how to clean up files that it
no longer generates since it understands you removing and adding build steps.

I can do huge refactors and it cleans up files without any additional work
from me. The filesystem watch automatically builds things so I never have to
run it directly. I use it on my production server when deploying since it's
100% accurate at incrementally moving between different project states.

There are definitely some disadvantages, you have to work around its syntax
and limitations. For complicated build functions I've written the config in
Lua and it works fine.

------
verytrivial
Friends don't let friends write recursive makefiles.

~~~
marcv81
Friends don't let friends write Makefiles at all. The author does not appear
to be familiar with them, except for the trivial examples in this post.

~~~
Pica_soO
Every folder gets its own little makefile. Calling those recursive shouldnt
result in trouble?

~~~
zwp
You might like "Recursive Make Considered Harmful", Miller.

[http://aegis.sourceforge.net/auug97.pdf](http://aegis.sourceforge.net/auug97.pdf)

Although I suspect the main complaint (slow, due to checking directories where
nothing has changed) has mostly been swept under the rug by increased CPU
speeds some of the other items (managing dependencies, ordering, ...) are
still worth considering today.

~~~
jhasse
Tup solves this: [http://gittup.org/tup/](http://gittup.org/tup/)

------
jolson88
Author here. Thanks for the great comments everybody. I'm honored this post
even got attention here (_long_ time reader and lurker here). I'm glad to see
a bunch of constructive feedback here that helps me clear up my ignorance
around make since I'm lacking in experience compared to other folks (okay,
really, I'm a Make noob).

------
Nanshan
Yeah, I'm writing a C build system just use shell+make, and it's work very
nice for learning C programming and try some new ideas.

[https://github.com/junjiemars/nore](https://github.com/junjiemars/nore)

------
welly
Recent experience of working with various task runners such as Phing and Robo
(plus others), eventually my previous employer returned back to simple
Makefiles and I in turn have done the same. For my needs of generating small
website builds, Make is perfect.

------
hashmal
TIL Make is not used widely anymore.

I may live in a bubble, but my last use of it was… maybe 2 hours ago (to build
someone else's project) and yesterday (to build one of my own projects).

~~~
abritinthebay
It’s used for basically most C/C++ projects and _somewhat_ outside of that.

It’s not been adopted (with good reason) in many newer language communities,
so depending on your tooling and platform you may see it less.

~~~
bryanlarsen
Yup, if Make was used extensively for Javascript, I wouldn't have had to file
this feature request:

[https://github.com/webpack/webpack-
cli/issues/152](https://github.com/webpack/webpack-cli/issues/152)

------
Walkman
If you would see our Makefile which can do literally_everything_ with the
project, you would not write this article. :)

Makefiles don't scale, hard to learn because of awkward syntax.

~~~
apple4ever
That's the think. The concept of Makefiles are great. The syntax is pretty
terrible.

GNUstep's Makefiles are actually really nice.

------
ruffrey
We have been using make for build and test scripts for docker+Node apps. I
have no complaints after a couple years.

------
jancsika
> Makefiles are simply a declarative way to transform one file (or series of
> files) into another.

 _Tab_ _echo_ it certainly is not _greater than_ side_effect.txt.

------
Yaggo
I used to be big fan of using Makefile for web development, but have since
changed my mind, because make is not very good fit for watch-mode incremental
building.

~~~
vortico
Interestingly enough, I tried using Makefiles for a web project yesterday, and
it was a success.

    
    
        SOURCES = index.pug layout.pug style.styl main.coffee privacy.md
        OBJECTS = index.html style.css main.js privacy.html
    
        all: $(OBJECTS)
    
        watch:
        	http-server &
        	while true; do \
        		$(MAKE); \
        		inotifywait -qq $(SOURCES); \
        	done
    
        %.html: %.pug
        	pug -P $^
    
        %.css: %.styl
        	stylus $^
    
        %.js: %.coffee
        	coffee -c $^
    
        %.html: %.md
        	markdown $^ > $@
    
        clean:
        	rm -rfv $(OBJECTS)
    

This could be extended to produce an out-of-source build by using `-o
"$(PUBLIC)"` on the command line generators, but I didn't need it for this
project.

~~~
jgrahamc
Do you know about the ; style for rules that have only one line of commands?
It helps reduce the number of tabs.

    
    
        %.html: %.pug ; pug -P $^
    

I find it super useful for compact Makefiles:

    
    
        %.html: %.pug    ; pug -P $^
        %.css:  %.styl   ; stylus $^
        %.js:   %.coffee ; coffee -c $^
        %.html: %.md     ; markdown $^ > $@
        clean: ; rm -rfv $(OBJECTS)
    

Also, worth making clean .PHONY since it's not a file.

------
user5994461
>>> Time for Makefiles to Make a Comeback

Until your build chain is f----- by the tab VS space issue of makefiles. Then
abandon makefiles again, for good.

~~~
bb88
This is really just an editor issue. Editors can parse a make file and do the
right thing these days.

------
sime2009
I really like the make approach but I want a modern updated implementation of
the idea. With the domain of web development in mind, my wishlist of
requirements looks something like this:

* Written in JS and installable via npm (i.e. runs on Windows, Linux and the other brand) * 'makefiles' are just JS code. * Supports using Shelljs for the command scripting parts and being cross platform. * Supports calling tools in "node_modules/.bin/" * Supports parallel builds for big modules projects. * No plugins, promises, streams, async or any other nonsense like most other JS/web related build tools or "task runners".

