
Notes for new Make users - henry_flower
http://gromnitsky.users.sourceforge.net/articles/notes-for-new-make-users/
======
dahart
> or you use special pattern rules

I used make for years before I understood pattern rules, even though they're
pretty simple. I kept trying to use foreach loops all over the place. If
you're using foreach loops, check whether a pattern rule can do the job.

> The popular clamour is that the autovars are too short & confusing.

I finally realized, after years of using make and then learning pattern rules,
that the three autovars listed here are visual mnemonics. $@ looks like a
target, a ring with a bullseye. $< is pointing left, so it's the first prereq.
$^ is like a horizontal bracket that groups all prereqs.

~~~
gkya
> I finally realized, after years of using make and then learning pattern
> rules, that the three autovars listed here are visual mnemonics. $@ looks
> like a target, a ring with a bullseye. $< is pointing left, so it's the
> first prereq. $^ is like a horizontal bracket that groups all prereqs.

I've been illuminated, sincerely! Meanwhile, bmake has some more memorable
synonyms for them like ".TARGET" and ".ALLSRC".

~~~
sjmulder
It's a shame that there is no name for .ALLSRC shared between bmake and gmake.
An alternative is to use a variable, but then you lose VPATH compatibility:

    
    
        foo: foo.o bar.o
            $(LD) foo.o bar.o ...
    

My other gripe with bmake is that this rule is needed as there is no implicit
rule for linking multiple object files, or so it seems.

------
gengen
What I've always loved about Make and related tools is how directly they
expose their fundamental concept in building artefacts: the DAG. I don't do
that much JS, but I've not found this kind of clarity in any of the JS
frontend build tools I've used.

Honest question for the webpack power-users here.

What are some important webpack features that you lose by using a Makefile
like this? Because it seems like pretty much every part of a regular webpack
buildchain has a command-line equivalent that you could potentially use in a
Makefile.

~~~
markmark
I don't know enough about make to know if this is correct, but the post last
week on using make for js started of by saying "If you are writing a frontend
app and you need code bundling you should absolutely use Webpack" which,
aren't the majority of people using webpack writing frontend apps and need
bundling?

The other issue is that essentially no one uses make in frontend development.
So you go and convert to make from something that was working fine and is the
standard in that part of the industry and what's the pay off? Now no one in
your team or who you hire knows how to change the build chain? Great.

~~~
dahart
> aren't the majority of people using webpack writing frontend apps and need
> bundling?

Webpack is pretty new. Lots and lots of websites were developed before webpack
existed, and some of them still use make for builds. The majority of new web
projects are probably webpack now.

> The other issue is that essentially no one uses make in frontend
> development.

I don't know how many people do or don't use make, but I've had two web jobs
that did. One was my own company and I wrote the makefiles, so that's not
exactly fair. :P The other was an established web app. In both cases, the
biggest reason for the makefile was to integrate the Google Closure Compiler
into the build. Last time I checked, Webpack didn't have support for the
Closure compiler. I am aware that the Closure compiler isn't exactly the most
popular js minification tool today, and isn't generally necessary with
webpack.

> So you go and convert to make from something that was working fine and is
> the standard in that part of the industry and what's the pay off?

I agree with you generally, but I'm old enough that this also makes me chuckle
a little. make was working fine and was the industry standard for decades
before webpack or npm.

Just before Webpack there was Gulp, and Gulp is essentially a re-invention of
unix pipes and lots of the unix tools. Build systems made with gulpfiles have
to bend over backwards to get right make's basic concepts like dependency
evaluation, building a target only once, and skipping already built targets.

~~~
markmark
There's a reasonable chance I'm older than you for what it's worth.

------
vortico
Pretty spot on. I use a pretty similar dependency system for all of my C and
C++ projects.

    
    
        FLAGS += -MMD
        CFLAGS += $(FLAGS)
        CXXFLAGS += $(FLAGS)
    
        OBJECTS += $(patsubst %, build/%.o, $(SOURCES))
        DEPS = $(patsubst %, build/%.d, $(SOURCES))
    
        $(TARGET): $(OBJECTS)
        	$(CXX) -o $@ $^ $(LDFLAGS)
    
        -include $(DEPS)
    
        build/%.c.o: %.c
        	@mkdir -p $(@D)
        	$(CC) $(CFLAGS) -c -o $@ $<
    
        build/%.cpp.o: %.cpp
        	@mkdir -p $(@D)
        	$(CXX) $(CXXFLAGS) -c -o $@ $<

~~~
babuskov
This looks very similar to the Makefile I'm using except that beside -MMD I
also have -MP flag. I created this Makefile a couple years back and I'm
reusing it for every new project, but I cannot recall really why I needed to
add -MP.

~~~
vortico
[https://gcc.gnu.org/onlinedocs/gcc-4.8.0/gcc/Preprocessor-
Op...](https://gcc.gnu.org/onlinedocs/gcc-4.8.0/gcc/Preprocessor-Options.html)
Looks useful, I'll add it to my configs.

~~~
babuskov
Yes, that's it. Removing a .h file from your project would trigger errors
without that switch.

I looked at clang manual when I searched for it, but unlike GCC it does not
explain it.

------
buserror
The thing people miss a lot about makefile, apart from what has already been
mentioned is the fact you can have "local" version of the "global" variables,
per rules.

It's very very powerfull, clear to read, and simplify all these if/else/end no
end...

For example:

    
    
      CFLAGS = -std=gnu99 -Werror
    
      release/%.elf: CFLAGS += -DRELEASE
    
      debug/%.elf: CFLAGS += -DDEBUG
    

Basically you can add your own context flags and variables for each individual
targets, and they are 'inherited' by any of their /sub/ targets...

Oh, people also need to learn about VPATH or course [0]!

[0]:
[https://www.gnu.org/software/make/manual/html_node/General-S...](https://www.gnu.org/software/make/manual/html_node/General-
Search.html)

~~~
henry_flower
yes, target-specific vars are neat.

I've thought about adding a couple of sentences about vpath (in a cautious
style akin to "think twice before using it" [0]), but ultimately decided that
the topic is too obscure; besides I practically never use vpath myself.

[0] [http://make.mad-scientist.net/how-not-to-use-vpath/](http://make.mad-
scientist.net/how-not-to-use-vpath/)

------
elsherbini
I use snakemake[0] as a replacement for make. The debugging tools in snakemake
are really good - it can output the DAG of your workflow as a dotfile, and you
can add breakpoints and use pdb to debug your workflow. It also integrates
really nicely with conda, so you can have different environments for different
rules.

The killer feature of snakemake for me is that it can run a workflow on a
single core or as many as you give it, or in the case of a cluster it can
actually submit jobs to a queue and wait for them to finish before submitting
more.

The tradeoff is that you need to install it (conda install -c bioconda
snakemake), and that it can be very slow if you have thousands of target
files. But for most workflows it's the right tool for me.

[0]
[http://snakemake.readthedocs.io/en/stable/](http://snakemake.readthedocs.io/en/stable/)

~~~
blcArmadillo
_> The killer feature of snakemake for me is that it can run a workflow on a
single core or as many as you give it, or in the case of a cluster it can
actually submit jobs to a queue and wait for them to finish before submitting
more._

Other than your cluster example isn't this the same as passing -j to make to
run jobs in parallel?

~~~
rflrob
Yeah, -j just about does it, though with Snakemake you can also specify other
resources (memory, I/o, etc) for each task, and it will keep all of them under
quota. So if you tell snakemake that task A takes six cores, but B-K take only
one each, it will solve the knapsack problem to come up with some kind of
optimal order for them, and on your 8-core machine you could have A, B, and C
running, or B-K.

For a while, I muddled through with Make and a custom cluster job-submitter
script, but the real Snakemake killer feature for me was multiple, independent
wildcards. So I can, in one rule, have dir/{databasename}/{query}.tsv and
rapidly hit multiple databases at once. If you are very clever (and rule one
of Make is don’t be clever), you can have a GNUMake wildcard get parsed into
multiple wildcards with the wimpy pattern substitution rules, but this is not
a fun way to relax.

~~~
jschwartzi
Oh, cool, so I can finally run make -j target without worrying about the LMK
in Linux killing my window manager!

------
ISL
The point about the manual is a good one. It is well done.

When I made extensive use of Make for my graduate thesis, I bought the book
and read the majority of the text.

The cover illustration brings me joy every time I see or think about it.

~~~
nwellnhof
I'd suggest to read the POSIX man page first:
[http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ma...](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html)

POSIX make is rather limited but if you start to rely on GNU make features,
you'll end up bound to a single implementation.

~~~
EpicEng
>you'll end up bound to a single implementation.

Which will be just fine for something like 99% of us.

~~~
wahern
POSIX make has the virtue of simplicity. GNU make : POSIX make :: C++ : C.
It's even worse than that because the way GNU make extends the syntax is
completely at odds with core make syntax. It's why to write a looping
construct in GNU make you need the contortion of using eval, whereas a loop in
the BSD family of makes is much more simple and concise.

I'm not advocating to use a BSD make, nor saying _not_ to use GNU make. But
appreciating GNU make's points of departure from POSIX make helps keep your
makefiles simple and to the point. Which is important because another virtue
of make generally (GNU make included) is that it'll likely still be around
long after all these alternatives have been long forgotten. But time is never
kind to complexity.

------
dblotsky
This is so excellent. I was smiling as I read it. Thank you.

My personal favourite "watcher" implementation is this:

    
    
        while true; do make; sleep 0.5; done

~~~
FrenchyJiby
I don't understand: why not the classic `watch make` (watch defaults to 2
seconds)?

~~~
teamhappy
Watch is Linux-only.

~~~
khedoros1
I see it in Cygwin, in the procps-ng package, and available on OSX through
Brew. Doesn't seem Linux-only.

~~~
teamhappy
My bad. I figured it needs /proc since it's part of the procps package.

~~~
pantalaimon
Why would it?

You can literally write it as a shell script

    
    
      #!/bin/bash
      
      N=2
      
      while true; do
              clear
              echo Every $N\s: $*
              $*
              sleep $N
      done

~~~
teamhappy
I though maybe it had some flag that makes it display process information or
something. Anything that would justify it being part of procps really.

------
LinXitoW
In all these discussions about make, everyone argues about using it for
everything. Personally, I don't know another tool that makes it straight
forward to glue together all kinds of different build tools (for example
webpack for frontend and maven/sbt for the backend).

Even if you don't directly use make, it makes for great, verifiable
documentation.

------
ejholmes
> Most important concept #1: DAG

Yes! Thank you! This needs to be banged into peoples heads when talking about
build systems. Every time you add an edge, you're automating an otherwise
manual task. Very important realization.

------
jstimpfle
I've been working on a small custom build system (of the "re-editable" kind)
for my own software project, and have been working on a little writeup. It's
not finished, and I think there was something in the implementation that I was
still unhappy about. But I'll take this opportunity to jump on the bandwagon.
I think for larger project it is worth it to just make your own system, so
here are my notes:
[http://jstimpfle.de/blah/buildsystem.html](http://jstimpfle.de/blah/buildsystem.html)

------
lolikoisuru
If you’re feeling mischievous, Make has divers ‘secret’ vars & targets for
you. For instance, you don’t have to ‘struggle’ w/ the ancient bash, for it’s
possible to program recipes in any language that supports evaluations straight
from the command line.

In addition you can have different shell for each rule with the rule specific
macros if you do desire:

.ONESHELL:

.SILENT:

.PHONY: rule1

rule1: SHELL := /usr/bin/python3

rule1:

    
    
         foo=bar
    
         print("python: {}".format(foo))

------
mahoro
Great article.

Also, author prepared a surprice for all coming to his blog from russian IP
addresses. That was the first time I came across such thing.

~~~
confounded
Can you explain it for us?

~~~
Igor_kh
The author has put an offensive banner
([https://imgur.com/a/84cuN](https://imgur.com/a/84cuN)) in front of his blog
page for Russian IP addresses. I just wonder why there is no such thing in the
article above

------
setheron
Great guide. The variable portion was the first time it was explained
succinctly and simply to me.

~~~
henry_flower
thanks!

------
abhishekjha
Wait. Did I read it wrong or didn't an article a few days back say that we
_can_ use make with file names consisting of spaces, its just that we
shouldn't? Although that article talked about using a non-breaking space as a
hack.

~~~
spc476
Make doesn't handle filenames with spaces (ASCII 32) at all! I wrote the
article
([http://boston.conman.org/2018/02/28.2](http://boston.conman.org/2018/02/28.2))
about using the non-breaking space (NBSP, Unicode 160) in filenames, and make
(as well as nearly everything else I tried) accepted it just fine.

But popular consensus (from here and Lobsters) seemed to be "avoid any type of
space at all cost and do not speak of it again!". Pity.

~~~
dasyatidprime
I think the main point of handling file names with character code 0x20 in them
is for situations where you _don 't get to pick_ the file names.

I have a project in which I'm juggling a large amount of files imported from
external sources, which need to be variably processed and used as-is and in
either case must have their names preserved. I wanted to use Make to handle
the dependency chaining, but munging file names when inducting them means that
I have to unmunge them on every output. Suddenly you can't just zip or grep -H
a selection of some of the files anymore; you have to do a whole dance of
copy-and-unmunge-name, clean-up-the-temporary-directory, and so on and so on
and.

If I could just assign my own names, I could just as well avoid spaces, if it
were purely an aesthetic issue. But it really isn't. You don't often need
“something that _displays as_ a space but you still get to decide its
representation”, you need “passing through _exactly the ASCII space_ without
having to worry about it”, or the interoperability concerns _still_ explode in
your face one way or another (if they exist in the first place).

~~~
lolikoisuru
If I were to write a makefile for something where I can't restrict the
filenames beforehand I'd probably just have a target for running `detox` and
then at the very end possibly an another target that changes underscores to
spaces and creates some dummy file as the real target.

~~~
dasyatidprime
But how does this address anything like the zip or grep cases I mentioned?
File names go _everywhere_. You will have spurious nonbreaking spaces (or
underscores, or whatever else) in outputs from here to /dev/null and back.
Yes, it is theoretically possible to work around it, but it's hardly ergonomic
for either the writer or the user of the Makefile.

------
lolikoisuru
One thing I sometimes wish was possible in make is using the automatic
variables in the dependency part of the rules. The one that I most often have
desired to use is $(@D) so I could depend on a file with specific name in the
same directory when the directory name isn't known beforehand. I got around
this with some touch trickery but it would be just easier to read if those
automatic variables were available at that point.

------
lolikoisuru
>When in doubt, don’t google until you read the official manual. It’s a
document of an astonishing quality. Go print the pdf.

This applies to most GNU software.

------
ghettoimp
The writing is so awful. “Some ppl will get overexcited & start...”? “2fold”?
Are we teenagers texting each other?

~~~
henry_flower
thanks for the feedback.

I've heard in the past about my "w/" and "&" misuse (apart from my pitiful
grammar) but I haven't released it can put people off. I'm sorry.

~~~
ghettoimp
Sorry for being so harsh. I was in a terrible mood, but that's no excuse.

~~~
henry_flower
it's alright

besides, you're right, "ppl" and especially "2fold" were a bit too much :)

------
pspeter3
Is it possible to have make use file hashes instead of timestamps?

~~~
gkya
There is DJB's redo[1], which has some implementations. Also, one could
probably modify make to use hashes. But personally I think that mtime is a
nice compromise, as it's often a correct heuristic, and hashing files is not
cheap, especially for big projects that'd have a performance penalty.

[1] [https://cr.yp.to/redo.html](https://cr.yp.to/redo.html)

------
vorotato
The only reason I don't use it is because I use windows. While you can
theoretically use it on windows, nobody does.

~~~
dahart
> While you can theoretically use it on windows, nobody does.

What do you use instead?

I think a lot of people use make on Windows. I've had several jobs with a
Windows dev environment, and they've all used make. My current dev env is
cmake on Windows, so I don't touch the makefiles, but it's still using make on
Windows.

There aren't that many options for cross-platform projects. A lot of people
use cygwin or the Windows Subsystem for Linux with make and/or cmake native.

I also go directly to make on either cygwin or unix for one-off builds all the
time, in non-C/C++ projects. make is super handy for resizing images or
running a batch of tests on a program generating graphs. Anything where you
need to generate files that only update when another file is touched is a good
job for make.

make -j is extra handy on windows for some situations since it's hard to get
gnu parallel to run in cygwin and the parallel perf wasn't very good last time
I checked.

~~~
skocznymroczny
Most Windows projects that I've seen in the open use CMake to create VS
solutions which usually just work.

------
andrewmcwatters
The first point is "Don’t try to be clever," and using Make in the JavaScript
ecosystem defies exactly that to begin with.

~~~
bringtheaction
No it doesn’t. Makefiles define inputs, outputs and the commands that
transform input to output. Make then compares the last modified timestamp of
inputs with outputs on each run and reruns the commands for the targets that
are out of date.

Make is in no way limited to C or C++ projects and there is absolutely nothing
wrong in using Make for a JavaScript project.

The only thing that sucks a bit is maintaining the list of inputs and outputs
if you have a lot of files.

Well, that and the fact that Makefiles are not generally portable between GNU
and BSD Make so if you use both Linux and a BSD you need to install either
gmake on BSD or bmake on Linux and then remember to invoke that rather than
just make for your project. Of course you could have a bash function that will
look and pick GNU make if there is a GNUmakefile in your directory and BSD
make if there is a BSDmakefile, but maintaining such things is a hassle as
well.

It always comes down to picking the right tool and sometimes Make will be it
and sometimes it won’t but whether the output is JavaScript or a PDF (e.g.
from LaTeX documents) or JPEGs or whatever has very little to do with it.

~~~
andrewmcwatters
> Make is in no way limited to C or C++ projects and there is absolutely
> nothing wrong in using Make for a JavaScript project.

The fact that you have to acknowledge this proves how unorthodox it is, and
ignores team concerns. So no, there is something wrong with using Make for a
JavaScript project: people don't normally do it.

> It always comes down to picking the right tool

You're right, stick to the ones people actively use and don't pretend features
exist in bubbles.

~~~
markmark
I can't believe you've been down-voted for these two posts. If you convert
your js projects to make you are going to be the only person on your team who
ever touches them again for no pay off.

~~~
andrewmcwatters
I've generally found that criticism, even if a reasonable sentiment, on Hacker
News trends downwards. It seems to be more favorable to support something or
stay silent, than to give critical commentary. It could also just be my tone.

I'm sure my comments would be more popular if I ignored the realities of
software development and only praised the novelties of it.

~~~
wruza
“there is something wrong with using xxx for a yyy: people don't normally do
it” seems like a logical fallacy even without considering the details, which
even more people are not interested in.

~~~
andrewmcwatters
how is that a logical fallacy at all? do you know what a logical fallacy is?
it's a measurable fact nowhere does make show up on the top of the list of
build tools for this ecosystem during regularly published analytics for the
industry

im astounded so many idiots have such high karma counts on hn, what a
wonderful place to have discourse

~~~
wruza
If you’re about _my_ karma, then it is not high as far as I care, nor am I too
smart, as you noticed. Most of upvotes I got were stupid rants and complaints
about modern ui and javascript with few points that I personally find
important. I think we all differ in intelligency, understanding, style, etc,
so my thought on karma is... it doesn’t matter; what matters is how one reacts
to others meaning, both structured or emotional.

I don’t see why your comments have to be grey at all, if that helps, just
pointed out where few may have ‘misheard your tune’. But if it is
unthoughtful, then it’s their problem, not yours. They’ll use make and suffer,
isn’t it fair?

