
The Ultimate Frontend Build tool: make - jmtulloss
https://algorithms.rdio.com/post/make/
======
forrestthewoods
God I hate make. I hate make so very, very much. Compiling code isn't that
hard. I swear to the programming lords on high it isn't.

Here's a wonderfully useful open source project - Google gperftools [1]. It
includes TCMalloc amongst other things. The Makefile.in is 6390 lines long.
Configure is 20,767 lines long. Libtool is 10247 lines long. That's fucking
insane.

Compiling OpenSource is such a pain in the ass. Particularly if you trying to
use it in a cross-platform Windows + OS X + Linux environment. One of my
favorite things about iOS middleware is that one of their #1 goals is to make
it as easy as possible to integrate into your project. Usually as simple as
add a lib, include a header, and call one single line of code to initialize
their system (in the case of crash report generation and gathering).

I work on a professional project. We have our own content and code build
pipeline. It supports many platforms. I don't want anyone's god damn make
bullshit. I want to drop some source code and library dependencies into my
source tree that already exists and click my build button that already exists.

</rant>

[1]
[https://code.google.com/p/gperftools/](https://code.google.com/p/gperftools/)

~~~
vezzy-fnord
And yet most contemporary hipster build systems are all just shiny make
reinventions.

~~~
Touche
Reinventions that are cross-platform (this actually matters).

~~~
oijaf888
What platforms doesn't make support? AFAIK GNU make supports pretty much
everything and there are versions of make for Solaris/BSDs.

~~~
GeneralMayhem
Make exists everywhere, but you have to explicitly write separate rules for
every system.

------
goalieca
With the theme of ageism and NIH being brought up a lot lately, I'm glad to
see that the neck-bearded unix philosophy (tm) of composition and single
purpose are winning over these complicated object oriented frameworks.

~~~
dfc
Why not drop the "neck-bearded" bit? Evi Nemeth had more Unix chops than
99.86% of the people on HN and no neck beard.

~~~
JetSpiegel
Neck-Beard is not about the beard, it's a state of mind.

~~~
test1235
Your 'inner' neck beard, as it were.

~~~
JetSpiegel
Yes, but that sounds even worse.

------
shadowmint
The problem with make is not that it's bad, it's that's it's only really
_good_ at doing two things:

1) mapping a source pattern to an output pattern

2) managing dependencies between rules

To be fair it's _good_ at those, and often the sorts of things you can do with
a rule are quite complex (being basically shell scripts).

However, the problem is that 1) it's an obscure DSL and 2) that it is really
_rubbish_ at doing more complicated things.

For example, grunt-contrib-clean lets you: delete any files that match a
regex, not leave alone any files that match a different regex. Grunt also has
a built in templating language that can be used to expand configuration files
from submodule into local build scripts _without_ copying the entire
gruntfile. grunt-open launches a browser to a dev url cross platform. The list
goes on and on and on.

Make is _terribly terribly bad_ at complex tasks like this, that's the
problem.

You _can_ write a custom shell-script / ruby-script / python-script for these
tasks, but why would you? Someone else _already has_. _Dont_ repeat all the
things every time with your own code.

If all you need to do is map .c to .o, or .scss to .css and .coffee to .js,
use make, totally. It's _good_ at that.

Otherwise, stay away.

~~~
gkya
There are other benefits to make. First, you'll need to ask users to install
grunt, whereas make is standard and is just there (except for Windows,
probably). Second, make provides a language that is optimised to express the
information it needs, whereas grunt uses a JSON file. It gave me headaches
when I needed to edit grunt config at my last job, and I never touched grunt
ever since. Third, all the tasks you've specified can be done with standard
UNIX utilities, whereas you depend on third-party libraries with grunt. Also,
I do not think that any of those tasks are complex.

I'd rather rewrite a 4-LOC shell script in my new project, instead of
depending on a build tool that depends on a non-standard, infant runtime
itself, and also depends on third party libraries for deleting files.

~~~
auvrw
i was going to say something in response to the article about dependencies
from a positive standpoint for grunt, but i think it makes sense to put it
here.

rather than having a unix development environment for the browser platform,
thanks to lighttable, grunt, and others, it's possible to use node/web-
browsers as a development environment as well. i guess node isn't as venerable
as unix or even the jvm for that matter, but is "infant" really a fair
characterization? i've heard that node can actually do some types of string
processing faster than the jvm.

i totally agree on starting with shell scripts, though. i tend to put short
scripts all throughout repos, even if they just run a single command with a
few arguments. sometimes they grow into longer scripts, and sometimes they get
changed to run different programs (e.g. grunt<->make), but having a few
consistent names for tasks like build, run, test, and deploy (.sh) across
projects and languages goes a long way to cut back on cognitive load.

~~~
gkya
> ... but is "infant" really a fair characterization?

The point I wanted to make was that one needs to install node to run grunt; I
don't believe there's any OS that distributes node in the base distribution,
whereas make and sh is a part of the POSIX standard, and perl/python is
available by default on most OS's.

Also, node is indeed infant and young, don't consider that a bad
characterisation, I do enjoy playing with node. My plea is that one should not
need to install some software to run the build scripts in a project they
download.

------
jdp
I'm glad to see the resurgence in make's popularity among front-end
developers. It really is a great tool for not just building apps, but
generating files that depend on other files in a declarative way. I manage my
website with just make, pandoc, and rsync; and I manage my various ssh
configurations with make and m4 (ssh config doesn't have an include
directive!). A while ago I wrote an article to help out some front-end dev
friends get acquainted with make, maybe someone else will find it useful:
[http://justinpoliey.com/articles/make-for-front-end-
developm...](http://justinpoliey.com/articles/make-for-front-end-development/)

~~~
danieldk
It's also very handy for anything you write in LaTeX: add figures etc. as
dependencies, make targets that run R or gnuplot to produce the figures in the
right format, etc.

make is great for a lot of build tasks.

------
pedrocr
redo[1] is almost never mentioned in these discussions. Has anyone tried it in
anything significant? It looks good and is a djb design.

[1] [https://github.com/apenwarr/redo](https://github.com/apenwarr/redo)

~~~
sigil
I've (ab)used apenwarr's redo for a couple big data processing projects, with
mixed results.

One was a news recommendation engine. We pulled down and parsed RSS feeds,
crawled every new link they referred to, crawled thumbnails for each page,
identified and scraped out textual content from pages, ran search indexing on
the content, ran NLP analysis, added them to a document corpus, ran
classifiers and statistical models, etc.

Every step of the way took some input files and produced an output file. We
used programs written in many different languages -- whatever was best for the
job.

So a build system was the obvious way to structure all of this, and we needed
a build system we could push pretty hard. Our first version used make and
quickly ran into some limitations (essentially, we needed more control over
the dependency graph than was possible with the static, declarative approach)
so we turned to redo, which lets you write build scripts in the language of
your choice.

One thing we needed almost immediately was more powerful pattern matching
rules than make's % expansion. No problem: invent a pattern syntax and a
special mode where every .do script simply announces what patterns it can
handle. Collect patterns, iteratively match against what's actually in the
filesystem, and then you've got the list of target files you can build. (This
already differs from make, which wants you to either specify the targets
explicitly up front as "goals," or enumerate their dependencies via a $(shell
...) expansion and then string transform them into a list of targets which are
ALSO matched by some pattern rule somewhere...okay you get it, it's make, it's
really disgusting.)

Another thing we needed was to say, here's a list of target files that appear
in the dependency graph, give me them in topologically sorted order. This
allowed us to "compact" datasets as they became fragmented, without disturbing
things downstream from them in the dependency graph. Again, this was not
difficult with redo once we had some basic infrastructure.

Now, was all of this maintainable, or was it just kind of insane? I think in
the end it ended up somewhat insane, and most importantly, it was an
unfamiliar kind of insane. The insanity that you encounter in traditional
Makefiles is at least well understood. And treatable.

With redo, you can do almost anything with your build. You can sail the seven
seas of your dependency graph. It's awesome. It's also terrifying, because
there is very little to guide you, and you may very well be in uncharted
waters.

But give it a shot anyway. YMMV.

------
rjd
I couldn't agree more here. After reading the source code on the twitter css
bootstrap makefile a few years ago I got inspired and wrote my own rewrite of
what they had... I was blown away by the gains at it gave me. Of note I bound
make to cmd+b in Sublime Text, and I just compile when I see fit.

I've now tweaked my make files so they are almost unrecognisable from the
twitter ones, it can run PHP, JS unit tests, fires up phantomJS and tests
individual modules, can release minified for production, or not for debugging
purposes. I can't stress how useful it is being able to just add a line and
you get such powerful support.

I haven't had time to add git hooks yet, but thats the next stage, I plan to
set the hooks to run tests and clamp down on poor quality code (I work with
interns quite often... sometimes I cry for just average quality code coming
in).

For a story of the production gains I've had. I moved the whole company CSS
into a CSS preprocessor and cleaned up all the existing structures to fit the
make file release procedure. It came back a thousand fold away a rebranding
and I had everything done in two days. I was blown away with that alone, I've
been involved in so many rebrands over the years that go on for months... not
hours.

Do it, if you're bit unsure where to start check the twitter bootstrap build
file and muck around with what they have done.

~~~
jeffbr13
They've used Grunt for builds for a while now[0].

It seems most of the twitter-style web developer community who happily
replaced `make` with `rake` when developing for Rails simply followed that
with `grunt` when they moved to JavaScript.

[0]:
[https://github.com/twbs/bootstrap/commit/0d33455ef486d0cf06c...](https://github.com/twbs/bootstrap/commit/0d33455ef486d0cf06cb29d427efda57f42f7ea9)

~~~
rjd
Cool, I've read good things about grunt but haven't had the time to look into
it. Its direct access to the CLI thats the killer for me with make ... and to
be frank ... if I was going to script in a third party language it would
probably be python and not javascript but each to there own.

~~~
Offler
Don't rely too much on Grunt, only use Grunt/Gulp when npm scripts doesn't do
the trick. npm scripts has the same direct command invocation that make has so
no need for plugins.

------
notduncansmith
While I'm a big fan of the Unix Philosophy, I've found gulp to be incredibly
satisfying as a build tool. I cd into my project directory and run gulp, then
all my SCSS/CS gets compiled, watched, and re-compiled on change, with
LiveReload pushing style changes straight to the browser; and whenever I make
a change to my server (Node.js) code, it gets compiled and run against my
tests. Then when I'm ready to push to production, I run gulp build and then
push it up :) Oh, and my whole gulpfile is <100 lines, a fair amount of which
is whitespace/stylistic. It's incredibly readable and flexible: composing
complex supertasks is easy.

I don't have anything against Make, I've never used it; judging by the code
samples in the article, however, and contrasting that with my gulpfile (which,
it's worth noting, didn't require me to learn a new language/DSL, just a dead-
simple API), I feel much more empowered by gulp than make.

Somewhat relevant: I've also found gulp much easier to use and maintain than
Grunt.

~~~
TheAceOfHearts
I like Gulp a lot, but I'd hardly say it's perfect. It's weird, it doesn't
really make sense that everything is a stream. For example, why is watching
something a stream?

With that being said, it's miles above Grunt.

~~~
notduncansmith
Yeah, once you commit to any sort of "everything is ______" philosophy, you're
bound to find some edge-cases where things get weird. The file-watch in
particular does make sense to me, albeit in an odd way: basically, the file
just gets polled on an interval, then it gets streamed into whatever tasks
you've attached to the watch. Thinking about the act of watching itself as a
stream does seem hinky.

------
antihero
The thing I like about Gulp is that all it needs is NPM to install all its
stuff. Make you have to worry about different system libraries, etc. Also,
with gulp I can leverage other node based libraries, so for one project, I can
get frontend guys set up using a dev server which automatically both proxies
API requests to a backend server, and also watches and livereloads files (as
they like), with three damn commands - npm install && bower install && gulp
dev. Bam. Working dev server running on port 8000 that does all that they need
it to.

~~~
Offler
I would use npm scripts as the first way to execute build steps in a package
and only fall back to Gulp when it's not powerful enough
[https://www.npmjs.org/doc/misc/npm-
scripts.html](https://www.npmjs.org/doc/misc/npm-scripts.html)

------
crussmann
For simple builds with dependencies, I find makefiles hard to beat. The only
thing that has me pulling out my hair every time is the whole tabs vs. spaces
thing, especially when on a machine that I haven't pulled in my .vimrc yet.

------
tdicola
Count me in as a fan of makefiles too. Haven't done much with them for
frontend stuff but will have to give it a shot. My biggest problem with make
is that, kind of like javascript, it's been around for so long it's really
hard to find good information on it. At least the core functionality really
hasn't changed all that much so even very old information is still quite
relevant.

~~~
McUsr
Me too.

I use make together with a markdown compiler, and the m4 preprocessor, to keep
devdocs up to date, in one huge document, where everything is included, and
the various sections as stand-alone docs. The markdown version of the section
files is almost uncluttered from m4 and html. I link from any external doc to
any other via links in the toc in the main doc index.html, to keep everything
as simple as possible. It's sweet.

------
hunvreus
I love make too, but still settled for Gulp (and previously Grunt). Writing a
Makefile just isn't for the faint of heart and can be very frustrating for
front-end folks.

It's a trade-off between simplicity and convenience.

~~~
criswell
I just couldn't get comfortable with grunt, with gulp I had no issues.

------
codezero
FYI, this only loaded after disabling ghostery and adblock.

~~~
dfc
It gets blocked by easylist+easyprivacy in adblock:

    
    
      Filter: /logger.js
    
      Exception: @@||algorithms.rdio.com^

------
sigvef
Although make does have a lot of weird quirks, its winning strength is that it
is installed virtually everywhere.

~~~
sdesol
The problem that I've found when I was trying to port my installer to FreeBSD
was their make doesn't like GNU Makefiles. Not sure when they diverged from
one another so having Make everywhere, doesn't necessary mean your Makefile
will work everywhere.

I liked how security centric FreeBSD was but they seem to be anti-enterprise
friendly. The process to install Oracle Java was painful and EnterpriseDB
didn't even bother creating a Postgres installer for it. Maybe the freedom
that the BSD license offers, is making their environment too stagnant when
compared to GNU and Linux.

~~~
kchoudhu
A cursory google search should have shown that gmake (GNU Make) is widely
available and used on FreeBSD.

~~~
sdesol
Yes it is available and I guess I sort of boxed myself in when I said not
available. It's not available as a default install ... at least not on the
FreeBSD 10 install disk I used.

~~~
oijaf888
FreeBSD tries not to install much third party software with the default
install. Everything is available in ports/pkg. The things included in the
default install usually BSD licensed/BSD versions. Personally I prefer having
third party software disassociated from the core operating system. For me at
least, it makes tracking critical updates a lot easier.

------
dubcanada
There us Premake which is Lua version and can be configured to do anything you
want, and will run on anything.

~~~
mercurial
Premake is pretty nice. Much better than CMake with its weird homegrown macro
language.

------
Offler
I think a lot of FE devs aren't fully aware of npm scripts
[https://www.npmjs.org/doc/misc/npm-
scripts.html](https://www.npmjs.org/doc/misc/npm-scripts.html) \- it's a handy
way of calling commands just like you would in make. Quite a lot of the tasks
that people use Grunt/Gulp for could be executed with it.

------
TheZenPsycho
I've been using make for front-end builds for a while. two tools I've been
finding to be indisposable are jq and mustache templates. I made a php based
mustache/cli implementation here:

[https://gist.github.com/Breton/7556390](https://gist.github.com/Breton/7556390)

This enables you to put a lot of your configuration in a json file, such as
actual lists of files (which make is actually terrible at), have a decent way
of getting json data out into make processes, and build files templated out of
that json configuration, such as ssi files that set whether to load built JS
or seperate script tags (also generated out of the same json config).

SO you get the declarative stuff in the declarative JSON format, and the stuff
that make is good at: incremental builds, stays in make.

------
rmrfrmrf
OK, I get the simplicity of using make, but, yikes, creating your own "little
programs" that _parse AND EDIT (?!?!) code_ is a simple solution? IMO the
author glossed over that part very smoothly without even bringing up the
potential pitfalls (bugs in author's "tiny little programs", invalid html
causing the parser to barf, etc etc etc). Really I think you'd have to be
pretty stubborn to not see the value of Grunt when you decide that you need to
implement an _HTML parser_ as a substitute for if-statements. Lol geeze.

------
dfc
Is anyone familiar with SCons? I stumbled across it the other day when I was
playing with gpsd.[^1]

[^1]: ESR lovefest:
[http://esr.ibiblio.org/?p=3089](http://esr.ibiblio.org/?p=3089)

~~~
hhm
I found it to be simple & quick to use, while providing great results.

------
tomgruner
I've started using Make to build and run docker containers in the dev
environment. It's not the perfect tool, and neither are my skills that great
with Make, but it sure gets the job done and enables docker commands to be
shared between devs. For example this Makefile (still under development) for
PostgreSQL [https://github.com/GlobAllomeTree/docker-
postgresql/blob/mas...](https://github.com/GlobAllomeTree/docker-
postgresql/blob/master/MakeFile)

------
Touche
You can't write a blog article about make while comparing it to a bunch of
Node.js build tools and not mention the word Windows once. Make is good for
you because you're not an open-source project built on a cross-platform
development platform. All of your employees use Macs and your servers are
Linux. For OS Node projects make is simply out of the question because it is
janky on Windows and every makefile ever written is full of bashisms.

Next.

~~~
CoffeeDregs
Certainly you can. In fact, the OP did exactly that and his article is at the
top of HN.

My hope is that we can just carry on with our dev work and Windows will slowly
fade away as a platform for developing anything but SharePoint intranets...

[Note: I don't have a neckbeard, but I do run Debian testing.]

~~~
recursive
But... but... some of us our doing our development on Windows and loathe
Sharepoint.

------
girvo
Now this is going to sound weird, but I use Make for my PHP development.
Seriously. It's perfect, and is installed basically everywhere I could want to
run it, and it's super lightweight but just powerful enough.

For my cross-platform game development I've moved to CMake, as it takes care
of some annoying bits for me. Not a massive fan of it though, and am tempted
to go back to Make for my C++ game dev stuff. Any recommendations for it?

~~~
nicholassmith
90% of the time if a project uses CMake I'll look for an alternative as I can
count the amount of times I've had hassle free compilation with it on both
hands. It's a perfectly fine build system that does handle a lot of weird
cases, but I've found reliability wise it's far flakier than standard make.

------
mamcx
I'm using [https://github.com/rags/pynt](https://github.com/rags/pynt) right
now. Is _very_ simple!

Have use [http://www.finalbuilder.com/](http://www.finalbuilder.com/), is a
great tool that put me in the _automated build_ mantra.

------
Bahamut
One thing I should note - the difference here is that make enforces working in
a Unix-like environment...which cuts out Windows users. I know, most devs will
use OS X or their favorite flavor of Linux anyway, but at least personally
speaking, there are occasions when I do some development on Windows.

------
shocks
I'm tethered on a mobile connection and all I can see when I try to view this
is a blank page. :(

~~~
jmtulloss
It's not the most robust blog in the world, sorry about that :)

~~~
shocks
Heh, it's okay. I'll read it when I'm not browsing on a potato! :)

------
droptableusers
What is the purpose of blending the text with the background and making it
barely readable?

------
soapdog
For those wanting to check out more about the GNU flavor of Make, there is
this little book:
[http://shop.oreilly.com/product/9780596006105.do](http://shop.oreilly.com/product/9780596006105.do)

------
badman_ting
I have seen this idea floating around… personally I would rather write JS than
learn another weird file format that I will have to Google every single time I
mess with the Makefile, but the idea is decent.

~~~
mathrawka
When you are learning that new JS framework, do you not keep going through
their docs and googling?

I would say the documentation and google results for a decade old tool would
be more available when you try to do something a bit complex.

~~~
badman_ting
> When you are learning that new JS framework, do you not keep going through
> their docs and googling?

Yes -- and then I proceed to use the thing every day. I wouldn't be adjusting
a Makefile very often so I would keep forgetting whatever I "learned" every
time I changed it.

~~~
mathrawka
Let's say the JS framework is Grunt.

If you are using Grunt everyday, doesn't that mean you are modifying it? Then
you would be in the same exact place as with a Makefile.

~~~
lmm
The point is that the JS syntax of the gruntfile is a syntax I'm using every
day. Whereas make's syntax rules are bizarre and used nowhere else (IIRC it
makes a distinction between tabs and spaces?)

------
hhm
For my own projects, I have found make's alternatives (eg. scons) to be a much
superior choice. Make is just too complex for what it is intended to do (at
least as used with autoconf, etc.).

------
Nemcue
It's shit like this, HN. Why you guys make this nonsense end up on the
frontpage is honestly beyond me.

So this guy re-implemented the most useful grunt tasks in his own language.
Grats. You've wasted time instead of using Grunt. Saving time and not re-
implementing things was the point all along. Very few people want to write
"small programs" to chain together build tasks. They should be readily
available, just work, be continously updated, and work cross platform. I don't
want to go look into the intricacies of compiling Handlebars templates just to
do that shit.

Also, a make file syntactically looks like garble compared to a well honed
Gulp / Broccoli file.

~~~
fnl
That's because, I may presume, you have never (really) used make. For somebody
who does, exactly the opposite will be true.

~~~
shadowmint
No, read his comment carefully.

Grunt comes with a set of existing (and maintained by the grunt authors)
plugins for many common tasks that lets you get away with having someone else
do all the complex shell-script-custom-language-scripting you need to do with
make for the 'complex logic bits'.

Writing those by hand is a terrible and tedious burden in using make.

------
_pmf_
The hipster rage is string in this thread.

------
asabil
Couldn't agree more

------
NAFV_P
IIRC make has a lot of macro.

------
icantthinkofone
More and more I notice everything eventually comes back to Unix and C or looks
and acts like it.

And that's a good thing.

------
leccine
This is the end if the tech industry right here. make, really?

