
GNU Make Jobserver Implementation - krakensden
http://make.mad-scientist.net/papers/jobserver-implementation/
======
lelandbatey
Toward the very end, the author gives a brief one paragraph explanation of why
they didn't use multiple threads:

    
    
        Obviously one idea is to use threads. It’s common to have a separate thread
        for dealing with signals and combining threads with semaphores, etc. would
        allow GNU make to wait on two different types of asynchronous events
        simultaneously. This was rejected because so far GNU make was single-threaded,
        and requiring multi-threading was deemed too much of a price to pay.
    

I'm confused as to why adding multi-threading would be quite so onerous. As I
understand it, one could restrict the use of threads to that specific feature
and thus avoid large architectural changes.

Do I not understand how Make is designed, or is "the price to pay" in
reference to performance or some other metric?

~~~
hyc_symas
As I noted in my own mention
[http://highlandsun.com/hyc/#Make](http://highlandsun.com/hyc/#Make) I
originally wrote that patch back in 1991. There was no such thing as threads
back then. Even in 1999 the POSIX threads spec was only up to Draft 6 or so,
not yet finalized. Nobody sane would write production code for such a moving
target (oh, except maybe Microsoft, using Draft4 to create the Windows NT
threads API).

~~~
JdeBP
That parenthetical history is wrong, at least according to Dave Cutler. He
traced Windows NT's threading back to his work on VAXELN in 1983, years before
POSIX was even thought of. He also was quite obviously deriving the NT API in
part from OS/2, again not POSIX, given the whole "mutant"/"mutex" affair.

~~~
hyc_symas
Windows NT RPC is DCE RPC - that is an indisputable fact. DCE used POSIX Draft
4, and Windows copied DCE pretty completely there.
[https://en.wikipedia.org/wiki/DCEThreads](https://en.wikipedia.org/wiki/DCEThreads)

~~~
JdeBP
Well now we know where all of your errors are coming from. Regrettably, you've
learned a lot of bunkum, that you now have to un-learn.

The source of that bunkum, or at least the source that you've pointed to, is a
canonical example of how rubbish turns into gospel via Wikipedia. It started
out, in August 2005, as an intentionally opinionated rant written by someone
who didn't supply xyr name, who clearly didn't know the subject, and who
pointed to entities that didn't exist at the time in question as xyr
authorities.

* [https://en.wikipedia.org/w/index.php?title=DCEThreads&oldid=...](https://en.wikipedia.org/w/index.php?title=DCEThreads&oldid=21947270)

Other people came along and tried to take the opinions out and leave the
facts. But the underlying problem was that the facts weren't factual at all,
and were as warped as the rest of the rant was; and no-one with any actual
knowledge of the topic, or research ability, has touched the article
significantly _in eleven years_.

* [https://en.wikipedia.org/w/index.php?title=DCEThreads&type=r...](https://en.wikipedia.org/w/index.php?title=DCEThreads&type=revision&diff=641382127&oldid=23011764)

And in the meantime, people come along, believe Wikipedia uncritically, and
regurgitate a load of old tripe.

Here are some of the things that you have to learn in place of the un-learned
junk:

* There were such things as threads in 1991. There were such things as threads in 1988 when Dave Cutler started on "NT OS/2" at Microsoft. There were such things as threads in 1987 when OS/2 version 1.0 was announced. And indeed in 1983 in VAXELN according to Dave Cutler.

* Threads are not RPC, DCE RPC or otherwise.

* The design of Windows NT threads pre-dated the _existence_ of 1003.4a draft 4 (dated 1990-08-10). By years.

For far better things than Wikipedia to read on this subject, see:

* _Inside Windows NT_. Particularly page 92 of the 1st edition.

* Cutler's own _Windows NT Mutant Specification_ , written in 1989.

* Mark Russinovich's writings on the subject of Windows NT and its VMS heritage.

* Any good book on OS/2 version 1.0 programming that details the threads API.

* Michael Yam's 1996 _Dr Dobbs_ article on porting his wrapper class library from DCE to Win32 threads.

* Any number of books on the subject of DCE, explaining the two quite different things that are DCE RPC and DCE threads.

------
amelius
> a traditionally hairy problem in UNIX programming: how to wait for two
> different types of events (signals and file descriptors) at the same time.

I've always thought one of the weak points of POSIX was that the event sources
were not unified. Imho, is a pity the relevant committee didn't work towards a
clean solution in the mean time.

~~~
okigan
Same here! Also annoying is inability to wake up when any (vs all) child
threads are done.

Even more annoying is that Windows API allows to wait on any combination of
file/socket descriptor and/or message[1].

[1] MsgWaitForMultipleObjects [https://msdn.microsoft.com/en-
us/library/windows/desktop/ms6...](https://msdn.microsoft.com/en-
us/library/windows/desktop/ms684242\(v=vs.85\).aspx)

~~~
uep
I know we're talking about POSIX here, but Linux does have the ability to wait
on signals the same way it waits on file descriptors. It does this by also
allowing signals to be delivered on file descriptors.

[http://man7.org/linux/man-
pages/man2/signalfd.2.html](http://man7.org/linux/man-
pages/man2/signalfd.2.html)

~~~
hyc_symas
Gratuitous fluff. Writing to a pipefd inside a signal handler works on all
POSIX systems.

------
some1else
We need to use Make for sensible (partial) rebuilds of static websites. I'm
exploring ways to build a dependency graph out of Handlebars files. That might
be enough to generate a Makefile and leave Gulp/Grunt out of the equation.
Anyone doing anything similar?

~~~
saosebastiao
Not for static websites, but I've definitely done very similar with Knitr for
repeatable analyses in R (except for autogenerating the makefile). I create R
scripts, markdown templates, database extract jobs, and even take care of
sending the analysis results via email. I've left the job where I was
responsible for those reports, but they're still in use with a distribution
list that goes up to the SVP level at Amazon, so apparently they've been
pretty successful.

------
jahitr
GNU make is one of the most complete task manager I have seem.

But there are a few major problems with it:

Newies often complain a lot about the esoteric syntax.

It's written in C.

It doesn't gets updated as fast as it should.

These points make it not very suitable for our current time.

I belive we all just need to pick something like gulp and port the "good
parts" into it.

There is no point in having 20 tools that do the same!

~~~
hyc_symas
"written in C" is an asset, not a liability. It is still the one language that
is portable across the widest spectrum of platforms, as well as the most
efficient high level language for execution.

