
Understanding Daemons - aryamansharda
https://blog.digitalbunker.dev/2020/09/03/understanding-daemons-unix/
======
zokier
With systemd (and afaik other service managers) you don't need/should to do
all that daemonization dance. The application can run in the foreground, keep
any fds open, don't worry about tty etc because the service manager will take
care of them for you. As a cherry on top, anything you write on stdout gets
automatically forwarded to journald or wherever.

further reading [https://www.man7.org/linux/man-
pages/man7/daemon.7.html](https://www.man7.org/linux/man-
pages/man7/daemon.7.html)

~~~
vimax
The more I use systemd the more I love it. It simplifies so much, and adds all
the little features I see so many projects using custom implementations for.

The people who keep complaining about it seem more and more like children who
just refuse to learn something new and better because they've already stumbled
through all the holes it fills.

~~~
pydry
I was a fan of upstart. It was simple, it knew how to perform the task it did
it, it didn't try to be everything plus the kitchen sink, it worked well and
it didn't get its tendrils in everything like systemd does (something that
concerns me as it increases the attack surface).

It was a technically superior solution murdered by politics.

What's worse was all of the people who were going "why are you moaning? what's
wrong? systemd _is still_ an improvement on sysvinit!" as if being better than
a bunch of bash scripts held together with bits of sticky tape and string was
all the qualification needed to be PID 1.

~~~
JdeBP
People liked to paint it as politics, but there were technical critiques such
as Russ Allbery's for starters.

* [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727708#172...](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727708#1729)

And yes, people do erroneously paint it as only a choice between van
Smoorenburg init+rc and systemd, as if they were the only two possibilities,
when in fact in the Debian Hoo-Hah (for one) it was effectively a choice
between systemd, upstart, and OpenRC, nearly all parties agreeing at one point
in the proceedings that van Smoorenburg init+rc was last choice, either before
or after "further discussion".

* [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727708#672...](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727708#6729)

~~~
pydry
The criticisms levied at upstart there are more about lack of features, not
really about overall approach. It's easier to add features than it is to fix a
deficient design.

Moreover, I'm not really so sure a couple of those features even _belong_ in
an init system.

Debian's decision effectively doomed upstart and led to an init system
monoculture. The practical upshot is that no competition for systemd means no
pressure to improve. Even for systemd it was a bad decision.

~~~
JdeBP
Actually it was about overall approach. Read M. Allberry's further discussions
in that bug with Colin Watson, Steve Langasek, and others. Josh Triplett
critiques it as well. Cameron Norman's discussion with Nikolaus Rath is also
relevant. There were actually quite a lot of technical discussions about the
fundamentals of the various architectures. Ian Jackson's and others'
discussions about readiness protocols is worth reading, for example.

One cannot truly portray this as either "politics" or only about lack of
features.

* [http://jdebp.uk./FGA/debian-systemd-packaging-hoo-hah.html](http://jdebp.uk./FGA/debian-systemd-packaging-hoo-hah.html)

* [http://jdebp.uk./FGA/unix-daemon-readiness-protocol-problems...](http://jdebp.uk./FGA/unix-daemon-readiness-protocol-problems.html#Choice)

------
pmiller2
> Simply put, a daemon (pronounced dee-mon) is an intentionally orphaned
> background process. Additionally, daemons are detached from the terminal in
> which they’re spawned, operate without user interaction, and are descendants
> of the system’s init process.

A memorable way to think about this is that a daemon is a child process whose
parent has literally disowned it, forked off, and died.

~~~
JdeBP
... except that that's most definitely _not_ the defining characteristic, and
the defining characteristic is rather that the process isn't part of a login
session, which ironically on modern operating systems with all of the one-way
trapdoors that now exist one cannot reach by forking.

* [https://unix.stackexchange.com/a/606902/5132](https://unix.stackexchange.com/a/606902/5132)

Thinking that forking makes a dæmon is much the same error as thinking that su
drops privileges, it conflates what happend to be a _mechanism_ from years ago
with the actual purpose, totally missing that things have changed since (long
since, decades ago in fact).

~~~
kreetx
What do you mean by the one-way trapdoors?

~~~
JdeBP
See the StackExchange answer.

------
macintux
Of all the great content in APUE, I think W. Richard Stevens’ (RIP) coverage
of daemonization is the only piece I’ve ever used.

Fortunately I recently replaced my lost 1st edition with a 2nd edition, and
the coverage looks similar. From what I can see there are two details left out
of this article:

* _Why_ changing the working directory is important (so the daemon doesn’t accidentally prevent a file system from being unmounted in the future).

* WRS emphasized logging.

Some of the details are different (umask as the first function instead of the
last), but the highlights are comparable.

------
emmanueloga_
I was hoping this would be an article for beginners to understand daemons, as
in the Ancient Greek daimon (δαίμων: "god", "godlike", "power", "fate") :-)

Growing up on a (quasi fundamentalist) christian household, daemons were for a
long while a very real threat, and my mom would not allow me to watch horror
movies, "receive things from strangers", an other procedures to protect us
from "inviting the daemons home".

My rational mind leaves no room for such creatures these days, but even so
under the right circumstances, sometimes my lizard brain begins to think maybe
the Bible (and my mom for that matter) was right :-). Hard to erase some
emotive memories from childhood I guess.

A while ago I read this book [1] which is pretty cool, covers the mind
mechanisms behind things like the placebo effect, "daemonic possession", alien
abductions, mass hysteria, out of body experiences, etc...

1: [https://www.suggestibleyou.com/](https://www.suggestibleyou.com/)

~~~
Skunkleton
Usually I see δαίμων latinized as "demon" not "daemon". You got me looking and
I found an interesting discussion on these words over on stack exchange [1].
TL;DR, "daemon" sometimes means "demon", but without the negative
connotations, other times it is just an alternative spelling.

[1] [https://english.stackexchange.com/questions/39266/what-is-
th...](https://english.stackexchange.com/questions/39266/what-is-the-
difference-between-daemon-and-demon-in-a-religious-context)

~~~
emmanueloga_
Oh! The good daemons! "benevolent or benign nature spirits...". Not in the
Judeo-Christian tradition, for sure :-)

Also got me curious about the C.S. ethymology of the word. According to the
venerable Jargon File (which has entries for both "daemon" and "demon"), it is
named in honor of one of Maxwell's thought experiments! [1]

1:
[http://www.catb.org/jargon/html/D/daemon.html](http://www.catb.org/jargon/html/D/daemon.html)

~~~
Skunkleton
Ahh, neat. Thanks for the link.

------
timw4mail
Going by the spelling, isn't it pronouced day-mon?

~~~
jpxw
No, in the same way that paediatrician isn’t pronounced paydiatrician

~~~
rgoulter
To be fair, it's not obvious how to pronounce a way it's spelt. e.g. "record"
and "record" are pronounced differently, but have the same spelling.

The poem "Dearest creature in creation" is a fun illustration of the
differences between spelling and pronunciation.

~~~
teddyh
It’s called “ _The Chaos_ ”:

[https://en.wikisource.org/wiki/The_Chaos](https://en.wikisource.org/wiki/The_Chaos)

------
tsimionescu
This function may do mostly the right thing in a process dedicated to it, but
it is not safe to call as part of a more complex process - especially a multi-
threaded one.

For example, malloc() may be holding on to a mutex in the parent process (on
another thread), so subsequent malloc() calls in the child process will block
waiting for the mutex to be released. Closing the file handles may help if
this was a kernel mutex, but it may not help if this was a user-space mutex.

In general, fork() without exec() is only safe as the first thing done in a
process.

Also, it looks like there is a minor race condition in the implementation:
since the original process exit()s immediately after the fork(), control will
return to its parent, which may close the session - this could happen before
the child had a chance to call setsid() and detach from the parent session.

------
wruza
A good practice for a classical daemon and any program in general after
closing all descriptors is to reopen first three (0,1,2) to /dev/null. After
that, any forgotten or unexpected diagnostics (rare printf in third party
library) would not corrupt your open files.

~~~
JdeBP
No, that's a _bad_ practice, because since the late 1990s service management
subsystems have been as the norm setting up standard output and standard error
so that they are connected to loggers. daemontools did it, once it gained
svscan; and so did the other toolsets in the daemontools family such as runit,
perp, s6, and others. systemd also does it, but is very much a latecomer here.

Furthermore, this bad practice lands one in hot water with library functions,
including ironically syslog client libraries, that open and retain file
descriptors internally for their own purposes. Yet another case of this was
just on Hacker News this week, in fact, at
[https://news.ycombinator.com/item?id=24329540](https://news.ycombinator.com/item?id=24329540)
.

~~~
wruza
That's still a good practice, _if_ you close descriptors. Such criticism and
trouble comes from environments that are unable to follow the routine because
reasons, e.g. they openlog() and construct other objects before daemonizing.
Given how much can happen before python's main() to be called, it is obvious
that daemonizing python-anything is a bad idea to start with. It only works
for few controlled steps at int main(), after that you're screwed. All these
pitfalls are discussed in literature, the real problem is that barely anyone
bothers to read it and think for a second on their options. One of them being
to use a proper service manager instead, like systemd.

~~~
JdeBP
No-one can follow the routine. "daemonization" is a fallacy.

------
nemetroid
> Conventionally, daemon process names end with the letter “d”. If you’re
> interested in seeing all daemons installed on your Linux machine, use this
> command:

> service --status-all
    
    
      $ service --status-all
      bash: service: command not found

~~~
hyper_reality
Perhaps you're trying to make a point, but the command should work with
/usr/sbin/service or prefaced by sudo. If you run `whereis service` you can
see that the binary is located in /usr/sbin (at least on Debian/Ubuntu
systems) which is typically placed in the PATH of root but not normal users.
However --status-all doesn't require elevated permissions so the command works
if you provide the full path.

~~~
nemetroid
A bit of both. My computer does not have the "service" command at all. While
it's true that many systems still do, it's typically there as a legacy
compatibility layer, to translate SysV style commands to systemd.

So while the command might work for many readers, I don't think it's a good
idea to send beginners that way, and its inclusion (without any footnotes or
other qualifications) suggests that the author's knowledge is out of date.

------
bfuclusion
So how does this work in windows? Edit, as for some reason it's getting
downvoted: In windows, do you have to do the double fork etc? Do you have to
close FDs etc?

~~~
tsimionescu
Windows doesn't have fork() or any direct equivalent (instead, Windows has the
CreateProcess() familiy of functions, that work essentially like
fork()+exec()).

Now depending on what you actually want to do, you have some options:

\- Use the Windows Services API to ask Windows to start and manage your
process as a system service - this is closest to a SystemD demon

\- Create a GUI process with WindowStyle=hidden (this will be part of the
current user's session)

\- Create a Console process with the DETACHED_PROCESS flag, which should work
similarly to the above

\- Use one of the CreateProcessAsUser() family of functions to create a
process for another user - this should allow the process to run in the
background, and allow it to persist even if the current user signs off

In all cases above, you can simply tell CreateProcess* that the child process
should not inherit any FDs. You should probably also use the
PROC_THREAD_ATTRIBUTE_PARENT_PROCESS flag to spawn this process as the child
of some other long-running process, to ensure that the child process is not
part of the same Job that the current process may be (usually, when a Job is
finished, all processes in that job and any child they spawned, recursively,
are killed).

~~~
Joker_vD
Jobs, more precisely, nested jobs, are one of the best things (IMHO) that were
added to Windows 8 from a technical point of view (then again, there ain't
many good things added in Windows 8). I had to write a work-manager sort of
application that would need to start worker subprocesses (which may also start
some other subprocesses, which may or may not be programs under my control)
and one of the requirements was that if the main process crashed, all the
worker processes should be terminated (imagine if a part of what a worker
process is supposed to do is to listen on some well-known TCP ports and
provide an HTTP API on it: you can't restart the main process without killing
the workers off first).

On Linux, it turned out to be surprisingly tricky. Process groups and sessions
are not nested, you can easily break out of them and in fact there were some
programs we needed to use that insisted on doing daemonization. Thankfully the
most important one had a "\--foreground" flag but it still was buggy: this
program sends SIGTERM to its process group as part of a shutdown — so it kills
its parent, too. Amazing. In the end I believe we ended running each worker
process as a separate Docker container.

Then there is a dance of waiting (or not waiting? SIGCHILD is truly horrible)
on children PIDs to detect the crashes and unexpected hang in the shutdown
(the worker process sends "I am done" notification to the parent, but then for
some reason doesn't exit). The main process had to be made a sub-reaper to do
all of that more or less reliably, and even then there were some strange
sightings (signal handling is fun!).

On Windows, we just shoved each worker process into its own unbreakable-from
job, and that was it. The main process dies for whatever reasons, its children
just disappear. A worker process dies, its children just disappear, without
affecting any other worker processes and their children. Detecting termination
of the child processes too was a breeze: on Windows, when you spawn a child
process you get not only its PID, but also a handle (think "file descriptor")
to it. This handle refers exactly to this process, and you can (interruptibly)
wait on it, and when the last open descriptor to a process is closed, the
process is deleted from the process table. That's why BTW there is no PID 1
that is constantly reaping its children on Windows: each process implicitly
has its own handle alway open, so if the parent exits before the child does,
nothing bad happens: The child process had 2 open handles on it, now it has 1.
When it itself exits, the last handle is dropped, and the slot in the process
table is cleaned.

TL;DR: Process management is horrible, use Erlang.

------
totemandtoken
Very timely to see this. I was trying to daemonize some celery workers/celery
tasks and I wasn't sure how to go about it. Glad this came up

------
tus88
They have a double fork - forked tongue and forked tail, and often cannot be
killed through normal means, re-spawning with the aid of an even greater evil
known as SystemD.

