
Count me as a systemd convert - KKKKkkkk1
http://changelog.complete.org/archives/9655-count-me-as-a-systemd-convert
======
bbayles
I didn't really know anything about init systems until 2014 (so I all but
missed the systemd wars) when I found myself in need of process
synchronization and supervision on various platforms.

My task was pretty straightforward: I had a group of processes that need to
start in a certain order, restart if they die, run as a non-root user, and
write logs somewhere. I needed to support RHEL 5, 6, and 7, which all use
different init systems (sysvinit, upstart, and systemd). I worked with all of
them and also experimented with alternatives (runit, s6, Supervisor).

For my task systemd was a pleasure to work with compared to the others. There
was a bit of a learning curve, but once I mapped out what I needed to do the
solution was obvious.

I found upstart to be almost good, but the version that ships with RHEL 6 is
lacking compared to the later versions that came with Ubuntu. The Python-based
Supervisor was surprisingly good. I sort of saw why some people might like s6,
but runit's charm was lost on me.

sysvinit was by far the worst. I wound up using sysvinit to hand off to
Supervisor in order to not spend my life worrying about the number of forks
and pidfile locations.

I can't speak to systemd's shortcomings and perceived hegemony in other areas,
but when it comes to acting like an init system it is miles ahead of
everything else.

------
nickysielicki
This is maybe pedantic, but something that irks me about systemd is the way
that systemctl commands are composed.

I find myself doing this frequently

> systemctl status someservice

> systemctl stop someservice

> systemctl edit someservice

> systemctl daemon-reload

> systemctl start someservice

> systemctl status someservice

> systemctl enable someservice

Why make the operation the middle parameter? I understand that "systemctl
start foo" more closely resembles the English sentence, " _System_ D, please
_start_ the _foo_ service", but I couldn't give two shits about that, and it's
a dumb design decision. Any time that I am entering a systemctl command, I'm
probably entering a series of 3 or more of them, and they're probably all
related to the same service.

Instead of being able to press <Up Arrow>, <C-Backspace>, and type my
operation, I'm instead having to position my cursor to the second argument.
It's a mess!

~~~
nostalgiac
That wouldn't really make sense at all.

Can you name any commands or interfaces that take the variable before the
argument?

~~~
vacri
cp, rm, rsync, scp tar...

    
    
        cp [list of a great many items] [here]
    

or service

    
    
        service [item] [action]
    

whereas systemd is

    
    
        systemctl [action] [list of great many items]
    
    
    

There appears to be no hard and fast rule, though - argument ordering is
different to flag ordering (-f -z etc). I never see anyone put flags at the
end, though plenty of tools work fine that way.

My problem with systemctl is that it's so woeful to tab-complete. At
'syst[TAB]c[TAB]', you may as well just type out the whole thing, because
tabbing only saves you two characters. Why not just take 'ctl', that doesn't
seem to be taken by anyone?

~~~
JdeBP
GNU toolset:

    
    
        cp -t [here] [list of a great many items]
    

And there is one area outwith Unix but still at the command line where one
does sometimes see people putting flags at the end (even though this is not
the documented syntax):

    
    
        dir *.txt /s
    

Be wary of the service command, incidentally.

* [http://homepage.ntlworld.com./jonathan.deboynepollard/Softwa...](http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/bsd-service-command.html)

As for "ctl": Bear in mind that there are _a lot_ of "ctl" commands. initctl
(from upstart), netctl (from Arch), rcctl (from OpenBSD), pfctl (from
OpenBSD), ip6addrctl (from FreeBSD), ntpctl (from OpenBSD), hastctl (from
FreeBSD), swapctl (from 4BSD), rctl (from FreeBSD), pppctl (from FreeBSD),
smartctl, journalctl, bootctl, localectl, busctl, coredumpctl, hostnamectl,
machinectl, timedatectl, loginctl ...

Pick unadorned "ctl" for the systemd one, and one begs the question of why
systemd alone is special enough not to require its name in the command.

~~~
majewsky
The "-t" switch in "cp" exists to support the use with xargs, e.g.

    
    
      find /some/directory/ -name *.txt | xargs cp -t all-the-textfiles/
    

The systemctl interface has the same useful property:

    
    
      systemctl --failed | awk '/.service/{print$1}' | xargs systemctl restart

------
digi_owl
[https://news.ycombinator.com/item?id=11639659](https://news.ycombinator.com/item?id=11639659)

Found this regarding programming languages (and JS in particular) a bit
interesting in the middle of the whole systemd debate, as i wonder if it can
be applied to unix as well.

This in that unix have from the early days been made up of loosely coupled
parts. This then allowed new parts to be introduced, tested, and either
adopted widely or discarded as seen fit. Maybe even adopted in some areas but
ignored in others. But you rarely ran into deep dependency chains.

Systemd on the other hand is tightly coupled, continually sprouting new stuff,
and foisted on the world via deep dependencies.

~~~
creshal
> Systemd on the other hand is tightly coupled

Not really. All its various components are loosely coupled over DBus, and most
are entirely optional. There's no technical reason why people can't e.g.
implement journald's API in syslog. Or replace the various userspace *ctls
(timedatectl, hostnamectl, …), or reimplement logind on top of some BSD
(already done, actually), …

~~~
digi_owl
You seem to be confusing the interface between systemd components and third
parties, with internal interfaces.

The latter are expressly non-documented, and non-stable, to allow the systemd
devs to change them as they see fit.

As for logind on BSD, got a link? Last i recall they were struggling because
it was a sprawling and moving target (Consolekit2 is also having that
problem). One example is that recently logind took over much fo the job upower
used to do, with upower now being a compatibility wrapper around logind.

Oh btw, dbus is massively more opaque that being able to run a command, look
at the output, and then pipe that to another with the proper flags and such.

------
combatentropy
The choice isn't binary: sysvinit or systemd. There is a third way. I'm not
much of a sysadmin, but I'm intrigued when I read of daemontools and its ilk.

[http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/dae...](http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/daemontools-
family.html)

[http://blog.darknedgy.net/technology/2015/09/05/0/](http://blog.darknedgy.net/technology/2015/09/05/0/)

~~~
ashitlerferad
There are many more than 3 init systems. There are tens of them.

~~~
combatentropy
I meant three families, but you still may be right.

~~~
r0fls
Where does upstart stand in this? That's what I've used so far.

~~~
ashitlerferad
It is dead and gone, Canonical/Ubuntu abandoned it after Debian chose systemd
and then Ubuntu did too.

------
digi_owl
One "interesting" tidbit about systemd is that the init part will keel over if
dbus goes down, and if pid1 keels over you are looking at a kernel panic...

~~~
kinkdr
I was bitten by this once. A typo in dbus configuration rendered my machine
unbootable.

~~~
ZenoArrow
I wonder if issues like this were part of the reason the release of kdbus
(dbus as a kernel module) was delayed.

~~~
tremon
kdbus is mainly delayed because the kernel devs are unconvinced. In the last
thread I read (which was likely over a year ago), there were architectural
issues with the design that nobody on the kdbus side was willing/able to fix
or explain.

~~~
ZenoArrow
I thought one of the main kdbus devs was gregkh (one of the most senior Linux
kernel maintainers), is that not the case? I did hear that there was due to be
a redesign, but not sure what the issues were.

~~~
lomnakkus
I believe this is the redesign:
[https://github.com/bus1/bus1](https://github.com/bus1/bus1)

EDIT: Replaced with better link. (Old link:
[http://www.bus1.org/index.html](http://www.bus1.org/index.html) )

~~~
JdeBP
[https://news.ycombinator.com/item?id=10724206](https://news.ycombinator.com/item?id=10724206)

(-:

------
pkaye
I have to say that it is super fast to boot up with systemd. Maybe 2-3 seconds
to boot to my Ubuntu desktop (excluding bios time.) I do use an SSD however.

~~~
nitrogen
On embedded devices in the SheevaPlug family, it's a bit slower than the old
init, presumably because it's doing a lot more stuff (and the OS is either on
slow flash or a slow USB-to-SD card interface). It doesn't seem to scale down
to small systems very well. I do like the colors, though.

~~~
ashitlerferad
systemd has boot profiling functionality, you might want to check out which
part is slow.

~~~
nitrogen
There's always lots of text flying by on the console; it might not be
traceable to one single slow part.

~~~
pkaye
Try "systemd-analyze plot >boot.svg" and it will produce a nice svg graph of
how much time each step takes.

~~~
majewsky
Also check out the other subcommands. For example, to find out why a certain
service is taking long to start up, "systemd-analyze critical-chain $service"

------
api
IMHO sysvinit and systemd both suck in that they're both overly obscure,
overly arcane, overly complex, and poorly documented. Systemd replaces a
legacy pile of hacks mess with an over-engineered tower of babel mess.

An improvement would be something with modern syntax, sparse features, and
good docs.

~~~
SEJeff
On your Linux system, type:

man -k systemd

There is extensive and _well written_ documentation for all aspects of systemd
so I'm going to call BS on your post. Also, the syntax is consistent and
documented. It is infinitely simpler than sysvinit. Either you're trolling, or
know very little about systemd. Note that I'm a fan, but far from an
apologist.

~~~
ktRolster
Here's the documentation for systemd-random-seed:
[https://www.freedesktop.org/software/systemd/man/systemd-
ran...](https://www.freedesktop.org/software/systemd/man/systemd-random-
seed.service.html)

What are the command-line options that are available to that tool? Without
looking in the source code.

>Note that I'm a fan, but far from an apologist.

An apologist is merely someone who offers a defense of something. It comes
from the greek word ἀπολογία, "speaking in defense". Thus you are an
apologist, even though you're not dogmatic.

~~~
Grishnakh
>It comes from the greek word ἀπολογία, "speaking in defense".

I just want to point out here that just because an English word comes from
some Greek word, that doesn't mean the English word actually means that today.
You are correct about "apologist" (according to Wiktionary at any rate), it
just means someone who speaks in defense of something, but there are other
words where the meaning has changed over time. English is a language that is
defined by its popular usage, and this can change over time so a word may not
mean the same thing today as it did just 100 years ago.

A good example is the word "pathetic". According to Wiktionary, it comes from
Greek:

'παθητικός ‎(pathētikós, “subject to feeling, capable of feeling,
impassioned”), from παθητός ‎(pathētós, “one who has suffered, subject to
suffering”), from πάσχω ‎(páskhō, “to suffer”).'

Obviously, those don't bear that much resemblance to the current popular
meaning, which is:

'1.Arousing pity, sympathy, or compassion.' or '2.Arousing scornful pity or
contempt, often due to miserable inadequacy.'

In fact, other European languages like German have this same word (I believe
this is called a "cognate"; I am not a linguist), from the same root, but
their meaning is entirely different and they get confused when they learn
English and encounter this word since we use it in such a different manner.

TL;DR: Greek or Latin roots are interesting from a historical and linguistic
perspective, but they're no guarantee of the current definition of an English
word.

~~~
tremon
_I believe this is called a "cognate"_

A cognate is simply a pair of words from the same stem (origin); it doesn't
matter if their meaning is still the same. The opposite term is "false
cognate", which refers to words that sound alike and mean something similar
but stem from different roots.

Regardless of origin (so cognate or not), if two words in different languages
sound alike but have different meaning, they're called false friends.

</pedant>

------
B1FF_PSUVM
> I was debugging the boot on a server recently. It mounts a bunch of NFS
> filesystems

Don't do that. It's a bag of hurt.

(Still sore about the hours that workstations with not-working-fs mounts spent
waiting for each other on boot after power failures, ages ago. Nevermore.)

~~~
ibotty
So you just don't use remote filesystems? That's a solution!

Honestly, whenever the init system is configured correctly, you don't run into
these problems all ops know a story about. It helps that systemd makes it
easier to configure dependencies so that only the units depending on remote fs
stall when it is unavailable.

~~~
jabl
> So you just don't use remote filesystems? That's a solution!

No, you mount them with autofs, of course.

~~~
karlshea
I found out about autofs when I was trying to figure out some SMB issues while
mounting NAS shares on a media server.

It was so ridiculously easy to set up. All of the "copy and paste" newbie
answers always involve modifying fstab and there are always issues on boot,
but autofs just works. It makes me wonder why it's not recommended as often.

------
oconnore
I think the systemd complaints can essentially be reduced to "my init system
now knows about cgroups, sockets, dbus, [and other things that didn't used to
exist]". I haven't heard anyone offer a replacement that accounts for these
features while maintaining "the Unix way".

~~~
pdkl95
None of those feature require systemd.

> cgroups

Works fine on my Gentoo (OpenRC) desktop. This meme that systemd is necessary
for cgroups is simply incorrect.

> sockets

We stopped using xinetd a loooooong time ago. If you want to use a central
point of failure, that's your business.

> dbus

Isn't necessary in the init process, though you're free to use it if you wish.

> didn't used to exist

Such as?

> replacement for these features

The replacement in some cases should be "nothing". You're trying to frame the
conversation in a way that presumes features like cgroups or "socket
activation" are desired features.

The entire point of this "debate" is that you shouldn't be tying extra
features to the init process. There are a wide variety of systems in the Linux
ecosystem, and there is no one-size-fits-all solution.

If _you_ want to run a heavyweight, opinionated, monolithic init like systemd,
that's fine. If it makes your life easier, that's great. Just remember that
other people have different requirements.

~~~
vidarh
> Works fine on my Gentoo (OpenRC) desktop. This meme that systemd is
> necessary for cgroups is simply incorrect.

That's not what it's about. The point of cgroups integration in systemd is
that it allows systemd to precisely track the processes started under it,
which means that e.g. applications where things like double-forking or similar
happens can still be completely stopped by systemd even if the parent process
des.

> You're trying to frame the conversation in a way that presumes features like
> cgroups or "socket activation" are desired features.

They are to me. They massively improves things vs. sysvinit.

> The entire point of this "debate" is that you shouldn't be tying extra
> features to the init process.

The entire starting point for systemd is that traditional sysvinit is woefully
lacking in the functionality required to build reliable systems. And when you
start trying to cobble together something on top of it, it quickly becomes
clear that it is not easy.

E.g. try to build a reliable process manager outside of pid 1. You still need
process restarting functionality in pid 1 for when your process manager dies.
What if it fails to restart? How are you going to log those failures reliably?
You end up duplicating a lot of functionality of a process manager in pid 1 if
you want a reliable system even if you run a separate process manager outside
of pid 1.

There are parts of the systemd design I don't agree with, and I think they'd
benefit from splitting out various components from the project, not least for
"political" reasons. But the design for the core parts of systemd-the-init-
system largely follow on from a very limited set of core requirements of being
able to reliably start, stop and keep services running and reliably log data
from them.

These are requirements that sysvinit basically throws it hands up at and says
"not my table", while by owning pid 1 they make it basically impossible to
meet them.

~~~
pdkl95
> The point of cgroups integration in systemd

I know why systemd uses cgroups. I was addressing the myth that systemd is
_required_ to use cgroups.

> They are to me.

That's nice. You should use systemd if its features satisfy your requirements
better than the other options.

Other people have different requirements. Is this concept difficult to
understand? We don't want the same features, for a variety of reasons.
Disagreement is _fine_ , but systemd advocates often seem to want a
monoculture.

> the functionality required to build reliable systems

That's your _opinion_ , which you're entitled to. I disagree, as I've never
had problems like you're describing. I value stability a _very_ strongly, so I
consider the methods that worked fine over the past 20+ years.

> requirements

That's the framing I was talking about. There is more than one way to design a
stable system, and your requirement list is only one of those methods. Some of
those requirements I consider _misfeatures_ that cause problems, but _it 's
fine if you disagree_.

> impossible to meet them

Then use systemd or some other solution that solves your needs. Is it so hard
to understand that we can choose different software?

~~~
vidarh
I'm not arguing against your ability to pick something else.

What I see, however, in most of the opposition to systemd is that most who
criticise it don't understand these design considerations and often aren't
even aware of the problems involved.

This is why they keep coming up. If most people designed systems that took
care of these things in other ways, it wouldn't be so much of an issue. But
most people don't. E.g. the fact that most Linux distros for the longest time
kept running critical system services straight from init scripts with no
process manager or other restart logic is a good example.

~~~
digi_owl
And oddly enough those "services" (or daemons as they used to be called,
wonder why that changed) could run for years if not decades without a hitch.

But now everything seems to be a case of web monkey balls of yarn that is no
better than Windows in staying upright.

Its like i can reuse old UF comics, only replace Murky's love for Windows with
Ubuntu or some such.

------
toomanythings2
>I still have serious misgivings

I guess that kinda tells us more than anything else he said.

~~~
kl4m
You cut the quote to change the meaning.

"Although I still have serious misgivings about the systemd upstream’s
attitude, I’ve got to say I find the system rather refreshing and useful in
practice."

~~~
toomanythings2
There are misgivings none-the-less. A new system being "refreshing" is not a
reason to use it. "Useful" is hardly a ringing endorsement.

What disturbs me the most is that systemd takes Linux farther away from being
a Unix-like system. Linux is now "Linux" and incompatible with anything else.

