
Interview: Lennart Poettering - talles
http://www.linuxvoice.com/interview-lennart-poettering/
======
vezzy-fnord
Lennart seems to harbor this very strange belief that developing your software
in one repository makes you Unix-like, and that this alone is sufficient
entirely.

So by this logic, a standalone Unix kernel is not Unix-like, but a fully
integrated DOS clone, is.

In addition:

 _Then we convinced the Fedora Technical Committee to adopt it, and then Red
Hat internal management accepted it for RHEL, and we managed to convince every
committee that mattered, bit by bit._

So there was, in fact, a PR campaign.

 _So Journald is a requirement, and Udev is a requirement. But pretty much all
other components are completely optional._

Though for some reason the build system doesn't let you disable a ton of these
components by default.

 _The committers group is quite diverse, and for us it’s quite an exercise in
making the diversity of the community be reflected in the diversity of the
people who work on it._

This is not as idealistic as Lennart portrays it. See Fallacy #13 here:
[http://judecnelson.blogspot.com/2014/09/systemd-biggest-
fall...](http://judecnelson.blogspot.com/2014/09/systemd-biggest-
fallacies.html)

~~~
mksaunders
"So there was, in fact, a PR campaign."

Er, how do you think the Systemd developers convinced these technical teams to
use Systemd? With bribes? Sexual favours? Or perhaps, you know, using the
software itself?

It's funny. The lead developers of Arch Linux, OpenSUSE, Fedora, Mageia,
Debian etc. etc. etc. have all chosen to use Systemd. Yet for some reason,
many commentators assume these people -- distro technical leads -- simply
don't have a clue, and have switched their projects to Systemd because Lennart
is somehow forcing it on them.

All these distro developers, people a LOT smarter than me, and putting in more
work than me, have chosen Systemd. That says a lot to me, and I'm not going to
judge them. I'm still on the fence about a lot of Systemd, but to say that
major distros have switched because of "PR campaigns" is ridiculous.

~~~
vezzy-fnord
Hold up. Please don't invoke the argument from popularity, it simply doesn't
work. The lead developers of major distros, as smart as they may be, do make
mistakes (Debian with cdrkit, sticking with libav for too long, the PRNG bug,
etc.) and they may not be fully in line with the intricacies and conceptual
differences between process management, init, supervision, hotplug and so on.
There's lots of misinformation from both sides and a lot of knowledgeable
people (like those maintaining powerful daemontools derivatives) have never
been ones to proselytize, but stick to their inner circles.

Furthermore, how are you sure the decision to adopt was entirely technical in
nature? Perhaps they realized that the incoming systemd-integrated Linux
desktop userland wasn't worth the maintenance hassle of trying to avoid it?

The way Lennart worded those sentences sounded funny. He is known to distort
and exaggerate when promoting, for which I don't blame him, since his goal is
to get his software adopted, but nonetheless. systemd proponents have been all
too eager to completely dismiss any political meddling on part of systemd's
adoption as totally unsubstantiated, insisting it was purely "technical
merits". Lennart's words paint a subtly different picture, at the least.

Most people don't fully grasp the state of Unix process management, so it's no
surprise they're susceptible. For example, the only alternatives that have
been debated primarily were Upstart and OpenRC. That's a red flag.

~~~
mksaunders
"Please don't invoke the argument from popularity, it simply doesn't work."

Even when almost every distro has switched? If the situation were 50-50, I'd
agree with you. But given that Systemd has been overwhelmingly adopted by
distros, I think that says alot.

"The lead developers of major distros, as smart as they may be, do make
mistakes"

Yes, but again, they surely haven't all made a giant mistake, right? I don't
buy the "they've switched because of upcoming desktop userland" argument
either. Distro devs -- like Linux users in general -- aren't ones to roll over
easily.

With overwhelming support from distro developers, the people who know their
stuff, Systemd must be doing something right. And if you look at the Benefits
section here, you can see a lot of things that are appealing to distro devs:
[https://bbs.archlinux.org/viewtopic.php?pid=1149530#p1149530](https://bbs.archlinux.org/viewtopic.php?pid=1149530#p1149530)

------
nurb
"if you do an init system but still invoke all the shell scripts and all the
other things needed to bring up the system, you’ve only solved part of the
problem."

That's exactly what I think is wrong with systemd. If a script bring a speed
issue on start up, it should fixed by optimizing the script, use/create
another script language with parallelism and higher pref, or even turn it into
a compiled program. This is not supposed to be the problem of the init system
developper.

In fact the only purpose of an init system is to start executables in a
defined order, not more.

This is going to be like X, a huge pile of unmaintainable code, and that's
definitely not the UNIX philosophy.

~~~
_delirium
> In fact the only purpose of an init system is to start executables in a
> defined order, not more.

A _good_ init system should also do process supervision and integrate with the
OS's resource-management subsystem (on Linux, cgroups). You could do that with
an even bigger pile of shell scripts, of course, and some Linux distros
towards the end of the sysvinit era have been trying to. But at some point a
giant tangle of shell scripts which are full of copy/paste boilerplate and
frequent bugs starts to look like not the best solution.

It's not like systemd is the first system to come to that conclusion, either.
Solaris dropped sysvinit 10 years ago, and OSX dropped its BSD-ish init in the
same year. Earlier than that, djb also wrote an init replacement called
daemontools, which was a bit of a step in that direction, though a smaller
one. That got some uptake but was hampered by some of the oddities of being
djbware (e.g. for years it was "license-free software", and it was only
intermittently maintained).

~~~
lawl
Process supervising is for process supervisors. Init's jobs is to be able to
start a process supervisor.

~~~
_delirium
Are there any examples of a process supervisor that isn't also intended to
replace the traditional init system? Even systems less ambitious than
SMF/launchd/systemd, like daemontools and runit, aim to replace sysvinit,
because otherwise you'd have to specify _twice_ , in different ways, how each
service should start and stop (though it's possible to set up configurations
where that works).

~~~
makomk
At the moment I'm using OpenRC which does exactly that on top of sysvinit, as
are most Gentoo users. You don't have to specify everything twice because init
never actually knew how to start and stop services in the first place - with a
traditional sysvinit configuration that was all handled by scripts that it
started, and OpenRC replaces those scripts.

~~~
the_why_of_y
sysvinit does have built-in service management via inittab(5); unfortunately
it is so primitive that in practice it is only used to spawn getty(8).

~~~
JdeBP
> unfortunately it is so primitive that in practice it is only used to spawn
> getty(8).

The AIX System Resource Controller is spawned by a record in /etc/inittab.
(AIX actually comes with utilities for maintaining /etc/inittab such as
mkitab.) So too is svscanboot in daemontools 0.75 and later. Gerrit Pape is
currently having problems with the Debian package for running runit, precisely
because runit has for some years been run from /etc/inittab in some
configurations and in Debian 8 it is suddenly no longer a file whose existence
is always guaranteed because it is part of an "essential" package. There are
some other packages in Debian 7 that have _undocumented_ dependencies from the
existence of an /etc/inittab file, because they too read/write it for their
own purposes. I have yet to verify whether they've been fixed for Debian 8.

Don't think for one second that in the second decade of the 21st century
people only ever use /etc/inittab for getty. (-:

This is one of the quieter on-going problems for Debian 8. It switches one to
using systemd, because that's the Debian 8 default; but there's no upgrade
path or compatibility mechanism for /etc/inittab. Every few months, another "I
was (unknowingly) using /etc/inittab; I upgraded; it broke." person seems to
pop up.

------
digi_owl
The part about Gnome and logind vs consolekit could have been more detailed.

Poettering was sitting as co-maintainer of consolekit for some time. Then as
logind got underway, the maintenance was to pass to Ubuntu under a new project
name. But i can't find any trace of this actually happening.

After that the state of consolekit as a project seems to have been in limbo,
until recently when a XFCE dev forked the existing code to github as
consolekit2.

Apparently said dev would have loved to continue consolekit in place rather
than fork, but the final email list traffic between Poettering and Ubuntu gave
him the impression that Poettering wanted to see consolekit dead.

So it may well be that people didn't maintain the consolekit code in Gnome
because they had the impression that consolekit was dead and buried.

but then Gnome and systemd devs seems to share a "one true way" attitude.

~~~
vezzy-fnord
Oracle actually picked up ConsoleKit and beefed it up significantly for
Solaris, giving it true multi-seat support. [1]

However, they did this in complete silence, so very few people actually know
about this.

[1] [https://hg.java.net/hg/solaris-desktop~spec-
files/file/79574...](https://hg.java.net/hg/solaris-desktop~spec-
files/file/79574a01c676/patches/) (15 patches)

~~~
digi_owl
Things like this makes me wonder if RH changed once Oracle bought Sun and then
forked RHEL into Oracle Linux. Thus Oracle went from being something of a
partner to being a competitor, and RH rolled back on various behaviors
(package patch sets etc).

------
geographomics
There's a lot of complaint about logical fallacies in the comments of this
article. Does anyone know what they're referring to?

Seemed like quite a clear interview to me, and interesting to hear a bit of
the history behind such a fundamentally useful piece of software.

~~~
bhaak
It's probably from readers of this blog post where "logical" and "fallacy" is
quite often used (careful, needs js for viewing):
[http://judecnelson.blogspot.com/2014/09/systemd-biggest-
fall...](http://judecnelson.blogspot.com/2014/09/systemd-biggest-
fallacies.html)

------
bkeroack
"But we believed that an init system should work the other way around, where
you say: this is where I want to go to, and you figure out the rest. Because
of that design, Upstart was very simple, but it put a lot of complexity on
admins and developers, because you actually had to write down all these rules"

...and this is the problem. Pushing this kind of complexity into PID 1 (or
however systemd achieves this) is exactly the wrong solution. The init system
should not be clever, or contain an "engine that can figure out what the
computer is supposed to be doing" as he says later. This really smells of
second system syndrome.

In reality even with a simple init that pushes complexity to the admins, end
users rarely incur the cost. The distribution authors set up init as needed
and end users don't need to worry unless they makes changes. Then you get the
best of both worlds: easy maintainability and a simple, reliable init system.

------
fsniper
I'm not sure if he is really clueless or just trying to flame wars.

~~~
dinergy
i honestly think he believes his stance is the only correct one. you can see
it in the way he answered the questions.

the bit where he picks apart upstart, gentoo, etc., are nothing more than
callous, myopic opinions yet he answers them with cheerful conviction.

technical arguments can be made for or against any piece of code. but, he
dismisses other projects with a wave of his hand firmly stating that systemd
is the "only" correct evolution.

i'm am neutral on systemd. do i think the old init system(s) are the right
way? not really. do i think systemd is the right way? not really. would i be
fine with systemd in the future? perhaps. would i be fine with another bit of
code? perhaps.

but to dismiss the concerns of a large number of community members, to neglect
or downright deny bugs and issues, to shoehorn a project into _everything_
while closing your ears to everyone else, these are my issues with the
project.

if it works, great. but at least open your eyes and ears and work with
everyone.

i migrated my home kit to freebsd from debian because i was having errors.
failed boots. crashes. config problems. and i didn't have a decade of old
crusty init scripts to begin with.

what used to be rock solid on my hardware became annoying to deal with.

i'm all for change and evolution in software/systems and i don't personally
have a problem with poettering or what he is trying to do, but he's doing it
in a piss poor way.

~~~
XorNot
I don't get how an opinion like this:

"...to make it clear, I think Upstart actually has its benefits. The source
code is very, very nice, and it’s very simple, but I think it’s too simple. It
doesn’t have this engine that can figure out what the computer is supposed to
be doing."

is callous and myopic?

~~~
digi_owl
I find myself wondering why the init is supposed to figure out what the
computer is to be doing. That kind of "automagical" behavior is what drove me
bonkers when trying to use Windows for more than a games console...

~~~
pachydermic
I dunno. If it's able to do what you want automatically that's amazing. If I
can just plug the printer in and have it work that's amazing. If I can turn
the wireless on my laptop on and it works that's amazing. And so on.

The problems arise when it _doesn 't_ automatically do what you want it to do
and you don't have a good way of fixing it yourself. I'm not in a position to
understand init systems so I will be in this camp if there's ever _any_
problem with _any_ init system. What matters to me, as the end user, is how
likely it is to work automatically.

If systemd 'just works', then what's the problem? If systemd is broken...
well, that's a problem, isn't it. But from what I've seen a lot (though not
all, of course) of the arguments against it are highly political/philosophical
and aren't based on whether it's actually working or not.

~~~
vezzy-fnord
Hotplug isn't the job of systemd to begin with, though. It's the job of the
device manager, which on Linux is usually udev (but alternatives like eudev,
mdev, smdev and vdev exist) - udev being a part of the systemd repository and
to an extent coupled with it (which will become complete when kdbus is merged
into mainline Linux), however it's still a rather distinct piece of software
that maintains the hardware database and listens to kernel uevents.

So if your init system is handling hotplug events, then that's some bad
design. systemd doesn't stoop that low, though it still has plenty of mistakes
- like putting the INI parser in PID1. As an example, even Apple put launchd's
XML parser as a separate process!

~~~
digi_owl
There is tight interaction between udev and systemd. Heck, udev was folded
into the systemd code (you find it now as a sub-dir of the systemd source tree
over at freedesktop.org) because they didn't want code duplication...

This after having udev exist as a independent project for a decade.

These days the only way to download and install udev manually is to download
the whole of systemd and then extract udev from that. Something that caused
Gentoo to fork udev into eudev.

------
wtbob
> So we slowly started doing stuff that all the other Linux distros did, and
> implemented that in simple C code that was fast and parallelised.

That's part of my issue with systemd: replacing safe (-r) shell scripts with
C. Sure, it's fast, but I trust C about as far as I can throw it (and yes, I
realise that the Linux kernel is written in C: it constantly has local root
exploits, which proves my point). I really wish that projects like systemd
were written in something of like Ocaml, Go, Rust, Common Lisp—anything fast
enough but safer.

Also, it probably doesn't help Lennart that he looks like the Rolfe Gruber in
the Sound of Music…

~~~
mmastrac
> That's part of my issue with systemd: replacing safe (-r) shell scripts with
> C.

Because shell scripts are always safe, right? [1][2][3] We used to get root on
Android devices by exploiting poorly-written shell scripts.

> Also, it probably doesn't help Lennart that he looks like the Rolfe Gruber
> in the Sound of Music…

Personal attacks won't help your case.

[1] [https://github.com/ValveSoftware/steam-for-
linux/issues/3671](https://github.com/ValveSoftware/steam-for-
linux/issues/3671)

[2] [https://github.com/MrMEEE/bumblebee-Old-and-
abbandoned/commi...](https://github.com/MrMEEE/bumblebee-Old-and-
abbandoned/commit/a047be85247755cdbe0acce6f1dafc8beb84f2ac)

[3] [http://www.exploit-db.com/papers/13199/](http://www.exploit-
db.com/papers/13199/)

~~~
pdkl95
While bugs happen in any language, shell scripts are _far_ easier to debug
than C. You tend to not get nasty interactions spanning multiple source files
in shell, where the bugs tend to on a single line (like in that Valve example,
where rm should have never been used in such a careless/negligent manner). On
the other hand, are you sure you've checked for _all_ of the potential
undefined behavior problems in your C programs? These are far more numerous[1]
than many people realize.

[1]
[https://www.securecoding.cert.org/confluence/display/seccode...](https://www.securecoding.cert.org/confluence/display/seccode/CC.+Undefined+Behavior)

~~~
digi_owl
That is, imo, the beauty of shell script.

At its core, they are the very same commands you would give manually. but
wrapped in basic programming logic (if clauses etc).

thus you can walk the shell script and get a good feel for why the script is
doing what it is doing.

If you want to do that with a binary, you pretty much have to either hit it
with a debugger (i think a previous thread here on Hacker News actually
recommended using strace to inspect systemd boot behavior) or have the source
code sit beside it for reference.

