Hacker News new | past | comments | ask | show | jobs | submit login
Why Did ArchLinux Embrace Systemd? (2016) (reddit.com)
105 points by kbumsik on Oct 21, 2019 | hide | past | favorite | 126 comments



Looking back, I still don't get all the hatred that SystemD got (and sometimes is still getting). A part from some nice-to-have missing niche features (I'm looking at you, retries management with one-shots), it Just Works™ for the vast majority of use cases, just like PulseAudio does. The fact is that the minority which indeed have problems is very vocal because those problems come from complex corner-cases that only a a very tech-savvy user could generate in the first place.


Until it doesn't.

Until very recently having wrong permission on one of the networkd config files would result in networkd ignoring the rest of the perfectly readable network configuration files too, cutting off all network connectivity. (Systemd also requested the permission change iself.)

Currently there's a bug where if you have a bridge and all interfaces are disconnected from the bridge (I use hot-pluggable USB network interfaces) when you re-connect them, networkd will keep the bridge off. If you restart systemd-networkd it will fix itself.

I can name at least two instances already where I lost networking in my entire SBC farm, and had to pop out all microSD cards and fix the breakage manually, just because of systemd-networkd bugs. And I don't do anything crazy, just a single ethernet interface and two overlay wireguard networks, and mostly default config.

And fixing it yourself is not easy. I can do C coding perfectly fine, but I run 3 different CPU archs, so fixing it myself means compiling (3 times) and distributing my own systemd version to all my machines until upstream fixes the bug. It's not pleasant, nor easy.

I certainly understand people who miss having an ability to more readily script the system startup/network config.


To be fair, this seems like an illustration of GP's point:

> complex corner-cases that only a a very tech-savvy user could generate in the first place

I agree that an init system should work on complex corner-cases, but systemd remains popular because it works for most people most of the time.

Tangent: have you found an init system that you like the most? I've only used four or five init systems and don't have any super strong opinions -- maybe because I'm not running and SBC farm. :~)


Having a bridge or wireguard interface and following suggestions in system logs is not really a complex corner case. At least I don't think so.

I like systemd and use it everywhere (I use Arch Linux / Arch Linux ARM mostly). I just don't think it needs to be painted all pink as "Just Works[tm]" software.


I've also seen it either not bring up the firewall (load iptables rules) or suddenly decide to take it down (remove the rules).

I'm not sure why it did that, as I didn't have time to dig into the why while trying to deal with the result.


There was recently a bug in nftables, that caused some configurations to not load at all. But I'm not aware of anything iptables related. I don't think systemd handles iptables at all internally, it's just another service.


I'd like something like systemd, but with all high-level internals written in a embedded scripting language.


Might want to look into GNU Shepherd. The whole thing is written in Scheme.


It seems to have a very limited scope of just managing services. I sort of like that systemd is more tightly integrated with Linux and for example listens on netlink for network changes, or device hotplug events, or on input events for (power) button events, and allows to reconfigure the system based on changes dynamically, etc.


Maybe someone should extend shepherd.


init.d was nice, using shell scripts. readable, editable, clean, extendable. I liked that


Just not maintainable, which is why distros switched away.


Before using SystemD for the first time, I also wondered if there is any grain of truth in what its opponents said about SystemD.

However, after trying SystemD for one week, I became convinced that the designers of SystemD are incompetent, so they cannot be trusted with a component of such importance for a computer.

I am normally a Gentoo user, but a few years ago I wanted to install Linux in a hurry on a small computer. With a slow CPU installing Gentoo can take several hours, unless you have a previously prepared image, so I decided to try ArchLinux.

The installation was indeed fast and without problems. However, SystemD behaved in an unexpected way. SystemD is advertised as booting quickly. It indeed booted quickly, but no faster than my optimized Gentoo systems.

On the other hand the shutdown with SystemD was very slow. This was a huge surprise, because never before and never after have I encountered any computer where the shutdown is annoyingly slow.

Moreover, sometimes the shutdown FAILED, which is also something that I have never encountered on any computer not using SystemD, and I have used thousands of computers, with many kinds of operating systems, including at least eight or nine UNIX flavors other than Linux.

The failure of shutdown was apparently due to some kind of race condition when some process was killed faster than some SystemD component expected and that SystemD component was still trying to communicate via DBUS with the defunct process and it blocked because its messages no longer reached the destination.

I suppose that this bug might have been corrected eventually, but this is a design error that I consider unforgivable, i.e. to conceive a shutdown sequence which depends on successful inter-process communication.

After seeing such a mistake I have lost any confidence in the abilities of the SystemD developers, so I have never tried it again.


The more you have to deal with systemd, the more you hate it. I wrote maybe two dozens of unit files at this point and can't take seriously any praise of systemd anymore, I immediately assume such people don't actually do anything with it beyond starting and stopping services.


I don't mind systemd and I've administered a lot of stuff with it at this point, well beyond just start/stopping services -- I've written plenty of unit files, written drop-in unit file overrides, written timer units, written auto-mount units, looked at startup timing profiles, debugged issues using journalctl, etc.

If I had to guess, it's probably because I've never ran into any of the weird systemd problems that I see people report sometimes. For me, it's always pretty much just worked, and more often than not it natively has the functionality that I want.

My largest complaint is that I have to bounce across several manpages to write a comprehensive unit file.


For the long shutdowns, you probably want to make the `DefaultTimeoutStopSec` config shorter, so that systemd doesn't wait as long for services to stop before sending them a SIGKILL.


Do you count signals as inter-process communication? I'd argue they are, and the init system definitely needs to cleanly shut down the system (using signals, at least). What degree of inter-process communication is acceptable, then?


Not OP, but I would consider SIGTERM as IPC, but SIGKILL as not really IPC (since the target can't respond to it).

Traditional RC does a SIGTERM to everything, and then waits a bit and then does a SIGKILL to everything. This means that if the SIGTERM failed to be handled properly, progress is still made.


systemd does that too. The problem is that the default wait time is insanely long (I think it is something like 5 minutes). My biggest complaint with systemd is the same as my biggest complaint with tmux: many of the defualt configurations are very strange.


Yeah, default configuration of systemd-logind is to prevent network access. This means if you use e.g. NIS authentication weird things happen.

Weird as in 3 minutes after someone SSHd into my workstation, X11 restarted. Until I tracked it down to a default systemd configuration I thought I was going insane.

Beyond the default configuration, which is somewhat defensible as NIS is probably not a typical case, it really bothers me that if someone else's login process has issues, everybody's logins get killed.


systemd did solve lots of problems, particularly for workstations, but it has done its work by tearing down and replacing plenty of components that didn’t need replacing, reinventing solutions that already existed, and eliminating pragmatic options for troubleshooting often out of some blindly overapplying some principles that don’t make sense in every context. And unfortunately its architecture is such that you can’t just pick the good parts. You have to take it all at once to have any chance of system coherency. That alone is evidence of poor design.


You mentioned no concrete examples, without them, your comment is just your opinion, not really adding to the conversation.


> just like PulseAudio does.

I think part of it actually stems from people that had problems early on with PulseAudio, and have developed a vendetta against Poettering. Whether that vendetta is justified or not, I don't know.

Personally systemd hasn't been a pain point. It's been far less of a headache than everyone at the office thought it would be.


Interestingly; Pulseaudio was a shit show until the distro maintainers got a good hold of it, now it mostly just works without much fiddling (or, any, in basic cases).

SystemD is much the same way, I'm coloured by my impressions from Fedora21-22, where it.. didn't work great.

The issue is with poetterware, it's _incredibly_ difficult to discover /why/ it's not working properly, which is dangerous for an init system (or, an "everything between your application and kernel"-system), a failure rate of 0.0001% is still an unfathomably enormous number of systems; and due to how it works it lends itself to an incredible host of threading issues.


I’ve found it to be an improvement. Writing pre-systemd init scripts was never at all enjoyable. Systemd units aren’t exactly works of art, but they’re much, much better.


Poettering used alloca() as a poor man's garbage collector. Proper manual memory management is just "too hard". Other lingering code quality issues surely remain when the primary author is taking these sort of shortcuts and not considering how to write robust C.


As long as the allocation size is bounded (due to alloca/VLAs lacking any mechanism to signal allocation failure), what's the problem with that?


The problem is that 1) you can't assume what the available stack space is and 2) you don't let tainted user input determine allocation sizes which is what Systemd did in the CVE disclosed earlier this year.


Nobody can write robust C. The correct answer is Poettering should've switched to Rust. With few exceptions, any code base that has not yet done so should be considered brittle and bug-prone.


The problem is the Java generaton that never learned how to do it.


Somehow members of the JavaScript and Ruby generations pick up Rust rather easily, while those who pride themselves on having learned to write robust C (or C++) are flummoxed by it.

Hypothesis: C is so pathologically bad that it breaks brains.


I think C requires more of people. Granted I am no C expert but having worked in the industry a couple decades I have made conclusions based on observations.

As a sysadmin I wrote a small kernel module to trigger a panic on a certain condition hoping to capture some info in a dump several years ago.

Today I cant find sysadmins who can use a CLI properly.

The goalposts have moved inwards IMO

Not sure its bad thing either but it makes for interesting comparisons


It's kind of like back in the day Emacs was considered a programmer's tool. Yet secretaries not only used it, they learned how to configure it to their liking in Lisp! (If you tell a normie they are "customizing" the editor rather than "programming" it, they won't even realize it's hard!)

These days, people calling themselves professional developers with years of experience won't even touch Emacs.


Plenty of people still use Emacs.

I picked it up several months ago after using Jetbrains IDE's for quite a long time, and honestly haven't felt the need for any of what Jetbrains brought to the table.

Mostly thanks to LSP.

If you're not a Java developer, Emacs is perfectly fine as a configurable extensible editor. With the additional caveat that its lack of multithreading can get in the way sometimes.


I use Emacs, but at my last few jobs I was just about the only one.


Hypothesis: Rust is a religion. The only times I've seen this amount of proselytization have been Rust and church.


Java or any other garbage collected language is memory safe by design.


> it Just Works™

Avoiding that kind of attitude is why I moved to Linux in the first place. Nothing ever "Just Works", most systems eventually break down, especially if you're doing anything non-popular. You can't just ignore things when they break, having an easy way to fix things is one of the most important parts of complex software. Saying "It Just Works" indicates to me that the developers don't care if the system is opaque and hard to work with, which has been my experience with systemd.


> it Just Works™ for the vast majority of use cases, just like PulseAudio does.

I am not in the vast majority of use cases, then, because systemd is a serious pain in my butt. But then, so is PulseAudio (which is why I make sure it's never installed).

My issues with systemd (both technical and as an andicator of the direction Linux is going) are severe enough that I'm actively preparing to move all of my machines over to BSD.


What issues do you run into?


My intention is to express my dissatisfaction with systemd, not to get technical support, so I'm not going to list them here. Nothing good can come from doing so. But I will say that in terms of actual pain points, the biggest ones have been around logging.

However, technical headaches aren't my largest objection -- my largest objection is that its getting increasingly obvious that as we move into the future and more applications rely on systemd-specific facilities, it's going to get more and more difficult to operate a Linux system without it.


> I'm actively preparing to move all of my machines over to BSD

You remind me of all those people who are going to move to Canada "any day now" because of Trump.


Thankfully, changing Linux distributions or switching to a BSD is significantly easier than the significant legal hurdle of immigrating to a country that doesn't have a reputation for being an easy country to immigrate to.


I think ultimately a lot of the systemd hate is just fear of change and not wanting to give up an init system they are famiiliar with.


For me it was the scope creep. Systemd is turning "init" into a framework of all the vulnerable and bad ideas that Microsoft moved away from after Win95/XP. binfmt mount, intercepting and changing the behavior of DNS, polkit implementation, changing user authentication, so much more than I could put here. There are many desktop concepts being shimmed into servers that are simply not appropriate for a lean mean web machine.

The good bits that systemd brought into the mainstream such as cgroups and lxc do not require systemd. They can be implemented independently. The watchdog timers and unit files were not a new concept either and did not require systemd.


> Systemd is turning "init" into a framework of ...

This is conflating the systemd project and they systemd init system. Most of the things you mention are seperate systems (systemd-resolved,logind, polkit, etc.) that are part of the systemd project and interoperate with the systemd init system, but are not part of the init system itself.


Agreed and the conflation is somewhat intentional. When you install a system, you get all of that. Custom kickstart files can strip away some of it. For the longest time, you would get polkit regardless. The rpm spec file was changed about a year or two ago so that it can be removed if someone did not exclude it specifically. To be fair, each OS maintainer is including or excluding bits of systemd in the default configurations, so that is not entirely systemd's responsibility. Every year systemd appears to be subsuming more and more system functionality.


Is it correct to even say that systemd "brought" cgroups and lxc? They seem like fairly independent projects.


systemd did bring them mainstream, but yea cgroups were around for years before systemd was a thing. But they were mostly independently managed by some small daemons that did things based off rules and other things like that mostly. Systemd changed how they got used, primarily because multiple management systems working with them tends to cause issues, but using them for low impact isolation and tracking of daemons automatically really turned them from something you'd independently setup to something that brings a lot of utility for just about every use-case everywhere.

You can still use cgroups independently with systemd, but it's recommended that you use a service and the delegate option to have systemd create a cgroup hierarchy and also know that it shouldn't mess with it after creating it. This lets you have your cake and eat it too if you have a case that systemd doesn't handle (I'm working on a setup with this myself for a coding sandbox setup).


I think Docker had a bigger influence on cgroup use than systemd.


I disagree because everybody who uses Ubuntu is using systemd, but only a fraction of them use docker.


Ubuntu uses snaps heavily bringing cgroups to play.


I agree with you and edited my wording.


Sigh, this argument just has to come up in every discussion on Systemd. Anyone who dare to be critical about Systemd is apparently just a Luddite afraid of change. You know what, I have no problems with change at all, only when the change is to something worse.


There doesn't seem to be any single, general, perfect answer.

I use systemd across distros, and am now confused when I run into an old SysV system.

When I'm sufficiently advanced to bump into one of these corner cases, it will be a Good Problem To Have.

The market penetration of systemd is about as good an empirical validation as can be had. How soon until it's codified in Posix?


Do you have concrete evidence that systemd is in fact worse from a technical perspective?

I'll readily admit that systemd isn't perfect, but my overall experience is that it is better in many ways to its predecessors, and did solve real problems.

And when I referred to "systemd hate" I was not referring to healthy criticism, but to those who take very chance to claim that systemd never should have been invented and their init system of choice is clearly superior. Rather than, say trying to make systemd better, or make their init system fix the problems that systemd was designed to remedy.


Something bothers me when they talk about fear of change. Some changes are consensual, some are mostly agreed upon organically and then you have cases like systemd who fueled lengthy debates. I don't like to ignore this as "fear of change".


If we're not afraid of change, Let's change away from systemd.


There was a switch away from SysV init for particular reasons, not change for the sake of change.


You know, everyone says this but every server I worked on (several thousands) which used sysvinit I cant remember any issues with startups that couldn't easily be scripted around.

I still dont understand what issues are fixed with systemd and I am aware of many it has created.


I am not convinced every sysadmin could ‘easily’ script around init issues on the servers they manage, but I’ll give you that.

The real issue isn’t with servers, it’s with desktop systems and laptops. When users plug in a USB sound card or plug their laptop into a dock, they don’t want to (and likely couldn’t) “easily script around” that; they expect an audio device, network connection, video card to be created without any user intervention, and go away when devices get unplugged.

Making a set of init scripts (and surrounding unload and reload scripts) robust against insertion and removal of any of many different devices isn’t easy even for init script gurus, and would lead to duplicated code for detecting and handling the various load/unload conditions.

That, I think is the primary reason Apple went with launchd. Systemd took that idea. There’s nothing wrong with that.

The main issues with systemd, IMO are a) initially it was buggy, and b) tremendous scope creep.


>The real issue isn’t with servers, it’s with desktop systems and laptops. When users plug in a USB sound card or plug their laptop into a dock, they don’t want to (and likely couldn’t) “easily script around” that; they expect an audio device, network connection, video card to be created without any user intervention, and go away when devices get unplugged.

Bingo !

Folks who know, and I mean really understand how to run servers know how to script init.

Folks who use Linux on the laptop / Admin Ubuntu do not and systemd makes them able to do stuff "most of the time"

There is no reason to force systemd as the default for servers. You dont plug things into servers and in fact you dont want some service monitoring servers so someone can stick something in a server and start rooting around it.


Unfortunately linux often STILL doesn't handle plugging and unplugging devices very well and it kind of seems like this has gotten worse not better with systemd.

Also you seem to be conflating dealing with devices with initialization of the system. Dealing with such devices has insofar as I'm aware always been handled by c code in and out of the kernel whereas at one time initialization of services and orchestration of components was handled by a scripting language.

This is to say that a bash script may have brought up different components and services but that c code that makes up the component is in charge of actually managing said device.

Its obviously feasible for both c (or any other language) code and orchestration of start up and shut down of services to be buggy but its not equally easy for end users to fix their own problems in all cases.

Ideally bugs get reported to those who are most capable and the underlying code is corrected however if said labor isn't paid to work on their project hours to do so may be in short supply and it may frequently fall on users to lend a hand.

This is a numbers game. If you have 10000 users and 10 of them are qualified to solve such problems you may be lucky to get the help of 1 to submit patches or post a solution to a problem on a forums for others to use before the upstream problem is fixed.

If 100 are qualified you are 10x as likely to find help to solve a problem. I have on many occasions come across a solution months or years before an official upstream solution came into being. You are suggesting that most sysadmins might find it challenging to solve their own problems. They don't always have to. Merely having comprehensible user friendly systems make it more likely that one of their fellows can help them do so and more likely that such solutions travel both laterally between users and hopefully upstream solving the problem for all.


Desktops and laptops are a tiny and insignificant minority of Linux installations, and almost completely related to why Linux is popular and important.


To what?


RunIt, say.


I suspect the OP is referencing Void Linux[1] which makes use of runit as its init system. If not, it's the first thing that came to mind.

I'm not sure it's necessarily a better solution as it requires more manual interaction out of the box (namely creating symlinks by hand among other duties), but that might be argued as a feature. There appears to be a wrapper or two out there to automate away some of the tedium.

[1] https://voidlinux.org/


s9


What should someone google in order to find out more about this? (i tried, i got lots of information about the samsung phone, even with -samsung and various incantations of 'init', 'processes', 'linux', etc)


I think the OP meant s6 and not s9.

runit, s6 and also nosh are all daemontool-style init systems, which have a somewhat different idea how to do things than systemd all, but all are inspired by the original daemontools package.

s6 and nosh have decent overviews:

https://skarnet.org/software/s6/ https://jdebp.eu/Softwares/nosh/guide/introduction.html

The original daemontools: https://cr.yp.to/daemontools/faq/create.html#why

I'll let others decide whether these are better or worse than systemd. I go with whatever my distro would give me, since picking a different init system is too much work...


The hatred has little to do with functionality.

Say you live in a city, and you have multiple options of transportation: walk, bike, scooter, tuk-tuk, bus. None of them are the most ideal, but you have variety, choices. They're all simple enough for anyone to learn how to use them quickly, and you can even hop from one to the other. But you think, I'd kind of like to be able to move around faster.

The city also isn't running perfectly. There are part-time job shortages, traffic jams, the education system is haphazardly put together. The city wants a way to solve its problems too.

Now say a consulting company comes in and convinces the city they should abandon the other options and adopt a new transit system. This system is fully automated, faster, cheaper. Not only that, but it also solves the job crisis, eliminates traffic jams, organizes the education system. Nobody who uses the transportation system will notice that they are using new transportation, because they're going to put out cardboard cutouts of the old transportation systems, so in theory, everyone should just be able to use it as-is. So the mayor, having heard the pitch, signs a contract: The company owns the city's transportation for life.

Only it's not as perfect as the consulting company claimed. Those cardboard transportation shims will move you around, but you can't choose your seat, you can't talk to anyone else, you have to use pre-approved payment methods and pre-approved routes. Sometimes your route will break down, and there's nobody on board to tell you what to do. Sometimes you get to the wrong destination, but you can't walk home anymore, so you have to just sit there until you figure it out. Sometimes it never shows up. You think, I just want to get around town; why is this so complicated?

You have no choice or control. You didn't vote for this, and you can't vote to change it. You could move to another city, but that is way too involved for most people, so they will hope for the best, deal with whatever pains there are, and learn how to navigate this new, more complex world, because somebody gave a good sales pitch to the mayor. And you think to yourself, "Didn't I move to this city to have more choices?"


The problem is when "It just works" doesnt, it's an undebuggable, un-understandable, documentationless kluge with binary logs.

I had it silently take over my DNS . I even have the 'official' way check-boxed to not override DNS settings in Network Manager. Does so anyway. I even removed the symlink from /etc/resolv.conf and made it root:root . SystemD changed it next reboot.


Documentionless? https://www.freedesktop.org/software/systemd/man/ and you have it locally and offline too when you need it. Like in the 80s/90s unix way.

And it's way more documented that your average init script sourcing distro-specific bash/sh libraries where you have to dig into and read the code just to understand how that thing works


I like systemd, but agree that binary logs are a horrible idea.


Binary logs seem great on macOS, why is this bad for Linux?


Because Linux and MacOS targets different roles.


Could you elaborate more? My use case is very similar between the two, and I don't mean workstation operation.


I'm thinking that Linux is primarily a server OS and MacOS is primarily a workstation. And yes people do use Linux on the desktop and visa versa.


> I'm thinking that Linux is primarily a server OS and MacOS is primarily a workstation.

Good for you.

> And yes people do use Linux on the desktop

Good for them.

But the fact remains that this post still ignores why binary logs are bad for the server use case. They seem great for me in that role.


the initial goals was laudable though, is there a way to have text logs and verifiability


> The problem is when "It just works" doesnt, it's an undebuggable, un-understandable, documentationless kluge with binary logs.

Init scripts were undebuggable, un-understanable, documentationless kluge with text logs, and systemd has a perfectly functional format converter. I'm not seeing the regression here.


2 things;

1) Init scripts varied by implementation very much, I have managed to successfully debug, make, read and otherwise modify/parse init scripts from basically every _major_ distro including OpenBSD and FreeBSD.

I am not particularly intelligent, so if I can do it, you can.

2) There is no reason to assume we go back to bash scripts per-se, there is a huge middle ground between a huge C program which execve()'s programs and watches them live, parses their output, logs it, enables socket activation, keeps an in memory DAG that it determines randomly on boot.... and bash scripts.


Re 2... there is a lot of middle ground there, but can that middle ground tackle all of the problems systemd is trying to solve without resorting to what they've resorted to?


From the original problem statement[0]? Yeah I think so, Runit solves all of those issues.

[0]: http://0pointer.de/blog/projects/systemd.html


Can we please retire "Just Works" - pretty much every time I have seen this phrase on HN there's been a response describing a "well it doesn't work for me, I am seeing <symptom>". "Just works" is as useful as the "works for me" cliche


> it Just Works™ for the vast majority of use cases, just like PulseAudio does.

So does any init system or process supervisor system that has more than trivially tiny installation base.

The entire point of developing a new one is to reduce the number of the minority cases when it does not "Just Works(tm)"

Look at monit. It "Just works(tm)" for 99.9% of the use cases but boy its single threaded, everything is sequentially controlled with multi-second granularity is infuriatingly bad if you are one of the unlucky souls that wants monit-like functionality where every group runs independently from the other groups.


I've been using Linux on the desktop since 2003. Linux audio support started out as a shit show that you could deal with tolerably well by buying supported sound cards. Pulseaudio seemed to simplify things until it didn't actually work. Until it inexplicably stopped working at intervals until load was such that it ended up out of sync by just a little bit etc. It simultaneously had a degree of complexity and configurability that suggested that maybe just maybe if you fiddled with a knob it would actually work until it didn't again.

If it just solidly didn't work at all it would have been annoying but this sort of kind of but never really consistently almost working is what makes you want to physically throw your computer through the nearest window.

Now it seems to work acceptably well save for the fact that guis still regularly have garbage interfaces and one weird stupid configuration.

Pulse by default remembers which device a particular app was outputting sound so you can switch the default device to headphones while say your browser isn't actually outputting sound via any tab and then you will open a sound producing tab and it "remembers" it was using the speakers and blares at full volume at 2AM. Thanks pulse.

This is configurable but who is going to figure that out?

I believe the relevant line is thus where the relevant part is restore_device=false

    load-module module-stream-restore restore_device=false
in /etc/pulse/default.pa

Regarding sound configuration virtually nobody wants different programs to output to different sound producing devices. This might be an important use case for some users but if you have 5 sound producing apps the process of switching from speakers to headphones should not be to move each one with 3 clicks per move. I almost exclusively perform 3 operations I turn the sound up or down and switch to and from headphones whereupon I expect everything including newly created streams to end up on the new device. The first 2 have dedicated buttons the third does not so I had to make it so.

Absent a dedicated button fewer steps would be nice last I looked a few years ago it was complicated in all environments I tried. Having the right answer to get a pleasant environment be write a shell script and fiddle with config files is probably not optimal.


I'm still convinced a part of it was trolling. Just straight-up "I don't know about this in any detail and I don't care about it, I just want to start shit" trolling, which took whatever legitimate criticisms there are and wrapped them in a thick shell of bad attitude and loudmouthed idiocy for the pure sake of it. Trolling doesn't mean someone's lying, after all; it just means someone's loud and obnoxious; look up Frankfort's definition of "bullshit" for more.


I wish this kind of opinion would've been spread more widely within the Arch universe (wiki / IRC) so outsiders can understand what we actually gained from switching to systemd.

It still feels like many people hate systemd but the reasons that are presented all sound totally valid from a programmers' perspective.


These reasons are valid; I think the main reason so many people hate systemd is how all-encompasing it has become.

Prior to systemd, the lack of a standard init/rc/service monitoring system meant that you could pick and choose 3 different solutions for those 3 problems. systemd is pushing for unifying not just those 3, but other systems too (gummiboot, consolekit, udev). As more software becomes designed to work just with the systemd ecosystem, more people who have love for one of the systems that has been subsumed will feel forced to switch.

Say you have a favorite fork. You eat lunch with it everyday. Now at your favorite restaurant for takeout, the manager says "We switched salad dressings, and the new dressing line isn't compatible with that fork, you'll need to use this one instead." The fork that the manager hands you may be perfectly fine (better in some ways, worse in others than your favorite fork), but it's not your fork. So you sadly stop going to that restaurant. But then you notice something: more and more restaurants are switching salad dressing suppliers, so now you have to either give up your favorite fork, or use only salads you make yourself. You're probably going to be mad at the salad dressing company.


Yeah, it's two sides talking past each other, with very different conceptions of what Linux should be. The systemd detractors tend to view Linux as sort of a grab bag where you can make a nigh infinite number of operating systems, all loosely compatible with each other. The systemd people view that as way too much stuff to test and support.


I prefer this approach:

>Systemd is included by default but not enabled. You can scan your MX system and discover files bearing "systemd" names, but those simply provide a compatibility hook/entrypoint when needed.

>MX Linux uses systemd-shim, which emulates the systemd functions that are required to run the helpers without actually using the init service. This means that SvsVinit remains the default init yet MX Linux can use Debian packages that have systemd dependencies such as CUPS. This approach also allows the user to retain the ability to choose his/her preferred init.

Source: https://mxlinux.org/about-us/


> MX Linux is a cooperative venture between the antiX and former MEPIS communities

Ah! SimplyMEPIS (a live version of MEPIS) was the first gnu/linux system I booted on my (dad's) P3 desktop computer in circa 2003-2004!


> What most systemd critics consider "bloat", I consider necessary complexity to solve a complex problem generically.

As a distro and philosophy, Archlinux was bound to adopt systemd eventually. e.g. Maintaining you're own init system/scripts is not KISS. Creating a custom init file to adapt 90% of software from systemd to another init system is not KISS.


Yeah. Most people misunderstand Arch's KISS principle.

If you look for most decisions Arch took in the last years, most of them were focused in simplifying the distro and reducing the maintainer burden. So things like removing the TUI installer (does someone remember it?) and migrating to systemd. Even the fact that Arch ships with most packages with few or no changes compared to upstream is focused on it: reduce the burder on maintainer.


Gentoo, Alpine and Void still default to using their own init systems.


It seems Alpine uses Gentoo's OpenRC init system from a quick Google.

https://wiki.alpinelinux.org/wiki/Alpine_Linux_Init_System

https://wiki.gentoo.org/wiki/OpenRC


Alpine has to use OpenRC rather than systemd because it's designed to run in containers. Using cgroups or lxc requires running your container with elevated privileges. This doesn't only affect systemd, it also makes it impossible for me to run Nix builds in unprivileged containers with isolation enabled, since isolation requires cgroups.


From the thread, 2brainz, the person maintaining the init scripts, responding to why systemd:

>> I haven't heard a good explanation of why systemd and not a different, new init system

> There is none. Systemd came along, we thought it was great, some of us were already involved with systemd and pushed the change. We could have done many different things, but we didn't.

> Runit was suggested by users at some point, and I think there is/was a runit implementation in AUR. However, runit was never considered in detail.

> First, we don't know if the other systems were really alternatives (at least I don't know). The answer is boring: Systemd solved many problems, it was there, it worked and we already used many of its tools in our initscripts at the time. There is no specific reason why we did not use $WHATEVER over systemd.

> About launchd: it was never considered. As I said elsewhere, systemd seemed to do what we want and we went with it.


After listening to the BSD Canada talk on systemd and BSD, I agree Linux does benefit from a system layer. But I still hate the implementation, and you can't easily swap out stuff.

I really like logrotate and syslog and not needing journalctl for stdout/err logs from services. It's one of the things that makes working with Docker containers so much nicer too, because you have sane commands to get logs for services; way easier than remember all the weird switches for journalctl.

I like NetworkManager (this still hasn't been replaced in systemd, right?)

I still use OpenRC and Runit on Gentoo/Void respectively. OpenRC does dependency management but runit not so much. When it comes to building out VMs and stuff, I see the use of systemd for mount targets, but like I said earlier, I really hate the implementation in target files. It's so hard to make drop-in replacements for systemd that can run the same target files because of just how far it goes into the system.

For VMs at this point, I honestly prefer minimal startup and running everything in Docker. Alpine or RancherOS seem to fit this bill nicely. If FreeBSD had a current Docker API implementation, it would be perfect.

For my desktop, I prefer OpenRC or runit. It's just easier to do things and I'm not fighting with the system layer like I am in Windows. Then again I use i3 and prefer to have more control over my hardware, so that's just my use case I suppose.


> After listening to the BSD Canada talk on systemd and BSD

For someone who don't know, here is the video "The tragedy of systemd". This is a really good talk.

https://youtu.be/6AeWu1fZ7bY


All respect to the author of that talk.. but "just get over it" and "embrace change" is not compelling to those who have very searing issues with systemd.

I do not consider this to be a good talk because it paints dissenters as "not embracing change" because.. they are stupid?

It also makes the argument that the only thing that exists outside of systemd is sysvinit.

He also seeks to make Lennart a martyr because of death threats. (which, btw, seriously it shouldn't even be said that that is unacceptable, some people are fucked up).

Oh, and systemd definitely didn't invent cgroups, an engineer at google did and they don't even use systemd to run GCP compute hypervisors.

Benno so thoroughly misrepresents those that have issues with systemd that I can't help but feel it was intentional.


I didn't get that from his talk. I think it was more of a, "BSD should have a system layer. How can we make one that's way better?" .. that doesn't do all the things people hate systemd for doing.


oooooo, the grandparents video is very different in tone than the version I watched, which is the same talk: https://www.youtube.com/watch?v=o_AIw9bGogo

I'm sorry, I had assumed I had seen the exact video, but I had just seen the talk from another venue.


I don't particularly care for Systemd, just out of principle, but I don't mind using it. Personally, I feel Arch Linux went downhill after Judd left. Maybe I'm naive, but I absolutely loved Arch's 'rc.conf', which was basically an entire system config in one file. That going away, and the adoption of Systemd, really made it feel like a different distro. Not better or worse, mind you, just different.


Take a look at NixOS, the entire OS is declarativily defined and can be stored in git.


Meanwhile http://without-systemd.org/ (linked from the post) throws database errors. Maybe they forgot to 'enable' their mysql systemd service?


I don't think so. It has been a while since I've done PHP but that reads like an inclue-gone-missing (likely with MySQL connection configuration information) causing the database to fail rather than MySQL breaking the include.


I think very few people criticise systemd for its init system, which is pretty decent as far as init systems go. The main complaint is everything else systemd does. It's trying to swallow the universe and there seems to be no limits to where its "scope" will end.


I would like to know what Arch Linux and the other distros thought of alternatives like runit, openrc, and BSD-style init.

I can at least sympathize with the arguments for systemd if the only other choice was SysV init scripts. As Richard Gooch once said, "These boot scripts, in the tradition of SysV, can do anything. They are flexible. They are scalable. They can run industrial-strength systems. Unfortunately, they're bloated and ugly. OK, maybe they're not ugly to everyone (not that I've ever heard anyone admit they're elegant, only 'it works'), but they sure are bloated. They are complex and hard to navigate." (http://www.safe-mbox.com/~rgooch/linux/boot-scripts/)

But the choice was never just between SysV init or systemd. Couldn't Linux have done something like FreeBSD's rc? Or I read about the daemontools family of init systems, like runit, which is what Void Linux uses. These systems sound like they solved SysV's problems without adding systemd's. That is, they are more declarative, easier to maintain, can handle complex dependencies, and can run things in parallel. But they remained unixy: they are small, file-oriented, and don't swallow up adjacent software.


This is a very good point. The reason that you have trouble if you just use a bunch of scripts to start up a contemporary Linux is that the underlying system has become quite complex. Now you have to deal with the effects of stuff like udev and dbus. Systemd can be seen as an attempt to paper over the underlying complexity problem with another layer on top. That of course if not likely to work but is depressingly common in the world of OSs. People add things to a simple system until it is no longer a simple system. Eventually it becomes unreliable enough that no one dares extend it further.

As a nice counterexample, OpenBSD doesn't have stuff like udev or dbus usage during boot. The rc.d system they use is literally just a bunch of scripts and is dead simple and reliable.


Ah, systemd. It's one of those things where I'm glad it exists, but I hate it. I'm glad it exists because more open source is good, and diversity of implementations is good. I hate it because it complicates my machine's startup process, and my life, unnecessarily, and puts a lot of shit in pid 1 that shouldn't be there. I don't want it to become a hard dependency of my entire software stack. (Same with PulseAudio, hence why I am not currently a Firefox user.)

I left Arch after the systemd transition and didn't look back, but I wish Arch devs and users well.


As someone who's been slowly picking their way along the narrow crooked path towards free software / tinfoilhattery / unixbeardedness for the past several years, I'm interested to know which browser you use.

I assume given context that Safari and IE are right out. Chromium and variants are probably out? If Firefox and variants are out (not sure if all of them have the pulse dependency, because honestly your comment was the first time I learned about it - probably because I'm using Arch), what's left? Qutebrowser? Konqueror?

I guess implicit in the above is an assumption of certain (non-basic) functionality that many people would consider table stakes in a web browser. Given that choosing to use any software involves making certain compromises (like "if you don't use Chrome, some Google applications will likely not perform as well") I guess my underlying desire is to know which compromises you are willing to make, given that you aren't willing to compromise your desire to keep pulseaudio off your machine in order to be able to use Firefox.

On a scale I'm having trouble naming but that goes from about 1 ("my son bought me this iPad so I can see my grandchildren on Facebook") to 10 ("I wget webpages to my server then email them to my Trisquel laptop") your pulse/firefox constraint seems to me to be solidly in the upper half, which I respect, but which also makes me expect you are willing to accept compromises that I'm not (yet) in choosing a browser. Maybe I'm wrong, though, because there's a lot of browsers on this chart: https://upload.wikimedia.org/wikipedia/commons/thumb/7/74/Ti...


Currently? PaleMoon. It doesn't have a Pulse dependency and it works with every site I throw at it. We'll see if that holds up.


Throwing out pulse after it was pretty much made to work consistently well seems poorly thought out as does running Palemoon.

https://www.howtogeek.com/335712/update-why-you-shouldnt-use...


Qutebrowser is QtWebEngine underneath. (which is Chromium).

So if Chromium is out, then so is qute.


Not necessarily. QtWebEngine strips out a lot of code from Chromium, including anything talking to Google servers.


Ah, I'm going to lose karma here because I'm a sysadmin and not a programmer and programmers seem to really like systemd, and this forum is represented strongly by programmers.

Regardless, every time these types of threads come about I feel like I _have_ to speak because there's a lot of bitterly divided people regarding this issue. (or, rather the people who are bitterly opposed and the people who don't care, think systemd is a net good in the vast majority of cases or those who think the backlash is very much unwarranted).

I'd like to say first off: SystemD solves real problems that nothing before it was solving, maybe it doesn't solve those issues in the way I personally like, but it definitely does and was one of the first to do so- so for this reason the original maintainers should be warranted respect, it's also a given that it's not the developers fault that this is so widely adopted or even forced. I see them as problem solvers first and foremost so any gripes I have with systemd as a thing which is foisted upon me is definitely not on them.

However, I do consider that systemd as a new -layer- between user land and kernel space (called "system space" by Poettering) necessitates a lot of breaking changes. I do not consider this by itself as a bad thing, but having a poorly defined interface for these things does not enable systemd to be replaced without some form of "systemd-esque" emulation in future.

Thus while we have improved from where we were, we do not have fertile ground to improve in the future.

Additionally; SystemD as a "modular" system is, in my opinion, a myth; even if you take for granted the fact that systemd+journald are the only core dependencies, you then quickly start to drag in more and more rapidly (udev and systemd-resolved being famous examples).

Some design paradigms are scary, for instance socket activation leaves PID1 listening for traffic on the network, by default bound to all interfaces. (a good example is to look at cockpit) this could be a very large footgun if there exists any form of exploit in systemd.. and since systemd is not written in a memory safe language, and the overwhelming majority of security bugs are memory safety bugs... this is mildly concerning.

There are other concerns regarding opacity of the dependency system, the fact that no-two-boots are the same and thus some race conditions are likely to never be solved by me. (a lowly sysadmin type who only knows enough C to make linked lists). The design very much reminds me of Microsoft Windows in many ways, but perhaps that was a good design to choose from?

FWIW: SystemD is amazing on my laptop, I am really happy with it. But for my servers I am very unhappy with it, but the major distributions chose to use systemd and I'm a sysadmin so I'm beholden to what the company considers stable and developers.

I very much begrudge the fact that I'm not allowed to have a dissenting opinion on this.

I also hate that when people talk about systemd vs "anything", invariably people assume I am talking about openRC or sysvinit, but there are much better init systems available today, such as Runit.

SystemD is not a panacea, it should leave fertile ground for replacement. If that cannot be done it should not be adopted because that is a scary amount of lock-in.


I'm mixed on this... as a developer I just want something that works, has decent interfaces and abstracts away so I can care a lot less about it. SystemD does most of this for me.

Also, echoed in another comment, I'd just assume use Docker and/or K8s and/or dokku for most of the services I write anyway. Frankly, I care about writing the applications and services that solve problems for me, I don't want to spend as much time on system administration or CI/CD infrastructure as I already to.

In the end, it does a better job at most of the problem space in most ways compared to the alternatives. I have to google for it any time I actually interact with it anyway, which is what I did with older initd systems, so I'm not really that enthusiastic one way or the other.


> as a developer I just want something that works, has decent interfaces and abstracts away so I can care a lot less about it.

As a sysadmin, I want something to be debuggable because I'm the one who gets paged when the failure rates increase.


On to microservices with you and give the developer the pager!

Embrace the change :)


I always say, "Give me a platform that works, or the power to fix it, I don't care which." I'd prefer something that works I don't have to take care of... but when under the gun and IT has other priorities, it sucks.


I mean.. I'm still on the hook for Kubernetes nodes being available..


Well-said. Hopefully this gets upvoted further.

I'm a sysadmin and not a programmer

SystemD is amazing on my laptop, I am really happy with it. But for my servers I am very unhappy with it

I think this is what it comes down to. The end-user experience is pretty good, just like the end-user experience for MacOS and even Windows nowadays is pretty good. But that also means you could replace Systemd with a similar system built on similar concepts, and the end-user experience would still be pretty good.


I agree with an event driven supervisor daemon architecture, and that shell scripts are inefficient.

But writing everything in C in non-scriptable way may be a bit of over-reaction to the old ways of having everything scripted in the least efficient way possible.

I also write my init systems in C (not for generic Linux distro, mostly for embedded HW/mobile use), but I'm planning to make it so that I have a program in C that offers some primitives and the core logic will be implemented using an embedded JS engine, so that it's scriptable and fixable without re-compilation.


(2016)




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: