In particular, if Gnome is starting a ton of background processes that stick around and cause problems, then the fix should be to change Gnome, not to change the default behaviour and break everyone else.
To use the OS sandbox example, this is the reason why OS X added application sandboxing and forced Mac App Store apps to use it, and not anyone else. Systemd's approach is like Apple saying "We're going to start sandboxing everyone's apps, App Store or no, unless you make these changes to your apps.
In this case systemd seems to be reinventing process groups, in a totally different way, instead of fixing whatever the reason is why GUI sessions don't use session leaders.
So it's pretty obvious there really is a problem that needs to be fixed, and apparently so far nobody else has made a real or successful attempt to do so.
No, we shouldn't need to rely on processes behaving nicely. But when a program is broken, you fix that program, not change the entire system's semantics.
Interestingly enough, none of the bugs in other software that is being papered over are in closed source, so it's not like the original issues are unaddressable in the most direct way possible.
Processes using daemon(3) have been around for 21 years, and are used to certain behaviour. Some of those processes behave badly.
systemd is adding functionality to kill all of a user's process when the session ends, no matter what. The problem is that they don't have any way of telling which processes are behaving badly and which ones aren't, so they're telling everyone (e.g. tmux, screen, etc) that they have to implement changes or they'll be killed too.
Fundamentally, the problem isn't whether the systemd behaviour is right, the problem is that it's a huge breaking change and they're asking everyone else to work around the shortcomings of their approach.
> shell managers like screen and tmux had hacks to make them survive sessions ends in the past
tmux, screen, etc. didn't have hacks to survive sessions; they just did, automatically, because of how things worked. systemd is asking the to add hacks to make them survive session ends from now on, when running specifically on Linux under systemd 230+, because systemd is going to change default behaviour for other people's processes.
They actually added the functionality over 5 years ago. Which is when the tmux project was approached to accept a patch to support pam (not systemd) in order to not be killed.
All they did recently was try to change the default configuration to enable the feature by default.
> Shouldn't this code be part of glibc instead of tmux? -- Nicholas Marriott, 2011
> If you want to change how processes daemonize, why don't you change how daemon() is implemented [...] ? -- Nicholas Marriott, 2016
Systemd developer asks libc to add systemd specific code
This whole comment thread is about an optional feature that has existed for 5 years and only recently had its default value changed. You are not affected unless you are running a bleeding edge distribution that has already upgraded to a version of systemd released 5 days ago.
I don't think a more long term solution will present itself in the near future and most likely what will happen is that the default will get changed back to disabled.
nohup is not a hack
What systemd is offering has nothing to do with nohup, it's not "a flavour" of nohup it's a completely different thing
Systemd is not correctly killing processes, this was NEVER DONE LIKE THIS, they decided this out of a whim because apparently Gnome can't do the right thing (how surprising)
I'm not looking forward to what comes next when Gnome breaks through the new system.
As a sysadmin for over 20 years, systemd offers nothing significantly new or different in the areas of process management than any of the other methods to do so, most of which are less intrusive.
A single tool providing a building block for an isolated, specific project says nothing about the general applicability or desirablity of that tool to the wider ecosystem.
People bitch about PulseAudio still and it was what made the fucking sound plug and play instead of massive mess
PulseAudio is still a massive mess. It solved a tiny subset of surface level problems, eventually, and introduced a whole crap load of other ones in the process.
PulseAudio has been around, what, 10 years now, give or take? I think I've only stopped having to fight with it in the last couple of years. When it first landed it was by no means an improvement from OSS, and it took a while until it could realistically compete. It brought a bunch of advantages with it, but it was far from easy.
If you want to see a fun example of how annoying PulseAudio can still be, try and make a linux machine act as a bluetooth audio receiver. It takes only a couple of minutes work installing and configuring the bluetooth side of things. The PulseAudio side of things will suck up hours of your time trying to persuade it to be consistent, running through a godawful series of inconsistent command lines.
It's pretty clear that systemd is determined to unfuck the linux desktop experience. They're not solving problems that exist on the server side. When you look at the justifications for various bits of really breaking stuff, it's almost always (as in this case) coming down to something related to the desktop experience. They've futzed about with stuff, breaking things along the way, because it "speeds up boot." How often do you reboot your servers, and does it really matter that it's 20 seconds quicker?
The goal is admirable (The linux desktop can be a crappy experience, I've been using it as my primary work environment for pushing on 10 years now, and using it in general for closer to 15.) The methodology is not, nor is the attitude of the developers. They continue to approach problems from the perspective of "we know best", "not invented here" and "The only fix is a ground up re-write". Along the way they're making the exact same mistakes existing stable and mature software made decades ago. Worse, they're breaking things that really shouldn't be broken and they're betraying a complete lack of understanding about how linux is run in production environments. This particular case is a perfect example. They've decided that processes should be reaped on all systems, in all environments, when users log off. Not because this was a particular problem anywhere, but because some processes weren't being cleanly stopped when someone logged out of Gnome. It's fixing a minor desktop issue, something that doesn't affect the large majority of the user base, in a fashion that breaks what a large majority of the people actually do.
Here's another classic example: http://www.spinics.net/lists/netdev/msg365477.html systemd developers decided that the way IPv6 route advertisement was processed in the kernel was wrong (despite having been fully functional and stable for a long, long time), and decided that they really should do it themselves in an incorrect fashion.
At its core, the *nix environments core strength has always been that it's composable, focusing on complect over complex behaviour, creating a cohesive whole out of individual and specialised components that have specific tasks. You take a similar when writing software applications. Just as with writing software and using libraries, you use the ones that meet your requirements the best.
This flexibility allows people to build platforms and infrastructure that does what they need, and allows people to solve issues developers couldn't have anticipated in the first place.
Systemd's approach is instead a highly opinionated and inflexible "this is the way things will be". It's likely a perfect approach for a desktop environment, but that's not where it's being restricted to. The primary consuming environment is servers where they're 'fixing' things that weren't broken there in the first place.
To them systemd is golden, as you can see with how CoreOS is basically built around systemd.
I agree; as a sysadmin, whatever init system the OS offers is very very useful in my job.
What I have not found is that systemd is more useful than the other major alternatives. Can you outline, let's say, three ways where you find systemd solves pain points you had with upstart or sysvinit?
Look at it this way - the primary use for tmux is having one's SSH client running all the time. Personal servers vs corporate ones. Barely anyone working on system software for Linux has anything to gain by supporting personal servers.
Its with a recent dbus update implementing yet another session concept (leaving it with no less than two, as best i can tell).
So you graphically log into a distro with a Gnome desktop.
This result in 6 different "sessions" being started.
In the kernel, that is largely transparent.
In X, again largely transparent.
In dbus, twice!
Seems to me like yet another systemd two minute hate. It's getting quite tiresome to see.
They then decided that, rather than encapsulate fixes for their breaking change in portable APIs like daemon(3) and PAM to allow user space to keep working as intended, it'd be easier to just outsource the labour onto the community by demanding every user space program that was working just fine do the work of adding a couple of hundred lines of systemd specific code directly, encapsulation be damned.
That's an insult to the tmux developers in particular, and disdainful of Open Source devs in general. Don't show up at my door telling me to do a bunch of work for you to fix your shit. You broke the contract; encapsulate the fix so I don't have to care.
Systemd is trying to build a new feature that is useful - reaping processes after logout should help improve security and performance. Think about Chrome having flags to "run background processes after app is closed".
Systemd is trying to maintain backwards compatible behavior by collaborating with developers. We should not do these things cost "that's the way they've always been" is killing innovation.
> This is no more an insult than removing cgroup write access from processes or any other new kernel feature.
False. Empoyees of a 2 billion dollar corporation deciding to demand free labour from the community to accommodate what they've decided by fiat is the new Right Way is incredibly insulting. The tmux developers do not exist to serve your whims. The community is not here to save you money.
> Systemd is trying to build a new feature that is useful - reaping processes after logout should help improve security and performance.
There's NOTHING new or innovative about this! Processes are already reaped at logout, unless they're nohup'd. Changing the state of affairs to "processes are reaped at logout, unless they use dbus to ask systemd to not reap them" is just a bunch of mechanism churn to get the same result
> Think about Chrome having flags to "run background processes after app is closed".
What would stop Chrome from doing that under this new regime? Literally the only difference is that they'd be accomplishing that via systemd-specific code rather than portable, SUS-compliant code.
Ultimately all this change is accomplishing is deprecating simple, portable interfaces for achieving nohup behaviour with baroque, dbus-laden, systemd-specific ones. Red Hat's goal here seems to be little more than increasing systemd's surface area; otherwise, why not simply hide the sausage-making systemd cruft behind the portable API?
> Systemd is trying to maintain backwards compatible behavior by collaborating with developers.
"Collaboration" is not a synonym for "I have altered the user space contract. Pray I do not alter it further".
> We should not do these things cost "that's the way they've always been" is killing innovation.
"Innovation" is not a synonym for "instead of a daemon(3) call, here's 150 lines of dbus crud to get right back to where you were before".
The difference is that pre-systemd, there is no safe way to guarantee that all processes are reaped on logout if some of the processes are written maliciously to try and avoid being reaped. The new cgroup-based system makes it impossible to evade being reaped by playing games with fork. This mechanism is valuable, but it is possibly not the best default for most systems.
And this is a red herring. If the process is malicious, it can just as well use this systemd method to keep from being killed and continue to do its dastardly deeds in the background. Logically, this "feature" can not be about controlling purposely malicious programs, because there's no way to determine if a process is malicious or not just by asking it or by having it assert that it is not. The only reason this exists is to paper over bugs in either the way these daemons are spawned (ssh-agent and pgp-agent, for example, work best when they appear in the execution inheritance hierarchy, rather than as separately spawned programs that run as siblings) or in the way they detect that a session has ended.
You can't guarantee reaping while allowing for opt-out semantics for things like tmux. Everyone who thinks that's some innovation being achieved here is misunderstanding it.
This is originally about some GNOME daemons sticking around when they shouldn't. GNOME should just fix that, of course, but it points to the hole of it being all-or-nothing when it comes to nohup'ing. There's no good mechanism for asking for more limited lifetimes.
Of course, the Right way to solve that would be to leave the existing API alone for the all/infinite lifetime case, float a proposal for a new API for the limited case, raise consensus, implement it, and encourage devs like GNOME to move to the new limited case.
But that might mean actually talking to the community and building an encapsulated API that could be reimplemented on other platforms. So much easier to break everything and demand the world keep up, with the added benefit of all those raw dbus calls and hard-coded end points making portability off of your 2 billion dollar cash-cow OS just a bit incrementally harder, again.
Isn't the usual answer to this, "If you don't like it, fork it?"
If you work on a large FOSS project, you will spend a lot of your time reacting to decisions made by other people. You won't agree with all of them. Some of those decisions will be made by $2B companies, some of them will be made by influential individuals like Linus and Lennart. Still others were made 20 years ago by some anonymous person in their basement.
Either way, if your complaints fall on deaf ears, your options are limited to living with it or forking it. That's just how it works.
Zbigniew Jędrzejewski-Szmek, systemd developer, is asking for tmux to change to incorporate some systemd-specific Desktop Bus code. Nicholas Marriott, tmux developer, replies that this should go in the GNU libc daemon() function for everyone whose programs do this to benefit, repeating what he said 5 years ago.
Following your model, which should Zbigniew Jędrzejewski-Szmek be forking? tmux? Or GNU libc?
It does seem like what is being requested is not a sensible change, but arguing with insults and getting so personal seems like poking a rather large bear with a very short stick because it is trying to make you move away from it even though you were there first. You can do that, but it's unlikely you will enjoy the outcome.
I think you should check what "backwards compatibility" means, because it's the total opposite of what you're describing
five years ago:
tmux dev asks:
> If it has to be done differently then - glibc has the daemon() function, and tmux uses it on Linux. Shouldn't this code be part of glibc instead of tmux?
Which is a perfectly sensible question. After all, what value is there in the systemd team being so resistant to maintaining user space contracts? What technical gain is there in pushing low-level systemd dependencies high up into user space? Wouldn't sane engineering be to encapsulate this kind of low-level platform detail behind libc interfaces like daemon(3), which exist precisely to abstract away this platform-specific knowledge? If the daemon(3) infinite lifetime semantics aren't desirable in all cases, why not add a second call, say user_daemon(3), to address those cases without breaking everyone else?
Zero response from the systemd devs, and 5 years later they change the default to break everyone without following up on these obvious critiques of their design.
If systemd tried to add user_daemon: "systemd is trying to add systemd specific patches to everyones software!"
They can't win. Haters gonna hate.
This critique makes zero sense. Encapsulating this kind of thing is exactly what libc is for. What daemon(3) has to do to achieve it's semantics varies by platform. daemon(3) abstracts that away. That's the point of it. Ditto everything else in libc.
Nobody would complain about libc doing system-specific things. That's precisely what it's for. That's what it already does to do its job.
> If systemd tried to add user_daemon: "systemd is trying to add systemd specific patches to everyones software!"
Again, you seem to not understand the concept of libc and its abstractions.
THere's nothing Systemd-specific about the idea of a user_daemon(3) call. On systemd-using systems it could be implemented using systemd-specific code, whereas on other platforms it could be implemented via OpenBSD specific code or FreeBSD specific code, etc. It could simply be a pass-through to daemon(3) on platforms that don't care about limiting the lifetime of some but not all daemons.
That's what libc is for. It exists to hide the low-level details of the platform from user-space.
Circulating a proposal to extend libc with a new call is exactly what you'd expect from a good citizen in the Open Source community. From a user-space developer's perspective, abstracting away hundreds of lines of dbus calls to systemd-specific endpoints is unequivocally The Right Choice.
That Red Hat seeks instead to push the systemd dependencies as high into user space as possible suggests that technical concerns are not their primary motivation, because Big Ball of Mud style of engineering is bad from any perspective.
> They can't win. Haters gonna hate.
It's reasonably clear that you simply don't understand the critiques being presented here.
Unless those system specific things are systemd specific things, then all people do is complain.
A reply to one of my other comments:
> Systemd is not a platform. And not every Linux runs systemd.
> Didn't know linux also means systemd.
You're not wrong, updating daemon WOULD make perfect sense and solve this problem in a single place. People would rather argue and fight than solve real problems though.
> There's nothing Systemd-specific about the idea of a user_daemon(3) call
True, but as soon as a systemd developer proposed adding user_daemon it would immediately become a "systemd" function. As soon as they propose modifying daemon to work with systemd, systemd would be accused of trying to take over libc.
One of the BSDs could modify daemon and no one would bat an eye, apparently OS X did so to play nice with launchd. But "linux is not systemd", so they can't win.
> Unless those system specific things are systemd specific things, then all people do is complain.
According to you. Although, again, I continue to assert that most people understand that libc is the correct place for this.
>A reply to one of my other comments:
>> Systemd is not a platform. And not every Linux runs systemd.
>> Didn't know linux also means systemd.
> You're not wrong, updating daemon WOULD make perfect sense and solve this problem in a single place. People would rather argue and fight than solve real problems though.
Again, libc is not a homogenous library. glibc need not be implemented the same on all linux systems. Implementing daemon(3) semantics could be done either via systemd or not, as makes most sense on a given linux platform. glibc already does this for other things. The criticism that this makes or assumes all of linux == systemd doesn't apply.
Either way, we'll never know, because the systemd devs did not even bother to try being good actors, here. They're ignoring libc entirely in favor of pushing systemd dependencies up the stack.
edit: I want to take this a little bit farther:
The idea that systemd devs have to push a bad engineering decision (to tightly couple low-level platform details) onto upstream code with no abstractions, which they are being heavily criticized for, was somehow necessary to avoid misguided criticism they might have received for trying to do this the right way, is simply a wrong argument that doesn't make sense.
I'm not wildly happy about adding more OS-specific code to tmux for something it already does perfectly well everywhere (including Linux right now).
If it has to be done differently then - glibc has the daemon() function, and tmux uses it on Linux. Shouldn't this code be part of glibc instead of tmux?
There doesn't seem to be any discussion following these additional questions from the tmux side. So what you're really saying is that systemd has been aware of the architectural issues with their design for five years, yet have not bothered to fix their end?
Not just tmux, every opensource project that used established nohup practices. For example, now all of my virtualbox instances are killed when I logout of Gnome on rawhide. This was a feature of virtualbox that’s worked perfectly for years now.
Obviously Red Hat's behaviour in general shows disdain for any devs in any project who had working daemonization code that just got broken.
"Hey, can you implement and maintain new platform-specific code to work around this problem we're going to introduce by breaking behaviour of programs using a 21-year-old API?"
It feels especially egregious because, as commenters pointed out, implementing the PAM API is a better (and undoubtedly more portable) way to do this that the developers of systemd didn't seem to consider. Instead, they jumped straight to "implement a custom fix just for us so our breaking change doesn't break your app too".
They already released systemd 230 with this behavior and the bug reports from users of unstable/rawhide sparked the "public" discussion. (The bug report in the link says this right, but it still comes a week after the release instead of before)
Seriously, every single person who feels compelled to offer their opinion based on what is an emotional response to Lennart's (or anyone's) "overreach/arrogance" is contributing little more than noise to the whole process. It's far too easy for someone who doesn't interact face-to-face with someone else to fall in a trap of assuming the worst about another person when there's a technical disagreement; this is a form of cognitive bias that pollutes development in the corporate world all the time. I'm as guilty of it as anyone, and I've learned to be wary of the temptation to project bad qualities on other people who I disagree with but don't interact with directly.
In the open source world it's so much worse though- not only do almost all developers interact indirectly (i.e. not face-to-face), but there are potentially thousands of people who are basically just casual observers who want to jump in to give their 2 cents. That's great! But it's only great as long as those 2 cents aren't just people bikeshedding, or actively trying to make things personal rather than technical. Unfortunately, more often than not it is bikeshedding, and literally the only thing they have to bikeshed about is to try to make it personal. :(
Instead they should take a step back and rethink their proposed solution every time they get strong feedback or face some resistance from the open source community.
I wouldn't consider that bit to be particularly egregious. The thought process of a systemd developer confronted with a regression caused by systemd changing some well-known behaviour would naturally lead to a solution where program X tells systemd that it needs to be handled differently. It's basically a version of the anecdote that experienced developers aren't better because they know the answer to every question, but rather that they know which question to ask. Apparently the discussion has lead to a better solution (not that I would be a good judge of that), so this seems like a bit of a storm in a teacup.
So now not only have we not achieved that goal, but now we've inserted an additional layer of complexity into the mix.
It's an entire middleware layer that sits between userland and the kernel that we did not have before.
The distinction being?
Edit: I'm out of my HN comment quota but kind of glad I am. Because I'm clearly not going to get anything productive from a thread where I wonder aloud why systemd is classified differently and where that distinction lies. When does an init system become middleware "that we didn't have before?" I can think of a few answers but was curious what others think. Fuck me, right?
Instead, ask a question, get snarkily called obtuse and downvoted because we have the misfortune of systemd being the topic of conversation. History shows us you might as well not even participate; if all the people who argued about systemd invested that time into productive activities we'd have warp drive by now. Which would be useful to flee systemd threads.
I swear, systemd is tech's abortion debate.
For years people have been locking their keys in their car and then having a locksmith open the door when they get back. Now, someone is trying to improve the locks so that a locksmith cant open the door and people are complaining that it ruins a perfectly good arrangement.
In this analogy, 'locking the keys in the car' is 'logging out' and the locksmith is 'nohup'.
systemd isn't really out to break nohup, they are trying to fix the logging out process. Sure, nohup has worked for decades, but 'logout' has never worked right.
The problem is that 'nohup' is designed to 'run a command immune to hangups' not 'run a command even after I logout' and there has never been a distinction between an individual connection 'hangup' and a full user 'logout'. With systemd, there is, and programs don't handle the distinction.
If you don't think there should be a distinction KillUserProcesses can be turned off.
This is an example of one of the concerns people had -- and were told to stop spreading FUD -- about systemd in that it seriously overstepped the bounds and desires of an init system.
Killing user processes after logout (when they have been set to not do so, through nohup usually) is not sensible.
In other words, if this functionality had been added 21 years ago as well, a lot of mess and confusion would have been avoided and we'd all be happier, but since it didn't the systemd guys are trying to implement it retroactively, which is going to break a lot more stuff than tmux/screen/etc.
Only if you can succesfully determine which ones the user wants to keep running. Otherwise it's just breaking stuff, as this story shows.
if this functionality had been added 21 years ago as well, a lot of mess and confusion would have been avoided
Not really. You will always have sub-standard software that doesn't adhere to the system's declared semantics, and just goes by "what works" (ask the WineHQ guys). Any API that has been around for 21 years will have corner cases where software just breaks. That does not mean you're then free to break the old API yourself.
edit: If you're already using a specific command to tell the application to stay alive (e.g. nohup), you can use the "new" systemd command as well. nohup never related to a "session" scope because that didn't really exist before systemd. A new command _does_ break historical behaviour, but thanks god we do that from time to time. That's how innovation works.
How is it so? I cannot see a single legitimate justification for such a stupid behaviour.
> "session" scope
Exactly. There is no such a thing, and it should not exist, it's a totally alien concept to anything Unix.
(By the way, I disagree with this request to change tmux)
Not after "logout", but after terminal is detached. For whatever reason. There is no such a thing as "logout".
When I log off (by giving the logout command) the shell will print ‘logout’ and
execute commands from the file ‘.logout’ if it exists in my home directory. After
that the shell will terminate and UNIX will log me off the system.
The concept is commonly called logout, even if you prefer to call it something else.
If you had two logged-in terminals on your desk and ran 'logout' on one of them, are you still logged in?
Dbus devs apparently asked themselves the same, added the concept of a user session to dbus alongside the existing login session.
Gnome code happens to get the two confused (or dbus fails to properly tell one form the other, likely by mixing old and new behavior).
Systemd dev notices leftover Gnome processes, and convince Poettering to flip the safety on the big gun.
All this while giving crap all about unix conventions...
What you are describing is systemd becoming a kernel sitting on top of a kernel, second guessing the actual kernel.
... with nohup. Which now doesn't work.
(Added:) The first runtime, in systemd parlance (I think), would be the session scope, the second runtime would be the user service and the third... well, something different, as you'd pass on things (group memberships, environment variables, namespaces) from your shell/session into a long-lasting service. One probably would want to extend the "once the last of my session has ended" state to the runtime of the "run forever" command.
Most non-technical users, I guess, would really prefer one of the first two lifetimes to be the default. As being able to start from a clean state after logging in- and out is rather convenient.
As nohup, in an interactive shell, is used for the third "keep running forever" option, I think it would be reasonable for distributions to alias it to the relevant "systemd-run..." command for all default shells. Then user-expectations would be perfectly met (and scripts will still run the binary /usr/bin/nohup).
> and no, unix signals are not a sane API
It might not be modern, but it's stable, widely used, and incredibly well documented. Hell, systemd is implementing this behavior using signals.
> Breaking with historical behavior is called progress.
It's not historical behavior when applications are today being developed that rely on said behavior.
Progress is building upon the best practices of the past, not breaking things because a bunch of very low quality, uneducated "developers" who all came from an alien culture simply could not think of a better, idiomatic solution.
This is the level of discourse I've come to expect from any discussion related to systemd
I think the problem people have is that systemd developers seem to actually believe this axiom.
Seeing such an emotional reaction so long after the war makes me wonder if systemd can survive in the long run, though.
Seems to be an issue of "The north remembers."
Requesting the specific desired feature/fix is almost always preferable.
I actually let slip an f-bomb reading that. It shows a terrible understanding of longstanding unix philosophy, (and also straw-mans the argument).
Happily, it doesn't seem that was from a systemd developer as far as I can tell from their website. Still, it's a reminder that the userbase (and developer base) that engage with Unix/Linux/GNU is varied, and may not have grown up with the same sensibilities that people my age have about where the right line between user, system and sysadmin should lie.
This is a very relevant remark and highlights the root of the entire discussion, I believe. There seems to be an ideological fault line here. And I think, it is between those who wish for some ideal, all encompassing design and those who have been burned by the short sightedness of such ideals. Grand engineering ideals cemented into closely knit structures are really great on paper. But some years down the line, you realize you have made too many commitments, optimizing for something not relevant anymore. And now you have to circumvent all those greatly engineered structures. It's a bit like looking at satellite cities, the visions behind them, and what has become of them.
The Unix philosophy is mostly about avoiding those pitfalls. But I guess, understanding this might only come with experience.
In practice though, to me that would mean something that while looking pretty and being better automated would come with many fewer hidden parts and many fewer interactions between parts than sysvinit. That is, I'm looking for simplicity, composability, editability.
The implied technical debt of one system to rule them all is a lot over 20 years, and you will inevitably have to get into line with a dominant system as a developer downstream from it.
At any rate, happily this behaviour was publicised enough that I won't spend a day googling it when it happens to me. :)
Thanks. I'm kind of in the same boat, age wise. I have the impression (and this comes from a number of comments online and a podcast with Poettering I listened to lately) that any criticism from the experienced crowd is seen as a sort of elitism of people entrenched in their abilities they honed over many years. This might be the case for many critics, and Poettering may be right to dismiss them. But I'm afraid that the very valid criticism regarding overall design goals is lost in all of this. The systemd team steams on, believing that all naysayers are just stuck in their ways.
This means that if the DE or X develops a fault, you can always fall back to the CLI to get it back up.
But with systemd, the CLI has become subservient to the DE.
I really like my CLI though, and really really want to be able to change anything about my system from vi if need be, so I guess I fall on the other side of that perspective.
Android got in the door by Google convincing the OEMs to switch Window Mobile for Android, with Google carrying the brunt of the development cost for free.
This happened just as MS was running around in a panic and managed to split their mobile platform in two with Phone 7.
In contrast MS bent over backwards regarding netbooks. They had started shipping Vista, and yet they gave XP another reprieve on execution when Asus started shipping their Linux powered EEEPCs.
Only with major cockups regarding Windows powered warships, and having a former general as chairman of the board, has Red Hat been able to get on the DoDs good side.
They can decide to die when they lose the terminal on SIGHUP (or on any other event), be nicely asked to stop with SIGTERM, and surely can be forced to die with SIGKILL. Everything is there for decades, sane and simple - so I don't really get the idea.
I mean, if some process hadn't considered dying on HUP (i.e. on session close) while it should - it's a bug in that program, go ask its author to fix it.
In fact, the question is should the shell and init environment overrun the (at least 30 year) unix convention that a user's background processes are not killed when they log out?
The reasons given for a 'yes' answer to this question had better be well and truly good, from my perspective. It's not nearly sufficient to say 'some users complain that things are running when they log out'.
If you read the original bug report, this all started because some userspace programs aren't very well behaved, and the typical windows or os x behaviour would be to log out, ensuring the OS kills them, then try them again.
Generally the advice from unix land would be:
1) Don't use those programs
2) Learn to use ps / top and how to kill a naughty process
3) logging in and out is meaningless when it comes to things running as daemons or in the background.
It's pretty shocking to me that someone thinks the best solution is to obviate all that advice. Not that users won't get an experience they expect from other OSes -- they will. But many users will lose a simple experience they rely on, even if they never touch the GUI. That has a bad architecture smell to me, and I think some other solution could be found.
justin@t420:/tmp$ git clone https://github.com/tmux/tmux.git
justin@t420:/tmp$ cd tmux
justin@t420:/tmp/tmux$ wc -l osdep*
Systemd is not a platform. And not every Linux runs systemd.
That would be before 1979. nohup was in 7th Edition. I haven't checked 6th.
Ironically, by pointing to the fact that one has to go back through 37 years of documentation, education, user experience, and scripts you are making other people's point for them. Especially by drawing attention to one of the major flaws in the "nohup was new once" argument, which is that in 1979 (or before) there wasn't a prior 37 years built up to contradict. (-:
has different branches for various unix versions
Seems the earliest reference I could find in there was dated november 1973, but it is from a full import in 77:
Author: Bill Joy <wnj@ucbvax.Berkeley.EDU>
Date: Wed Nov 23 12:43:50 1977 -0800
Start development on BSD 1
Create reference copy of all prior development files
trap "" 1 15
if test -t 2>&1 ; then
echo "Sending output to 'nohup.out'"
exec nice -5 $* >>nohup.out 2>&1
exec nice -5 $* 2>&1
Author: Ken Thompson <firstname.lastname@example.org>
Date: Tue Nov 26 18:13:21 1974 -0500
Research V5 development
Work on file usr/source/s2/nohup.c
Co-Authored-By: Dennis Ritchie <email@example.com>
Author: Ken Thompson <firstname.lastname@example.org>
Date: Wed Jan 10 15:25:04 1979 -0500
Research V7 development
Work on file bin/nohup
Co-Authored-By: Dennis Ritchie <email@example.com>
This is due to a larger discussion that most users don't expect lingering process after logout. https://github.com/systemd/systemd/issues/3382
In these situations, screen/tmux must include changes that enforce this behavior. Today the operating system itself does not reap processes ... but systemd is not doing anything wrong. By adding this change, tmux/screen users will still get the behavior they expect and the rest of the world get a better behaving operating system.
It's breaking historical behaviour
However making tmux try to use systemd-run is a much better solution than making it depend on libsystemd
You might want to see all of the people at https://news.ycombinator.com/item?id=11782364 who are expecting nohup to effect exactly that, with reference to the Single Unix Specification no less.
Who should ever care about the opinion of the non-contributing, useless users coming from the irrelevant platforms?
(Btw, I'm ambivalent on systemd itself)
This can be done at the distro package level. Packagers would edit /etc/systemd/logind.conf and set KillUserProcesses=no to disable it. It's a single line change if people don't like the new default. The behavior itself (killing user processes when the session ends) is quite sensible, and should have been the default over a decade ago, but people got used to terrible defaults and are now complaining.
This is handled at the packager/admin level, but they merged a patch two days ago (https://github.com/systemd/systemd/pull/3373) to fix the policy to let users manage it themselves with loginctl enable-linger. I think it missed the v230 merge window, but could easily be backported individually or at the distro level until the next release.
So, not only can the behavioral decision be managed entirely at the distro level, that decision can be punted down to the individual user if they don't want to explicitly mark the processes they want to continue running.
In any case, wasn't killing user processes already the norm except for the ones which called daemon() or equivalent? And won't the new system kill user processes except for the ones who call StartTransientUnit? I'm having some trouble understanding the improvement, except for the short-term gain until the broken processes start calling the new API.
Whatever you think of it, systemd is doing the "ground-up rewrite" correctly. Normally, you get 90% of the features for 10% of the code in a rewrite, but then the last 10% of the features are the "plugging this thing into the ecosystem of other crufty things" features which bloat the codebase into something exactly as bad as the original.
The only successful way to do a rewrite is to scope your rewrite not to the size of one component, but rather to the size of the whole component ecosystem, such that you get to redefine all the crufty boundaries requiring convoluted code into clean and simple ones. This means you do have to rewrite more things, but each one can stay at that 10%-for-90% size, because none of the bad-API bloat has to come along.
systemd is consuming components of the Linux userland and leaving cleaner boundaries in its wake. The sheer scope of that effort can make the project look bloated—but people forget to compare the codebase with the combined codebases of all the projects it took to have equivalent functionality before.
People also forget to compare the functionality of the entire ecosystem of software—not just the big stuff everyone uses, but the niche stuff—before and after. Linux's original bazaar approach to system software meant that everything worked with whatever components it explicitly planned to work with, and nothing else. Do you know how many codebases contained some random subset of support for logging via STDOUT, logfiles (with or without rotation), syslog, rsyslog? That's what you get with a bazaar approach. systemd's "hub-and-spoke" cathedral approach means that everything just has to work with systemd, and then systemd can ensure that everything works with everything else.
> I really dislike the idea of having one project handling everything on a box
Which is crazy talk because that is one of the selling points to something like openbsd.
If linux tried to add pledge you'd have people coming out of the woodwork claiming that linux was breaking UNIX and requiring "linux specific" patches to every project.
Second, and much different from what SystemD just did, pledge does not affect any program that doesn't use pledge. From an external caller point of view, each changed user land program should act in the exact way that it did before the change.
A programmer can safely ignore pledge and nothing has changed for them. A user can safely ignore pledge as it shouldn't change their usage.
If Linux (I would guess more appropriately the maintainers of the GNU user land) tried this under those conditions, it would still be an internalized change that shouldn't affect anyone outside the project except in the positive as an exploit mitigator.
The bottom line is: OpenBSD is not requiring changes in other developer's programs or in user usage to continue to run in the same way they have run in the past.
Now SELinux isn't perfect by any meaning, but the key difference is that applications can be SELinux aware if they choose, but you can still have full protection without the application knowing that it exists.
I would much rather try to improve SELinux than try to make pledge work for precisely this reason.
Developer knows better what program should and should not do and is much more capable to set this kind of rules with pledge.
And that concept in turn means that a general can put a trusted system into hostile territory and trust it to not leak secrets.
And NSA still care for it. Around the time of the last kdbus dustup, someone from NSA voice a worry about how kdbus would compromise certain parts of SELinux.
Many of these tools have to run in other *nix. Tmux came out of the OpenBSD community. Why would they necessarily want to put in effort to make sure your house is built correctly?
Instead of moving everything to a new platform-specific init system like freebsd's rc system, I instead moved everything to runit. Runit can run ontop of an existing init system, so now I can have this code running ontop of bsd, upstart and systemd based systems without having to change a thing. It's really be a savior for me. I don't touch anything, I write scripts that are about the simplest thing I've ever seen and it just runs anywhere I toss it. I used to avoid writing init scripts like this like the plague but runit has made it quite enjoyable. I've even moved a lot of dev stuff where I had everything running in tmux to runit based control.
I've spent 20 years with this platform and seriously worry about it being as useful the next 20.
With systemd it's not so simple.
iface wlan0 inet dhcp
It actually works rather well but it doesn't have every feature that NetworkManager does.
For non-wifi use cases you probably don't need anything at all (maybe dhcdcd).
* connecting to different wireless networks that use wpa enterprise authentication (which I think some of the bsds only recently added support for)
* bringing up usb networking interfaces via cell phone tethering on demand
* managing multiple different vpn clients and profiles from a single interface
Or do you mean things like
* Bringing up a networking interface with a static ip address
Because I use network manager for the former on laptops, and remove it on servers that only need the latter.
As with many things, yes, it took OpenBSD and FreeBSD longer to get that functionality. The tradeoff is that they delivered a well thought out, consistent, and well-engineered solution, rather than a poorly thought out mess of CADT-driven bloat.
More importantly, I think that some of the systemd maintainers have a track record of not really knowing what they're doing, as exemplified by this tmux issue. I have serious worry about running their process as PID 1.
Other than that, totally positive effect: It deepened my interest in OS security and hastened my switch to OpenBSD which was probably happening anyway.
"Stability" in the sense that OpenBSD provides though is something that I valued more over time of running Arch and my experiences with other things on the side (mainly QNX).
The one that really annoyed me (and is still present) is when it replaced cryptsetup init scripts but did not implement support for root file-system key-files.
For release-upgrades this will result in broken, unbootable, systems - it isn't difficult to imagine how this causes swearing and bad feeling!
For new installs it means loss of required functionality, and lost time and productivity exploring and implementing work-arounds to counter the regression and return to the status quo.
The reason for the lack of key-file support in systemd-cryptsetup? The purist idealism that key-file support patches will only be accepted if they use the Linux kernel's key-ring API (rather than replicate the existing cryptsetup init script behaviour).
Canonical does not spend so much on employing core developers and feeds mostly on upstream.
That in itself is not a problem...but your presumption is wrong. If you track the systemd inclusion process in Debian, you will find that it was not subverted but was a free and open fight. Systemd won on its perceived superiority by people much smarter than me.
The only thing the TC decided is that daemons must be able to run under systemd -- which can be achieved by providing a (cross-init) sysv script, or a systemd service file.
While RH and Oracle has a long history of cooperation, Oracle now have their own distro thats a fork of RHEL.
This fork prompted RH to end some policies that made life easy for CentOS (who now happens to be sponsored by RH), regarding the packaging of various patches.
They are not tying together originally independent projects so that others has to conform or else. They build their own thing to their own needs, and if you find you have the same needs then your free to help out.
This behavior can be disabled by the distro or admin with a single line change, and there's a policy for users to be able to set the behavior for themselves with a single line. They fixed a problem with that policy two days ago, but the fixed policy could be backported for those who need it immediately before the next release. This is in addition to being able to control it on a per-process level. Obviously they gave us so many ways to control this because they wanted to break things.
"Breaking everything" is nothing but hyperbole. This change won't make it to the users unless the package maintainers just let it through. And I can guarantee after this stink that every systemd packager will be informed of the decision.
Yes, asking tmux to request this behavior by dbus was amazingly short-sighted, bordering on stupid. I don't think anyone really wants to argue that. Plenty of developers get caught up in the importance of their project and fail to recognize that not everyone else wants to use it. So he's easily earned some criticism.
I suspect it was well intentioned; he probably heard people complaining about how this change affected tmux, and wanted to help people "resolve it". He obviously didn't understand that this is not the way tmux would want to handle it.
> Redhat is leveraging its strong position ... to bend all Linux distributions to its way
> guided by a desire to keep Canonical penned-in
This reactionary stuff is completely out of hand. We have people:
* claiming that the default should always be to let everything run forever because that's the way it's always been isn't much better. Since the moment Linux supported multiple users, letting processes continue after logout was a bad default.
* insisting that any compiled code beyond the absolute minimum in the boot process is a bad idea.
* Insisting on bad faith when developers don't consider their singular workflow.
* Stating that standards don't matter if only one system implements them. Despite the fact that most standards start with 0-1 implementations.
* engaging in constant tinfoil hat conspiracy claims against Gnome, Redhat, and any other group of contributors larger than 2 people. Obviously they're trying to control us and not, you know, trying to make things work the way they want to work. The fact that these corporations fund major portions of the ecosystem and contribute heavily to all levels of the stack doesn't matter anything - they're obviously trying to get a "monopoly" on linux through open source code. Or working with the NSA. Yes, someone's actually said that.
It beyond ridiculous to think that there's anything guiding these decisions other than the perception of what makes a good product to those involved. They might be wrong, but they're not malicious.
It's disgusting that these systemd threads always spiral out of control like this, stirring up a disturbing level of vitriol. We have constant mudslinging over things that can be fixed with default configurations at the distro level. The fact that we still have these periodic flamewars and monthly hatefests about any groups or projects (even systemd) makes working in the FOSS community an undesirable minefield. We should be embarrassed as adults that we lack the self-control to deal with this in an adult fashion.
The old state of affairs is dependent processes are killed by default on log-out, but there's an opt out method.
The new state of affairs: dependent processes are killed by default on log-out, but there's a new opt out method.
The only difference is the old opt-out method is now ignored, in order to reap the misbehaving Gnome processes that use it - until of course those processes adapt to the new API or some other processes do something similar, and we can start the game anew, instead of ever fixing the misbehaving code...
This behavior is neither old nor bad and painting it as such is just rhetoric.
When the 'session' that spawns a user daemon ends, usually via logout, the child processes receive SIGHUP which notifies them that their controlling process has exited. Most processes respond to this signal be terminating. However, if a process is intended to persist, then it will simply ignore the signal and continue running.
Now, if we're going to have a rational discussion about this then you need to explain what is wrong with this model, and why transitioning to a systemd specific dbus interface is an improvement.
> new better way
Which has a hard dependency on a specific init system reducing portability. UNIX signals are stable, documented, and so portable you can even use most of them on Windows.
As someone in the linked issue remarked, this will end with everyone calling bar() if they want to persist (the same way they already called foo()), and we'll still have the same level of persistent process issues, but with a systemd dependency.
Worse, this isn't "the systemd developers have proposed a new common API for simplified persistent processes across platforms", but a hard dependency on this particular revision of the systemd dbus interface, or using the systemd-run application to generate a user service file.
I personally don't think that systemd and subverting unix philosophy is worth the trade of faster reboot times on my servers. It solves a "problem" that I don't have with a middleware solution that I don't want.
I have a eight-core CPU. Do I want to wait 80 seconds waiting for my laptop to boot because it is using only one core of CPU? Or do I want to wait 20 seconds waiting for my laptop to boot using all the cores available? I do not know what your use-case is, but to me, the choice seems pretty obvious. Can you provide any reason why it is a "perceived" issue, not a real issue?
EDIT: 80 seconds or 20 seconds are not the hard numbers, these are just used to show 4x speed-up on 8-core CPU. Also, why the downvote? Is HN becoming another echo chamber where people can not accept different opinion without downvoting?
And for reference I have compared sequential vs. parallel on rotating disks and SSD for myself. And there was almost no difference in boot time between both (only significant speedup was from replacing the rotating disk by the SSD).
> if using Linux as a desktop OS, aren't pestered by the extra handful of seconds.
Your last point seems argument for the sake of argument. We are in a technical discussion, not a religious discussion. I did not buy a 8-core CPU to wait extra seconds paying electricity bill for the unused CPUs to validate someones hatred for some specific init system. If systemd can better utilize my system resource and cut my waiting time, I am going to use that. My lost time is the real issue, someones irrational hatred for something bears no value to me.
Init-time amortized over uptime. 5 seconds to boot is a big deal for a box that comes up, lives for 60 seconds, then dies. Not so much for a box that comes up in 5 seconds and stays on for 2 years.
> Your last point seems argument for the sake of argument. We are in a technical discussion, not a religious discussion.
And yet, you're making up numbers to paint a picture you want others to see. You suggest that I'm arguing from opinion when you're making up details because you want to be seen as correct.
> If systemd can better utilize my system resource and cut my waiting time, I am going to use that. My lost time is the real issue, someones irrational hatred for something bears no value to me.
Then use systemd. Or don't. This is what I meant about "perceived." You perceive init's boot time to be your/an issue. I haven't had such a problem.
You're coming off as dramatic and aggressively-emotional, which is my cue to walk away from our exchange.
Parallelized init sequences have benefits, but I suspect this exaggerated '80 second' boot time, and the related '20 second' ideal, stems from either a misconfiguration, complete hogwash, or very particular worst-case setup (assuming an 8 core laptop has an SSD, and assuming a very conservative 200MB/s random-read speed, 80 seconds is enough time to populate 16GB of RAM from disk).
Of course. But an init system should not be optimised for often reboots, since it's a marginal and unlikely scenario.
Starting up is a subset of rebooting, which is fine to spend time optimizing for, even though the Linux architecture lends itself towards long uptimes (well, until the complexity of systemd gets into the mix ;)).
systemd myth #2: systemd is about speed
I'm aware of their propaganda.
> it's working on "perceived" issues with Linux, e.g. plaintext logs being too large or sequential boot processing being too slow.
So you are claiming that the point of systemd is faster boot times.
I attempt to point out that this is false. As the myths page states:
> Yes, systemd is fast (A pretty complete userspace boot-up in ~900ms, anyone?), but that's primarily just a side-effect of doing things right. [...] reducing systemd to its speed is certainly quite a misconception, since that is certainly not anywhere near the top of our list of goals.
This comes up a lot because one of the most visible changes when distributions first switched to systemd was how fast it booted, which led to a lot of people talking about how fast systemd was.. which leads to a lot of people thinking that the speed was the whole point...
The usual response to this from people such as yourself is "who cares, boot times are irrelevant for servers anyway" which is why the next myth points out a couple of reasons why you might care about boot times on a server anyway.
>So you are claiming that the point of systemd is faster boot times.
I was stating that was one of their intentions, which I'm still not convinced wasn't one of their goals (that same blog even suggests that is one of the major advantages (reasons for existence?) http://0pointer.de/blog/projects/systemd.html).
If it's not a symptom of NIH or wanting speed-ups for init, I really don't understand its purpose. If they (systemd/redhat crew) wanted to write a better babysitter, write a faster "better" ("invented there") supervisord. If they want faster startup times, patch init to support N-routes and some hot-path shell scripts with C-based alternatives.
Disclaimer: I do see systemd as an overly complex, desktop-focused (e.g. if it works for GNOME and benefits laptop-users, out it goes), bloated, opaque (binary logs, pretending switches on a binary == having a toolchain) piece of RedHat-funded feature-creep that will be inferior, less stable than any init solution that follows a Unix philosophy. And one that, due to RedHat keeping a number of important players on the payroll, will end up boxing out decisions that would prevent it from being an underlying dependency.
Maybe I don't get it, but I really think I see it for what it is. I know I'm talking a lot of garbage, but I'm quite serious: if I'm foaming at the mouth like I just saw lizard people setting global policy in a secret meeting, please drop some knowledge on me.
It seems that the problem is that some apps need a feature/toggle that would kill them on logout, but the daemon call doesn't allow that - so a reasonable fix is to make such behavior possible (i.e., new api call), and change those apps who need new behavior instead of changing those apps who rely on this behavior staying as implemented, documented and intended originally.
"Desktop" users with their silly "issues" should go somewhere else and leave Unix alone.
Were you unintentionally playing a parody of a traditionalist here?
I can't really say I have much of an opinion on systemd, aside from using it on newer systems and discovering that it hasn't really affected much. But it does seem that there's an unnecessary amount of antipathy between views on this issue, to the point where any discussion about it is difficult to extract any value from due to the noise. This comment kind of sums that up - issues of desktop users are important for desktop users of UNIX, and dismissing that seems a little irrational.
What I was trying to express was that if tmux had changed itself to accommodate systemd, this would have negatively affected the Linux "environment" by making it more acceptable for breaking changes in underpinnings to occur. By future parties not being particularly careful, I meant that this could easily lead to "screen, tmux, and pulseaudio agreed to run as a systemd delegate, and we already wrote the code expecting that you would, so we see this as your bug, not ours." This is particularly concerning as a behavior because RedHat has the money to buy its way.
Systemd is ill-designed throughout, and all of its design decisions do not have any place in Unix world. They're solving the silly desktop "issues" (note the quotes again - I do not believe any of this is a genuine issue at all) always in the most retarded and convoluted way possible.
I do get the impression that systemd isn't particularly well-designed, in that it feels like it would be equally possible to implement a more modular solution with better-defined and -designed APIs, such that the problems it solves could be handled collaboratively across the whole UNIX platform. I get the impression that this would make many people happier.
I do reject your characterisation problems as 'silly desktop issues' however – and it's shallow arguments like that, rather than genuine objections to the implementation of systemd that make the discussion so unconstructive.
A far more reasonable remedy to such a problem would have been to stop using Gnome, KDE and all the similar crap once and for all.
Another issue with tmux, is that maybe you want to have your ssh-agent available in it. To do that in a user friendly way, you have to start the ssh-agent first, then start tmux from a shell where you've exported its variables. Good luck with this now...
Very dissapointed with RedHat.
In this case, the management of user processes is a real problem. It's very easy to accumulate lots of forgotten processes on shared machines with long uptimes. Think ad-hoc scripting servers and jumpboxes here. Think nohup'ed perl scripts that hung waiting for something that'll never happen.
When you run "screen" or "tmux" you expect them to survive a broken session (i.e. you expect them to behave like an independent session). However, when you run something under "nohup" you mostly expect it to survive the controlling terminal going away (a particular screen window or SSH connection being killed). You may be okay with it being killed if you no longer have a session to the server. A user process running in the background without a controlling terminal isn't a service, and from my experience most users don't expect it to be. There is a subtle difference between a "session" and a "connection" that's not properly implemented without systemd (and can be argued not to be properly implemented _with_ systemd).
The thing is how to implement this without "screen" or "tmux" explicitly informing systemd that they want a new session. Touching any API that's used beyond terminal multiplexers defeats the whole purpose.
Why make such a change instead of just making it optional? What would be the impact in such a case?
Did they announce it well in advance and document this?
If not, then the question that springs to mind is "I don't want these guys in charge of helping me administer my systems."
Imagine updating via apt or yum, and having a bunch of important processes killed with no warning when you log out for the day.
But as pointed out already on this page, that would be to miss the wood for the trees. There are many programs that are hit by the fact that they call daemon() with the idea of escaping being killed at the end of the login session and yet are still (now, as of systemd version 230) killed by systemd at (final) logout. The systemd developers themselves mentioned screen and tmux. But others have already mentioned mosh-server and deluged (ironically in the bug report that supposedly drove this systemd change). It will almost certainly affect things like emacs --daemon as well. The list grows on and on.
Should the systemd developers fork emacs so that they have a logind-aware version of emacs --daemon?
A user_daemon() interface is the Right Way. Processes should not survive past the user session without some fancy footwork, not as a default matter of course. Applications will have to be reworked in some cases, but that's because linux has been subject to bad userspace practices all over the place for decades and after the big issues of drivers have been hashed out, time for a little objectivity in handling processes.
Either way, dissent is futile. Jump to a niche distro or ditch GNU/Linux entirely if systemd bothers you.
I think there is some good intent in systemd's change. But they should have approached tmux/screen/community before making the change and a release which broke things. They should have asked for feedback and such and see if tmux is willing to do what they propose. Simply hammering over things as if you hold powers over other's codebases doesn't come off as very positive.
But oh well...
It's nice, no drama, things are where you expect them to be.