
Re: [BREAKAGE] Since 4.18, kernel sets SB_I_NODEV implicitly on userns mounts - Valmar
https://lkml.org/lkml/2018/12/22/221
======
ivraatiems
I've been extremely critical of Linus Torvalds in the past. I was glad to see
him decide publicly to change his behavior, and I am glad to see him doing
better here.

Personally, I think this level of disappointment/anger/trust issue would be
best dealt with face-to-face or via a phone or Skype call, or at least an off-
list email. However, given the unique situation of the Linux kernel, I think
it was appropriate to send to the public list.

The only line I'd remove or finesse is "Yeah, this is complete garbage."
Otherwise, I think this is a great example of how to convey anger without
personally attacking anyone. Linus doesn't swear, he doesn't call Eric stupid
or curse him out, but he still says what he means: The behavior was out of
line, it has to stop, and, most importantly, it represents a breach of trust.

He's not saying Eric is a bad, stupid or worthless _person_ \- he is saying
that in this instance, Eric's _behavior_ is bad. That's the key difference.

~~~
geofft
He's still saying that Eric's behavior is so far beyond acceptable that no
defense can be offered, and _the entire rest of his work_ is worthless until
he admits that. I think Eric probably had a reasonable explanation for this
change, and the fact that Linux containers work so well is due far more to
Eric than Linus, so in a healthy project it would be worth hearing out Eric
and letting the more meritorious idea win by technical argument. But Linus's
email closes that possibility.

~~~
floatingatoll
At no point does he declare Eric's other work worthless or otherwise
"permanently" anything.

He indicates that trust has been breached, temporarily, withdraws his
willingness to accept patches due to the breach of trust, and specifies
precisely how Eric can rebuild trust:

> until you have made it clear that you comprehend this very fundamental issue

~~~
geofft
The problem is that Linus has transformed a technical disagreement into the
language of a breach of trust. There is _no_ argument that Eric can offer to
demonstrate that he weighed the factors and determined breaking userspace was
the right call (it's certainly a call commonly made in kernel development,
even by Linus himself on occasion!). And Eric can't argue fully because he
feels a duty to his users to continue the rest of his work, which is now being
held hostage. (Should he refuse to concede the point, then permanence kicks
in. It reminds me of certain teachers I had in high school who I still believe
were wrong on the facts but the threatened social consequences for pushing the
point weren't worth it.)

So fundamentally we have the same _practical problem_ as with personal
attacks: that it's bypassing what should be a discussion on technical merit.
Maybe Eric's feelings aren't hurt, but it's still not clear we're building the
best kernel we could.

~~~
_asummers
If you read the rest of the chain til the end, Linus acknowledges that
occasionally userspace has to break. He mentions security issues and extremely
old code that provides no other option but breaking it, but then suggests that
this was not one of them.

> ( _) Yes, there are exceptions. We have had situations where some interface
> was simply just a huge security issue or had some other fundamental issue.
> And we 've had cases where the breakage was just so old that it was no
> longer fixable. So even rule #1 can sometimes have things that hold it back.
> But it is _very* rare. Certainly nothing like this.

------
akerl_
I think this email underscores something people miss when discussing Linus's
communication style.

It is entirely possible, as we see here, to convey the criticality of
something without crossing over into personal attacks.

~~~
mmirate
Convey to us reasonable people, reading it at a distance as 3rd-parties?

Absolutely.

...

Convey to the intended recipient, with his inevitable personal/emotional/etc.
attachments to the topic, in such a way that it takes the intended effect?

A pending question-of-fact, and one that can only be answered very indirectly
and in the long run.

Heck, if the answer really is "no", then it's possible we may _never_ know!
(until the NSA pwns us all, or at least pwns the subset of us who didn't
kowtow to whoever the central socio-econo-political authority of that era will
be)

~~~
akerl_
I don't think it's actually a pending question whether avoiding personal
attacks makes people more likely to listen to a message.

~~~
kryogen1c
> I don't think it's actually a pending question whether avoiding personal
> attacks makes people more likely to listen to a message.

That depends on the context. When you're talking about changing ingrained
behaviors, emotional shock and awe can definitely be necessary and the
shortest, most painless path to resolution.

the underlying incorrect assumption is that everyone will do the right thing,
if only nicely and logically talked to. This is not true and obvious to anyone
with experience managing large amounts of people, especially so when firing is
not an easy option.

~~~
darpa_escapee
> _the underlying incorrect assumption is that everyone will do the right
> thing, if only nicely and logically talked to. This is not true and obvious
> to anyone with experience managing large amounts of people, especially so
> when firing is not an easy option._

I've yet to see abusive management tactics work in the long term. It might
work in the short term, but people with the means will take it as a signal to
jump ship because no adult is going to accept abuse if they don't have to.

I've heard your exact sentiment echoed by people who aspire towards
management, and it always comes across as a weird power trip fantasy.

~~~
kryogen1c
> I've yet to see abusive management tactics work in the long term. It might
> work in the short term, but people with the means will take it as a signal
> to jump ship.

Like I said, depends on the context. Being mean has it's place, as does being
nice. Those are the extremes and the Lions share in the middle is non-
emotional transactionalism.

------
philwelch
Damn, cold-bloodedly nice Linus is _terrifying_.

~~~
mikeash
Indeed. I think this illustrates that being civil is not just about making
people more comfortable, it’s also more effective.

------
raverbashing
A bit offtopic, but it seems the issues the kernel is handling are getting
even more esoteric

Iteractions between different permissions and conditions is getting ever more
complex

But yeah, kernel shouldn't break userspace, and this should have been
internalized already by the maintainers

~~~
stestagg
Strengthening controls on a system while simultaneously not breaking any
client is an incredibly difficult goal.

Unfortunately, unless there is a single, united team focussed on this single
goal (kernel is not by design). Then some fingers will be burned, because of
the complexity of getting it right.

Some contributors will struggle, and others will fail at it. Unfortunately,
that’s just stats.

~~~
hedora
The systemd and various “secure” linux teams keep adding poorly thought out
access controls that no one understands, that have escoteric loopholes and
break things in terrible ways.

I wish they’d just move back to the original unix permission scheme, plus
jails.

BSD and solaris did much better jobs of this a decade ago. Hopefully one of
those will eventually take over again.

~~~
JdeBP
Solaris and (some of) the BSDs have NFS-style ACLs, and do not have the
original Unix permission scheme either.

------
Havoc
Haha well at least he isn't cursing anymore

~~~
mmirate
Only time will tell whether that change is for better or for worse.

(Unless your chosen metric-of-value relies on feelings instead of actual
concrete results.)

~~~
tyingq
I like the new tone. It is clear and conveys severity/urgency without the
swearing and personal attacks.

Compare to:
[https://lkml.org/lkml/2012/3/8/495](https://lkml.org/lkml/2012/3/8/495)
That's the same guy, so clearly, the swearing didn't have much lasting effect
:)

~~~
mmirate
That's one person of many. And it's not a question of aesthetics of the
message in the short term, but of the quality of the overall results in the
long term and the effort needed to achieve them.

Hopefully this won't happen, but it's entirely possible that this change will
result in Linus finding himself with a much greater number of broken, half-
broken and/or subtly-broken PRs to review than ever before, increasing the
chances of something like
[https://lwn.net/Articles/57135/](https://lwn.net/Articles/57135/) succeeding.

And having a kernel backdoor ship worldwide is way worse than having a few
recalcitrant kernel developers be exposed to - oh no! - coarse language.
Results > the "fee fees".

~~~
saagarjha
Are you saying that Linus has been able to review fewer patches because his
personality has driven people away, and by being nicer he’d be overwhelmed
trying to review code?

~~~
mmirate
I'm saying that the patches he has reviewed may have been given more
attention-to-detail by submitters and lieutenants because of his personality,
in which case by being nicer he would lend his time to be abused to a greater
degree. Note that I said "more patches that are broken/semi-broken/subtly-
broken", _not_ "more patches overall".

------
PavlovsCat
I'm not a kernel dev, I hardly know what the kernel is, but even I know this
by now.. _if a change breaks something in user space, it 's always a kernel
bug_.

~~~
geofft
The funny thing is that you've got that drilled into your head even though
it's not true. Userspace breaks all the time, it's just that some of them
matter to Linus and some don't, and he's never been clear about which is
which.

Several months ago at work I found a vendor app that was calling listen(0),
i.e., listen for up to 0 connections, and worked fine on older kernels because
they rounded to a minimum of 8. Newer kernels stopped doing that. Nobody
cared. Nobody got criticized, constructively or unconstructively. (By the time
we noticed it was several kernel releases upstream so bringing it up would
just be asking for _another_ change to userspace.)

Linus himself broke userspace once in the hope nobody would notice. Spoiler:
they did.
[https://lkml.org/lkml/2017/5/29/541](https://lkml.org/lkml/2017/5/29/541)

And that's not counting all the things that aren't part of the stable
interface like procfs and sysfs nodes, device names, etc. There is a clear
version of what Linus is trying to say. He's just never said it.

~~~
xani
So your point is that if you don't report breakage it won't get fixed ?

------
Arnavion
Heh, I know of one time the kernel did break backward-compatibility,
intentionally even -
[https://github.com/torvalds/linux/commit/6b99e3569ba17b9fd38...](https://github.com/torvalds/linux/commit/6b99e3569ba17b9fd38514d66591ae728b778e3b)

My fan controller used the sysfs paths mentioned in the commit message
directly. So while the commit is probably correct that users of the lm-sensors
library would not be affected, _my_ application was.

It was relatively easy to fix since I was the only user of the application, so
I didn't care about being compatible with older kernels.

------
jrwr
I've got a good one, I found a Kernel Bug in the memory manager of all places
by being a tryhard in mergerfs (and upstream fuse was broken as well)

[http://lkml.iu.edu/hypermail/linux/kernel/1610.0/00878.html](http://lkml.iu.edu/hypermail/linux/kernel/1610.0/00878.html)

------
ec109685
This attitude is what has allowed Linux containers to work as well as they do.
You can put software built and tested on a very old kernel and it runs
successfully on a modern one. Container operators do not have to deeply worry
that every kernel update will break all their developer’s applications.

~~~
greenyoda
This attitude is what allows _any_ operating system to be usable. Nobody would
want to develop code under Windows (or buy that code) if they knew that it
would break when the user moved from Windows 7 to Windows 10. In the IBM
mainframe world, people can still run code written in the 1960s (without even
recompiling it).

If Linus didn't fanatically enforce backward compatibility, nobody could rely
on Linux for mission-critical systems.

~~~
ec109685
There were BC breaks across Mac OS X’s kernel. Windows does a good job keeping
things backwards compatible as well.

