
Avoid speculative indirect calls in kernel - grover_hartmann
https://lkml.org/lkml/2018/1/3/797
======
userbinator
Looking at the patches Linus is commenting on: as an Asm programmer, this is
absolutely horrible --- basically every indirect call (one instruction) turns
into a _seven_ -instruction sequence that will, due to preventing speculation,
result in massive slowdowns:

[https://lkml.org/lkml/2018/1/3/770](https://lkml.org/lkml/2018/1/3/770)

Unlike the KPTI patches, which only affect things on each system call, this
happens _on every indirect call_ and probably bloats the code considerably
too. I can see why Linus is not happy.

Edit: To give a bit more background, predicting indirect calls and speculating
into them is absolutely critical to getting good performance from OOP-ish code
which tends to use them a lot (virtual functions in C++, function pointers in
C). The Linux kernel is (thankfully?) not very OOP-ish, but it does rely on
indirect calls (function pointers) extensively.

~~~
keldaris
I'm frankly shocked Linus was anywhere near as restrained as he was upon
seeing this monstrosity. Nothing of this sort should ever make it into the
kernel without explicit and loudly announced opt-out switches. If anything
close to this does make it in, even as an opt-in, this will easily be the
biggest performance regression in recent (perhaps all-time?) kernel history.

~~~
redshirt
Nothing like this should have ever made it into hardware. It's lazy
implementation (or performance at the expense of security).

~~~
keldaris
Performance at the expense of security is a compromise made daily in every
aspect of IT. The only way to make systems perfectly secure is to make them
unusable.

Now, you can certainly argue that this particular tradeoff wasn't worth it,
and in hindsight it's certainly a questionable one. However, do note that side
channel attacks have been known for decades (since out-of-order CPUs became
widespread in the 90s) and until this point have rarely presented a
significant threat to most systems.

------
jcranmer
This is my understanding, please correct me if wrong:

There's a basic class of bugs (let's call it Spectre-class) that's about
exfiltration of memory contents via side channels introduced by speculative
execution. Basically every processor that can speculatively execute code
(i.e., every modern one in the past two decades or so) is potentially
vulnerable.

Of known Spectre-class bugs, one (Spectre) relies on branch predictor tables.
All tested CPUs (ARM, AMD, Intel) were vulnerable to Spectre. The other bug
publicly announced (Meltdown) relies on when the #GP(0) exception actually
gets thrown and is specific to Intel CPUs. The fix for Spectre is to basically
banish branch prediction, and the fix for Meltdown to is to unmap kernel pages
when in userspace code.

Beyond these two bugs, there may exist other Spectre-class bugs specific to
different architectures/vendors that are not yet thoroughly investigated.

What did I get right and wrong?

~~~
AlphaSite
AMD alleges that the fix for Spectrr can be ameliorated with minimal
performance overhead.

------
ysleepy
Considering Intel is working on this for the last 6 months, it is indeed a
little telling that they did not have the patch behind a feature flag. It also
explains why AMD was so passive aggressive while inserting the exception for
their CPUs. Seems like Intel was trying to make this some sort of act of god
type of deal instead of writing out the if(intel){horrible mitigation}.

~~~
jhanschoo
Can I see an article/discussion about AMD being passive-agressive about it? I
want my popcorn.

~~~
none_to_remain
It is dry -
[https://lkml.org/lkml/2017/12/27/2](https://lkml.org/lkml/2017/12/27/2)

~~~
cryptonector
Hardly passive-aggressive. What would the alternative have been? Leave PTI on
on AMD just out of sympathy for Intel?!

------
mrmondo
Well said and I think Linus was especially well restrained or perhaps
'careful' with his wording here.

~~~
alexeiz
Linus is getting old.

------
dexwiz
So is there no way to disable the performance crippling fix? If you ran an air
gapped or highly controlled environment wouldn’t you want th option of
disabling this to keep performance? How many Giga Terra or Exo FLops will be
lost to this fix in super computers? Or is the way to disable it just not
merge this patch into your custom kernel code?

~~~
Decabytes
Correct me if I'm wrong, but I don't think super computers are updating there
kernels at all.

~~~
striking
New software is written with newer kernels in mind. To continue running newer
software, you will need to update your kernels.

Additionally: you might update _infrequently_ when it comes to feature
releases, but you'll definitely need security patches.

------
revelation
Love it, an LFENCE on every indirect jump in core 32/64bit entry assembler
code, not to mention the cost of the trampoline itself. Linus is right to say
that this hack should be reserved for configured as broken CPUs..

~~~
bewaretheirs
If I'm reading it correctly, the lfence is never reached for real (only
speculatively..)

code invoking the thunk pushes the target address on the stack, then branches
to or calls the thunk.

The first instruction of the thunk is a "call" to a nearby address, skipping
over an infinite loop (2: lfence; jmp 2b)

the lea n(%xsp),%xsp discards the return address pushed by "call", and the
subsequent "ret" "returns" to the branch target rather than to label 2.

perhaps the speculative execution engine doesn't take the stack pointer
adjustment into account and so gets stuck in the lfence loop, only being
snapped back to its senses when the lea completes..

------
zorangus
Linus missed the bus. The Spectre attack doesn't require speculation across
protection domains.

------
StavrosK
Why does he mention ARM64 instead of AMD?

~~~
hacknat
Because Intel writes almost all of the x86 arch code for Linux, so AMD gets
stuck footing the bill for hardware bugs like this, even if AMD isn’t
affected. From Linus’s perspective it’s the same arch, because, well, it is.

~~~
anfilt
Well he does say patches should written that don't treat all CPUs as crap. AMD
had to basically say "Wait a moment! This does not affect our CPUs what are
you doing Intel?"

~~~
StavrosK
As far as I understand, one of the two attacks _does_ affect AMD, and it's
hard to solve as it relies on normal branch prediction.

~~~
doikor
Amd claims that two of the three spectre attacks don't work on their hardware
and the last one (variant one) is fixable with negligible performance impact.

[https://www.amd.com/en/corporate/speculative-
execution](https://www.amd.com/en/corporate/speculative-execution)

~~~
m_mueller
I still don‘t get how variant 1 is fixable with negligible impact. Do we have
any details on this as of now?

------
yuhong
My favorite is espfix. espfix64 is worse than espfix32 too. I have suggested
limiting modify_ldt to root instead.

------
senatorobama
Does this disable the branch predictor for all function calls?

------
qaq
Very reserved for Linus :)

~~~
anfilt
Indeed. Although he did say: "... and that really means that all these
mitigation patches should be written with "not all CPU's are crap" in mind."

------
maxxxxx
Somebody needs to tell him that the plural of CPU is "CPUs", not "CPU's" :-).
I am afraid that even more people will start overusing apostrophes.

~~~
andrewflnr
Using an apostrophe is the technically correct way to pluralize acronyms.

~~~
noncoml
How do you distinguish between plural and ownership if you use ‘s for both?

~~~
madez
It's not uncommon for natural languages to have homonyms, so people are used
to deal with them. The context allows deciding which is the correct meaning.

------
shaki-dora
This seems unnecessary hostile. It's quite obvious that these patches had to
be completed in a hurry, and in addition to any number of similar patches for
other systems. Configurability is currently useless for Intel CPUs, as all of
them seem to be affected.

Depending on how their CPU development pipeline works, it's quite likely that
even the next generation will still be affected, giving everyone plenty of
time later for such niceties.

It's also slightly too harsh to call all of Intel's work "crap" when this bug
has apparently existed for the better part of two decades without being
noticed.

~~~
maxlybbert
I didn't take the comment as "all Intel CPUs are crap," but that (1) the fix
is needed for crap CPUs and (2) the fix is turned on unconditionally on Intel
CPUs, even future CPUs. That is, _the code itself_ assumes that all CPUs that
could be crap actually are crap. Torvalds wants a flag that could be set
appropriately when Intel starts shipping CPUs that don't have this problem.

------
geofft
This would be more convincing if the x86 CPU manufacturer he used to work for
were still in business.

I mean, I could sit here and criticize how Linus runs his kernel development
project and how so much insecure code gets in, and talk about what a
"competent" kernel developer would do, but since I don't run an equally
successful, more secure kernel, it's just armchair quarterbacking.

~~~
foxylad
Interesting concept - we should discount all criticism except from equally or
more successful companies than Intel?

    
    
        if geofft.karma >= foxylad.karma:
            print('No comment.')
        else:
            print('This was a really badly thought out comment.')
    

So... No comment.

~~~
geofft
I wonder why it is that when I say "I don't think that X should be doing Y in
this situation because Z," I so often get responses of the form "So you think
that nobody should be doing generic-version-of-Y ever?" That's probably a flaw
in my own ability to convey my points. Let me see if I can convey this one a
bit better.

First, I think that there's a distinction between constructive, humble
criticism, of the form "Why can't the CPU be designed in such a way that ___",
and statements like "Well, a competent CPU designer would have done ___". That
distinction is not just politeness (although politeness is of intrinsic
worth); that decision leads to _better technical outcomes_. There might be a
good answer, like "It can't be designed like that because of the following
constraints" or "We tried it and couldn't get it to work"; there might just be
an answer like "That's totally the right thing to do and we just cut corners"
or "We never thought of it but our competitor does that and it seems to work
for them, yeah." Unlike arguments about how competent people actually are,
these discussions contain actual technical content, and help different people
understand each other's points of view (which is the whole point of having an
online, public, decentralized development community, is it not?).

Second, yes, I think that we should _discount_ , though not by much, criticism
from people who say "I could totally do that in a better way" and have not
actually tried to do that (or have tried and failed to launch a product).
There's a lot of things about building complex systems (like modern CPUs) that
aren't apparent on paper, and only become apparent once you try to reach some
challenging level of scale or performance. This is not at all to say that
there's _no_ place for outside criticism; often outside criticism can offer
fresh perspectives, or suggest a way to simplify the system and avoid the
emergent problems of complexity. And often there are people who are genuinely
qualified and really could build a better system, but happen to be working on
something else. But it's still fundamentally outside criticism. The only
people who can really say "Yes, you totally could have done this instead" are
those who actually did.

Third, in this particular scenario, it happens to be the case that ex-Intel
employees with insider knowledge have called out Intel for cutting corners
(see the "Update" section of [https://danluu.com/cpu-
bugs/](https://danluu.com/cpu-bugs/)), which seems like a much more productive
basis to have a technical conversation.

And if those stories are to be believed and if they're relevant to this
problem, the problem is not the competence of engineers - the problem is the
competence (or the confused incentives, really) of management who are not
telling their engineers to invest time into doing things the engineers know
perfectly well how to do, or are telling their engineers to invest time into
building other features that make those things harder. (For instance, quite
possibly some part of management got worried about benchmarks of syscall
performance and told their engineers to prioritize that over making sure that
there aren't side-channel leaks between rings.)

~~~
foxylad
Sorry, I have a weakness for reductio ad absurdum, so fair enough to call me
out on that. But in this case I'm glad I gave in to temptation, because your
response was really valuable.

