
Sandsifter: find undocumented instructions and bugs on x86 CPU - argorain
https://github.com/xoreaxeaxeax/sandsifter
======
mcculley
This is great. That a program can learn about and exploit the CPU on which it
is running from unprivileged userspace reminds me of the notion in Charlie
Stross' Accelerando of running a timing attack against the universe to learn
about the virtual machine in which we are being simulated.

~~~
qubex
I'd never heard of Charlie Stross or his _Accelerando_ book. Thanks for
mentioning that, it looks right up my hard-sci-fi alley.

~~~
vsviridov
Then definitely also check out the Quantum Thief trilogy, by Hannu Rajaniemi.

~~~
eridius
Amusingly, the cover of The Quantum Thief (at least on iBooks) has the
following quote:

> "The best first SF novel I've read in years. Hard to admit, but I think he's
> better at this stuff than I am." -Charles Stross

~~~
moomin
I remain completely unmoved by HR's output. But I'm most definitely in the
minority.

------
_wmd
tl'dr of the slides:

    
    
        Found on one processor... instruction
        Single malformed instruction in ring 3 locks
        Tested on 2 Windows kernels, 3 Linux kernels
        Kernel debugging, serial I/O, interrupt analysis seem to confirm
        Unfortunately, not finished with responsible disclosure
        No details available [yet] on chip, vendor, or instructions
    

He's found a new f00f bug, winter 2017 is going to be interesting :)

~~~
qb45
Observation: the length of the censored "XXX hardware bug" text on the slides
matches neither Intel, AMD nor Transmeta. Unlikely to be VIA too.

Either it's deception or perhaps some obscure low-end embedded vendor.

edit: for the curious, it's "(redacted) hardware bugs" :)

~~~
Veedrac
You mean the black bar on the PDF? That just says "(redacted)".

------
hellbanner
Related:
[https://www.theregister.co.uk/2013/05/20/intel_chip_customiz...](https://www.theregister.co.uk/2013/05/20/intel_chip_customization/)

"Everybody hates the golden screwdriver upgrade approach, where a feature is
either hidden or activated through software, but the truth of the matter is
that chip makers have been doing this sort of thing for decades – and charging
extra for it."

""We are moving rapidly in the direction of realizing that people want unique
things and they are going to want them in silicon. In some cases, it will be
done in software," said Waxman."

Also, Github says "several million" undocumented instructions.. is that right?
I don't know much about assembly but that number sounds absurdly high.

~~~
bem94
>> "several million" undocumented instructions.. is that right?

Bear in mind that doesnt really mean that there are several million operations
/ opcode mnemonics which are undocumented but each distinct instructions.

It is more likely they are "loose" decodings of other instructions, where
changing a single bit of the opcode still causes the CPU to decode the same
instruction.

Toy example: If I encode my (imaginary ISA) 8bit instruction for "ADD EAX EBX"
as 0101_X000 where X is "don't care" then regardless of whether the core gets
0101_0000 or 0101_1000 , it will still execute the ADD instruction.

Now imagine your instructions can be upto 16 bytes long, and you see how loose
decoding can lead to a lot of instructions which are undocumented, but that
the processor is perfectly happy to execute.

~~~
hellbanner
Thanks. The scale is still hard to wrap my head around but I see what you're
saying.

Could this tool find hardware backdoors?

~~~
duskwuff
> Could this tool find hardware backdoors?

Only very crude ones. A competently implemented hardwre backdoor would
probably be data-dependent. For instance, it might trigger when REP CPUID is
called with four specific 64-bit values in R8, R9, R10, and R11 -- and if that
were the case, there would be absolutely no way to discover it by searching.

~~~
teeray
There's also the fascinating variant where a control line charges a capacitor
over time to activate backdoor behavior. Triggering it would look like a bunch
of nonsense instructions that just so happen to keep that control line
energized long enough for the capacitor to cross some activation voltage.

------
dtx1
This is highly interesting. I assume a lot of those are going to be debug and
instructions to help the binning process. Some of these might even unlock
access to parts of the CPUs we aren't supposed to have access too, opening the
doors to custom microcode (unlikely that anyone outside the CPU OEM can do
that though) but may allow us to disable "security features" such as the
Management Engine. This is a really interesting approach and i would love to
see the results ported to other hardware/vendors. The same could potentially
be done with GPUs, ARM-CPUs, etc.

~~~
duskwuff
Separate research has been done on microcode. The general consensus is that
Intel's microcode binaries are encrypted, and are secured with a
RSA2048-SHA256 signature.

[http://inertiawar.com/microcode/](http://inertiawar.com/microcode/)

~~~
webreac
I am surprised private keys have never leaked.

~~~
duskwuff
I wouldn't be surprised if those keys are in a HSM, so they _can 't_ be
leaked. That'd be the safe way of handling it, and it's well within Intel's
resources.

------
fovc
Here's a link to the slides [pdf]:
[https://github.com/xoreaxeaxeax/sandsifter/raw/master/refere...](https://github.com/xoreaxeaxeax/sandsifter/raw/master/references/domas_breaking_the_x86_isa.pdf)

------
partycoder
Also from the same author
[https://sites.google.com/site/xxcantorxdustxx/visual-
re](https://sites.google.com/site/xxcantorxdustxx/visual-re)

~~~
chungy
That looks fun, but the site doesn't seem to have any downloads available?

~~~
partycoder
There was a demo around somewhere. Not hard to find.

There is a similar --but less featured-- open source project.
[https://github.com/wapiflapi/veles](https://github.com/wapiflapi/veles)

------
SAI_Peregrinus
Christopher Domas does some very cool work. His System Management Mode exploit
a few years back was quite nice. It will be interesting to see which processor
it is that he found the ring 3 hard lockup instruction in...

~~~
rasz
He works for spooks - Battelle Memorial Institute, a long-time NSA/CIA
contractor. One of the places that hires officially retired spies.

~~~
azinman2
and therefore.... ?

~~~
qb45
Therefore there is suspicion that spooks may have been using that for years
and revealed it once they've had enough for some reason. I'm not familiar with
any proven case of such situation, but people speculate.

------
d33
...isn't the usability of the tool limited because it's running in userspace,
which has fewer privileges in terms of what instructions can be ran?

~~~
c12
I was wondering this myself until I read the pdf:

> For effective results, the injector should be able to identify instructions
> in more privileged rings, even if it cannot actually execute those
> instructions.

>This approach allows the injector to detect even privileged instructions:
whereas a non-existing instruction will throw a #UD exception, a privileged
instruction will throw a #GP exception if the executing process does not have
the necessary permissions for the instruction. By observing the type of
exception thrown, the injector can differentiate between instructions that
don’t exist, versus those that exist but are restricted to more privileged
rings. Thus, even from ring 3, the injector can effectively explore the
instruction space of ring 0, the hypervisor, and system management mode.

~~~
michaelmior
So basically the same as throwing a 403 instead of 404 for authenticated
resources in HTTP :)

------
partycoder
Lot of weird stuff done happening nowadays in CPUs.

There's a lot of mystery in microcode (equivalent to the CPU firmware), the
"system management mode" aka protection ring -2, and the infamous management
engine.

------
tonyg
I wonder what dbe0, dbe1, and df{c0-c7} do? They are present and undocumented
in all of Intel, AMD and VIA's variations (see p4-p5 of the paper).

------
pbsd
For what it's worth, the size-prefixed jcc/call binutils bug had already been
fixed a couple of years ago:
[https://sourceware.org/bugzilla/show_bug.cgi?id=18386](https://sourceware.org/bugzilla/show_bug.cgi?id=18386)

------
pwdisswordfish
The slides mention an 'apicall' opcode 0ffff0; searching the web turns up
nothing but these same slides. Does anyone know anything about it?

~~~
wmu
It seems to be a MS antivirus bug:
[http://securityaffairs.co/wordpress/60434/hacking/microsoft-...](http://securityaffairs.co/wordpress/60434/hacking/microsoft-
windows-defender-flaw.html)

------
rurban
Regarding the ring 3 hard lockup he didn't disclose yet: isn't that the recent
kaby lake/skylake error, released about a month ago?

------
ngneer
Chip vendors do the same in the course of validation, and technically even
before any silicon has been fabricated, using simulators.

------
shdon
No instructions there to disable the IME?

~~~
pgeorgi
If anything, I'd expect such a flag to hide behind MSRs
([http://wiki.osdev.org/Model_Specific_Registers](http://wiki.osdev.org/Model_Specific_Registers))

That's a mostly unused namespace of 2^32 64bit registers. To hide things even
better, it would also be possible to change behavior based on officially
unrelated registers (eg. MSR $x only acts as IME-switch if the calling address
also ends in $y and esi is $z)

~~~
cesarb
They could also be multiplexed (MSR $x is address/command, MSR $y is data). Or
require a sequence of operations (write this magic sequence of numbers to MSR
$z). Or memory-mapped/IO-mapped (with the mapping enabled/disabled by MSR or
PCI registers). Or be locked by the BIOS during the boot sequence.

But IMO, it probably can't be disabled at all. The "disabling" would be to
change the program it runs to a program which does nothing. So there wouldn't
be a "disable IME" bit; there would be bits to either make its memory visible
to the main CPU cores, or to read/write to its memory, and it's possible that
these bits are accessible only from the IME side, or from SMM.

~~~
Kliment
The easiest ways to access them is to rewrite that section of the BIOS
directly, such as [https://github.com/corna/me_cleaner/wiki/How-does-it-
work%3F](https://github.com/corna/me_cleaner/wiki/How-does-it-work%3F) which
literally overwrites them with nops

------
egberts1
found another that is QEMU-specific.

[https://github.com/unicorn-
engine/unicorn/issues/364](https://github.com/unicorn-
engine/unicorn/issues/364)

~~~
egberts1
It is more about modifying executable code space and not making it stick. Good
enough for fooling AV.

------
purpleidea
wow... anyone have a link to the video of his talk?

------
pmarreck
Is this basically a CPU fuzzer?

~~~
deathanatos
The subtitle at the very top of both the page and the README…

> _The x86 processor fuzzer_

------
brawny
Out of curiosity, are there any toy compiler projects out there that try and
make use of the incedental instructions? Could you possibly expect to see a
with while performance boost (I'm thinking it would be unlikely...)

------
m00dy
Someone built a fuzzer for cpus

