
PCILeech - peter_d_sherman
https://github.com/ufrisk/pcileech/
======
floatingatoll
Two related links previously on HN (no comments):

 _Introducing the Memory Process File System for PCILeech_
[http://blog.frizk.net/2018/03/memory-process-file-
system.htm...](http://blog.frizk.net/2018/03/memory-process-file-system.html)

 _Using your BMC as a DMA device: Plugging PCILeech to HPE ILO 4_
[https://www.synacktiv.com/posts/exploit/using-your-bmc-
as-a-...](https://www.synacktiv.com/posts/exploit/using-your-bmc-as-a-dma-
device-plugging-pcileech-to-hpe-ilo-4.html)

~~~
loeg
And from the other side of things, securing the host OS from devices, I found
this article a few years ago interesting:

 _Fuzzing PCI express: security in plaintext (2017)_
[https://cloud.google.com/blog/products/gcp/fuzzing-pci-
expre...](https://cloud.google.com/blog/products/gcp/fuzzing-pci-express-
security-in-plaintext)

------
blibble
I've had intel_iommu=on in my boot cmdline on my personal machines (ones I do
my banking on!) for 5 years without issue

(board+CPU for all machines both support it, dmesg confirms it is indeed on)

~~~
hedora
From the article:

> _Does not work if the OS uses the IOMMU /VT-d. This is the default on macOS
> (unless disabled in recovery mode). Windows 10 with Virtualization based
> security features enabled does not work fully - this is however not the
> default setting in Windows 10 or Linux._

For PCI (shared bus) the device could just spoof packets, but for PCIe, there
are dedicated lanes for each device. I wonder if a physical MITM device is
practical.

------
anonsivalley652
Roughly 10 years ago, I knew a guy who sold various exfil/border-extraction
products to state actors including FireWire dongles similar to Inception (they
may even be effectively the same thing, IDK). He specialized in memory dumping
attacks on several platforms including Windows, FreeBSD, Linux and macOS.

The core problem is that buses should be authenticated, authorized, encrypted,
and selective-/least-privileged channels. Exposing a memory and expansion bus
to the outside world in the name of "convenience" is insane. A trusted set of
components in the OS and in hardware should:

0\. Be able to do a HwIDS checksumming of all firmwares to detect tampering.

1\. Limit devices' ability to connect unless they are authorized by the user,
much like a "hardware firewall" UI.. vaguely similar to say VMware
Workstation/Fusion's dialog when plugging in a new USB device mixed with
something like Little Snitch'es dialog for a process wanting to connect to a
particular port.

2\. Authenticate devices with public/private key certs that are burned in, a
function where the device can answer challenge requests, and Signal protocol-
like construction properly modified for PKI. Then, and only then, can a host
talk securely to a device over an encrypted channel.

~~~
hoistbypetard
We never sold anything for it, but for some demos back around 2004/2005 we
flashed some firewire iPods with custom firmware that performed the attack
described here:

[https://web.archive.org/web/20071011191205/http://md.hudora....](https://web.archive.org/web/20071011191205/http://md.hudora.de/presentations/firewire/PacSec2004.pdf)

(To be clear, those slides aren't mine, but I can no longer find the firmware
we based ours off of, and never did get permission to post source to our mods,
which were never distributed.)

I know a couple of our customers took entirely the wrong lesson from our
demonstrations and banned mp3 players from their office buildings entirely
afterwards :)

~~~
anonsivalley652
Hahaha. That's awesome. :)

Speaking of things that look silly dept.:

Many moons ago around 1998 in uni, I had an HP 48 graphing calculator which
was both programmable and had a very powerful IR LED that was usually used for
serial transfers between similar devices. It so happened that some
enterprising soul made a customizable, preset and _code-learning_ IR remote
control app for it that worked, so I put the brand of the lecture hall's 4 TVs
in it. From the back of the room, some 15 meters away, I subtly turned on all
of the TVs with a phreak'n calculator. Disbelief, confusion and hilarity
ensued.

~~~
hoistbypetard
TV-B-gone before its time.

That’s great. I hope I’d have done exactly the same.

------
tezza
So an Action Replay for x64

[https://gamehacking.org/wiki/Action_Replay_(Amiga)](https://gamehacking.org/wiki/Action_Replay_\(Amiga\))

~~~
teddyh
Also for PC:

[https://www.youtube.com/watch?v=usaioMbE8EQ](https://www.youtube.com/watch?v=usaioMbE8EQ)

------
jjoonathan
How widely deployed are IOMMUs these days? I thought they became a standard
thing a few years ago.

~~~
drewg123
The problem is that, used properly, IOMMUs are horribly expensive.

Consider a NIC driver where you're mapping an outgoing packet for DMA. What
used to essentially be a virtual to physical translation becomes a virt to
phys + entering the phys in the iommu + removing the mapping when the transmit
is complete. This is expensive for hardware and software reasons. At one point
I benchmarked a 100g setup on linux, and with the IOMMU enabled, we lost about
90% of the bandwidth and most of the CPU time was spent in lock contention
over the red-black tree that managed the IOMMU tables. This was 5-ish years
ago, so perhaps things have gotten better.

So that makes people want to just enable the IOMMU for SR-IOV (and full
device) pass-thru to VMs. This is cheaper, since you just set the mapping up
when you allocate phys mem for the guest, and tear them down when freeing phys
mem.

MacOS used to use a really cool trick where they pre-mapped all mbufs into the
IOMMU. That made network traffic transmit and receive comparatively fast.
However,it also prevented lots of optimizations that modern operating systems
use for zero-copy IO (like attaching pages from sendfile directly to mbufs,
similer to skb_frags).

~~~
emmericp
A problem on the hardware side is that Intel's IOMMU TLB is tiny (64 entries),
so using huge pages for all DMA-accessible memory is absolutely required to
get a good performance out of it.

We've done some benchmarks here:
[https://www.net.in.tum.de/fileadmin/bibtex/publications/pape...](https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/ixy-
writing-user-space-network-drivers.pdf) (Figure 9 on page 10)

Only a very basic benchmark, working on more...

~~~
drewg123
Nice paper, and thanks for the reminder of how small the IOMMU tlb is. We
never hit this because we were testing full-sized packets (and really bigger,
because of TSO) and hit host IOMMU management overheads at ~100K to 200k TSO
sends/sec.

~~~
emmericp
Interesting, did you use huge pages?

I think ~100k to 200k TSO "packets" per second should be doable with the
IOMMU. But I guess it depends where the data is coming from. Could be one of
the odd cases where copying data is faster than doing zero-copy, e.g., just
copy everything into the same small set of small-ish buffers to keep the
number of pages that need to be present in the IOMMU small?

------
peter_d_sherman
Related: [https://github.com/ufrisk/pcileech-
fpga](https://github.com/ufrisk/pcileech-fpga)

------
heeen2
Weren't these used in a famous counterstrike cheating scandal in Norway?

~~~
landr0id
Yep, they're also pretty popular with people who cheat in ESEA/FaceIT third-
party ladders. One of the guys, ra1f, eventually outed himself [1]. The
software-based anticheat is fairly decent, pushing people to hardware-based
cheats built off of PCILeech to avoid detection.

[1]:
[https://twitter.com/rra1f/status/1067518342595006466](https://twitter.com/rra1f/status/1067518342595006466)

------
hoistbypetard
Does DMA over PCIe work using USB gadget mode with a Linux device? i.e. could
a Pi be used easily and inexpensively to build an acquisition device for this?

Edit: Bleh. Nevermind. I saw this photo:

[https://gist.githubusercontent.com/ufrisk/c5ba7b360335a13bba...](https://gist.githubusercontent.com/ufrisk/c5ba7b360335a13bbac2515e5e7bb9d7/raw/2df37be67047e19ea2c3f73be67a0ba06fea203d/_gh_mbp.jpg)

with a pcie adapter connected over what looked like USB3 and forgot that it's
thunderbolt on the macbook. I was not quite to the middle of my first cup of
coffee when I asked that.

~~~
q3k
Well USB is not PCIe and doesn't do DMA, so no.

~~~
hoistbypetard
Aren't most of the devices on the linked page doing PCIe over USB3?

Edit: Bleh. Nevermind. I saw this photo:

[https://gist.githubusercontent.com/ufrisk/c5ba7b360335a13bba...](https://gist.githubusercontent.com/ufrisk/c5ba7b360335a13bbac2515e5e7bb9d7/raw/2df37be67047e19ea2c3f73be67a0ba06fea203d/_gh_mbp.jpg)

with a pcie adapter connected over what looked like USB3 and forgot that it's
thunderbolt on the macbook. I was not quite to the middle of my first cup of
coffee when I asked that.

~~~
saagarjha
USB 4!

------
_aleph2c_
An interview with Ulf Fritz:

[https://www.youtube.com/watch?v=MIfY8g73xms&feature=emb_rel_...](https://www.youtube.com/watch?v=MIfY8g73xms&feature=emb_rel_end)

~~~
DrRobinson
Another interview with him:
[https://www.youtube.com/watch?v=W5Yb3q9iJao](https://www.youtube.com/watch?v=W5Yb3q9iJao)

------
MaupitiBlue
Doesn’t this change everything wrt cheating?

~~~
Namidairo
Not really.

A few of the better anti-cheat products have had detection vectors for these
for a while now, from simple things like detecting the driver to outright
probing the device.

~~~
Polylactic_acid
Surely you could combine a cheating chip with a legitimate device like a
network card so to the OS it looks just like a network card.

