

Jellyfish: GPU rootkit PoC - adamnemecek
https://github.com/x0r1/jellyfish

======
the8472
> Can snoop on cpu host memory via DMA

don't linux kernels use IOMMU mappings to protect most of the physical memory
from arbitrary device access?

~~~
semi-extrinsic
If I understand it correctly, at least for nVidia cards, when the GPU is
running code in kernel space it is capable of going around the IOMMU and
access physical addresses directly. This is required to make stuff like
GPUdirect RDMA over Infiniband work.

~~~
the8472
>GPU is running code in kernel space

I'm not up to speed with GPU-computing. Do you mean the GPU drivers executing
in kernel mode or is there some magic going on in the GPU itself that it has
non-MMU'd access to the main memory?

> This is required to make stuff like GPUdirect RDMA over Infiniband work.

Well, let's assume that many people don't need this. Would IOMMU protection
apply then?

~~~
semi-extrinsic
To answer your first question: I don't think there's any code running on the
GPU in kernel space except for the DRM driver part of the GPU drivers. As for
the second question, I'm not sure. nVidia at least uses the same driver for
GTX and Tesla class cards, but I assume the driver runs different code based
on what card is installed, and presumably turns features like IOMMU on/off
based on the card present.

Whether SLI also requires working around IOMMU I don't know.

~~~
wtallis
I doubt that they could _disable_ the IOMMU based on the presence of a pro
card, since the virtualization uses that require an IOMMU are a lot more
common than anything RDMA that might conflict with an IOMMU.

~~~
semi-extrinsic
Per the nVidia documentation:

RDMA for GPUDirect currently relies upon all physical addresses being the same
from the PCI devices' point of view. This makes it incompatible with IOMMUs
and hence they must be disabled for RDMA for GPUDirect to work.

Sounds like they require the user to disable it.

~~~
wtallis
So the _real_ requirement is probably just that the GPU and the NIC get
compatible mappings, and you could probably compartmentalize them as a
pair—the same as how all the legacy PCI devices behind a PCIe to PCI bridge
need to dealt with as a unit for IOMMU purposes.

------
revelation
There seems to be nothing here but malware that uses GPU memory for storing
captured data.

