
Is Your VirtualBox Reading Your E-Mail? Reconstruction of FrameBuffers from VRAM - brtmr
https://hsmr.cc/palinopsia/
======
thegeomaster
Something similar happened to me, but with a different scenario. I hibernated
my Arch Linux system and then booted into Windows (I unmount all of my NTFS
partitions beforehand), did some work, and then rebooted into Linux again. The
GNOME lock screen doesn't play nice with my NVIDIA drivers (closed-source), so
it doesn't reinitialize the lock screen texture when it wakes up from suspend-
to-RAM or suspend-to-disk. The lock screen now sported a portion of the
desktop from the Windows system which was shut down before. It was a really
interesting thing to see, but a bit unsettling too. Lots of surface for
information leakage through VRAM, it seems.

~~~
cesarb
I've seen worse.

An old Dell laptop, which originally came with Windows, repurposed as a Linux
laptop. It had been used exclusively as a Linux laptop (the Windows partitions
had been overwritten by the Linux install) for several months if not years,
when one day, its lock screen displayed a Windows desktop (I don't recall if
the laptop had just returned from suspend or not). Moving the mouse dispelled
the "ghost" screen and showed the normal lock screen.

The only explanation I could come up with for that was that, somehow, that
particular screen had survived intact in a corner of the video RAM, for
months, until a bug in the Linux video driver made it appear.

Makes one wonder how long can information survive in a laptop's video RAM. The
laptop had never completely lost power (it has a battery, after all), but it
had been powered off daily; it wasn't left on suspend all the time.

~~~
brokencog
Just to clarify ... the laptop you describe was powered off daily? Even Once?
Powered off?

Then what you saw was NOT a VRAM artifact. Aside from OP clearly stating it is
a re-boot issue, powering off the laptop would have removed power to the video
RAM chips -- thus they would have lost their stored data.

What is likely you saw was a screen saver program displaying a random image
which it found on the laptop hard drive. How this came to be found (assuming
it is what you saw, just to give you the benefit of the doubt) would be very
hard to guess without the laptop and it's configuration.

~~~
cesarb
That was a long time ago, so my memory is a bit hazy on the details. If I
recall it correctly, it was the laptop I was using at the time, and I almost
always powered it off when leaving work. I still have another laptop of the
same set at work, so I could look up the exact model tomorrow if you want
(it's an old Dell Latitude IIRC).

But I can clearly recall that I thought at the time precisely that: "how is
that even possible, powering off the laptop should have cleared the video
memory, and this laptop didn't have a Windows install for years!" Yet, there
it was, a Windows desktop with one window (or dialog, can't recall precisely)
open. (I don't recall how I knew it was from its old Windows install; it might
have been showing some work-related information.)

It wasn't the screen saver; IIRC, I used the "blank screen" screensaver (or
some other equally simple screensaver), not a "slideshow" or "random"
screensaver.

Yeah, it's hard to believe. I myself would have found it hard to believe if I
hadn't seen it. I even took a photo, but it was several mobile phones ago, so
I'd have to search through my old backups to see if I can find it.

The only explanation I could come up with was that the video memory on that
laptop for some reason retained its contents as long as the laptop battery
wasn't removed (while the laptop had been powered off many times, its battery
had not been removed), and the Windows driver used the video memory in a
different way than the Linux driver, so that desktop snapshot never got
overwritten. Then one day a glitch made the "read the screen contents from
here" pointer point to it, and so it showed in the screen. It's ridiculously
unlikely, but I couldn't come up with a better explanation; it's even more
bizarre in that the "ghost" image was perfect, with no visual glitches.

------
userbinator
_On a laptop with an Intel HD4000 and a dedicated nVidia card where the OS can
switch between cards, one has to force the usage of the dedicated card to read
from VRAM. The internal graphics card seems to be unaffected at the moment._

Makes sense; integrated GPUs (UMA) share the same RAM as the rest of the
system, which will get cleared as the BIOS does its RAM test at POST (this
might not always be the case if 'fastboot' or similar features are enabled.)

Another observation from this would be that GPUs do not do any memory tests on
their VRAM, which also agrees with the fact that most of the time failed VRAM
just causes artifacts to show up and no message/etc. upon boot. Models
designed for GPGPU use may behave differently.

~~~
mschuster91
> (this might not always be the case if 'fastboot' or similar features are
> enabled.)

I haven't seen a computer or laptop with fastboot disabled in over a decade...

------
plicense
I recently wrote a library that rasterizes a PDF page to an image and was
observing a very similar behavior. It was also highly system dependent and
erratic as well. I would randomly see a distorted image of my own screen when
I run my program on any PDF. Then figured out I was not initializing the
buffer I was using to render the PDF page to image. After I memset'ed it to
zero, the problem went away. I still couldn't quite comprehend what was
exactly happening in the background.

~~~
brtmr
Yes, this is the same way I found this - In the course of another project I
had the same "Huh, this is weird" moment. Then I started experimenting with
different setups (Linux, Windows, VMs) and was surprised that I apparently
almost always succeeded in recovering stuff.

------
rsync
A few thoughts ...

First, tails scrubs memory (or something like that) when you shut down -
should they be scrubbing vram as well ?

Second, wouldn't it be quick and simple to "scrub" vram by filling it up with
intensive usage after working on sensitive information ? What happens if I
watch 2 minutes of 1080p video - wouldn't we expect all previous framebuffer
data to be flushed at that point ?

Finally, I would be very curious to see how this changes as you work with
multiple graphics cards in a multi-monitor setup. I tend to have all of my VMs
in windows confined to a particular monitor that is driven by its own graphics
card - presumably those guest operating systems are only accessing that
framebuffer ?

~~~
datenwolf
> should they be scrubbing vram as well

Yes, but VRAM is a tricky beast. Modern GPUs actually have MMUs and the VRAM
address layout the CPU sees may not cover the whole of the memory of the
graphics card.

> What happens if I watch 2 minutes of 1080p video - wouldn't we expect all
> previous framebuffer data to be flushed at that point ?

No, because the video frames will be queued in a circular buffer and anything
outside of that doesn't get touched.

~~~
cesarb
The only way I can see to reliably scrub VRAM would be to use the GPU as a
"dumb framebuffer" (without video acceleration), like in the good old days.
Then to scrub what was used in the session it should be enough to draw a black
screen, do a page flip, and draw a black screen again.

------
lxgr
This is an issue for any software that is able to access the GPU:
[http://blog.lxgr.net/tag/opengl.html](http://blog.lxgr.net/tag/opengl.html)

~~~
zurn
Just in components that claim to provide security boundaries but then do
OpenGL passthrough without precautions. Web browsers handle this this properly
in WebGL. VirtualBox is at fault here.

Your linked article is about just running two native desktop apps side by side
under the same user account, there isn't any supposed security boundary
between them so it's a different case.

~~~
brtmr
Oracle also discourages the use of 3D acceleration for untrusted hosts. See
here:
[http://www.virtualbox.org/manual/ch04.html#guestadd-3d](http://www.virtualbox.org/manual/ch04.html#guestadd-3d)

------
viclou
I've noticed something similar playing Mini Metro last month.

I was running a Windows 7 VM on an OS X 10.9.5 host with Virtualbox 4.2.28
using my mid 2013 Macbook Air (Intel HD Graphics 5000 1024 MB). Instead of
blurring the game for the weekly upgrade menu, it showed the center of my host
desktop wallpaper, upside-down and flipped (other than that, there was
distortion whatsoever). It really creeped me out.

[http://i.imgur.com/OqYvYjq.png](http://i.imgur.com/OqYvYjq.png)

------
_wmd
Does this have any bearing on WebGL?

~~~
brtmr
The webgl standard has rather strong memory safety considerations:

WebGL resources such as textures and vertex buffer objects (VBOs) must always
contain initialized data, even if they were created without initial user data
values.[0]

As long as browser vendors implement this (and chrome and firefox seem to),
this should not be an issue.

[0]
[https://www.khronos.org/registry/webgl/specs/1.0.3/](https://www.khronos.org/registry/webgl/specs/1.0.3/)

~~~
exDM69
This _was_ a problem with WebGL a few years ago on some OS/GPU/Browser
combinations. It was possible to write a WebGL app that created uninitialized
textures, drew them on screen, grabbed a screenshot and then uploaded that via
HTTP(s) to a server.

This issue should now be fixed, but bugs and/or old unpatched browsers may
still be out there.

~~~
greggman
That browsers have bugs that might leak info is not special to WebGL or GPUs.
The bugs are found, they get fixed.

~~~
exDM69
Oh sure, but in this particular case it seemed like no-one had thought about
the issue that uninitialized textures can be used for snooping screen content,
this wasn't mandated by the spec (IIRC).

------
raverbashing
That's very interesting. I guess VRAM doesn't get powered off during reboot
and (today) it is huge, so the chance of interesting information remaining is
big.

Mitigation strategy: Ctrl-Alt-F1 and work in text mode. (or just fill up video
ram doing stuff in your machine after doing the sensitive work)

~~~
Ciantic
Best mitigation strategy is not to share VRAM by disabling 3d acceleration
with Guests you absolutely don't want to snoop on your host pixels.

~~~
raverbashing
That takes care of the "Guest snooping on the Host" case, but not the case of
physical attack on the machine.

------
anirudhrata
Will shutting down erase the vram completely? Also why does rebooting not
erase the memory completely?

~~~
jerf
[http://en.wikipedia.org/wiki/Cold_boot_attack](http://en.wikipedia.org/wiki/Cold_boot_attack)

I don't specifically know the answer to your first question if the VRAM is not
DRAM (shared memory), but certainly rebooting will not simply automatically
clear out DRAM.

~~~
binbash
shutdown clears, reboot doesnt.

~~~
nitrogen
Shutdown actually doesn't clear RAM, hence the "cold boot attack" terminology.
Eventually the data will degrade, but eventually can be a long time.

~~~
binbash
Yeah. My comment was ment to answer the question above yours...

------
greggman
This is actually one thing that bugs me about OpenGL, DirectX? and Vulkan.

Basically all the insentive so far is best speed. That means there's almost
zero insentive for security in GPU drivers. You can try to implement security
on top of the GPU. WebGL does this. Chrome also does this for most (all?) of
its GPU access meaning even its page rendering is going through Chrome's
secured GPU system that clears buffers and makes sure one process can't read
another's GPU resources.

But, it seems to me this should really be at an OS level for both CPU and GPU
memory. Otherwise it requires all programs to be vigilent. Clear every buffer
before deletion and hope there aren't handles behind the scenes leaving older
copies around.

~~~
DSMan195276
I think it's not really that simple. Clearing buffers certainly isn't free if
you're doing it all the time, and considering that high-end gaming already
pushes GPU's to their max output, the difference between clearing buffers and
not clearing them could be noticeable in the end. Of course, it depends
completely on _how_ you implement this and when you go about clearing. For
example, if a game allocates a pool of GPU memory at the start and then grabs
memory for this pool as it needs it, then the OS clearing the buffers hardly
matters. If it's constantly asking and releasing memory though, then the
buffer clearing is important. I honestly don't know how the bigger titles
handles this, mostly because I don't know exactly how different GPU memory is
from CPU memory (And if the pool approach is possible).

What could be an interesting idea is having a 'GPU security switch' on the OS
level. If it is set on, then GPU memory buffers are cleared at every release.
If it's off, then they're not cleared. My thinking would be that you setup a
system in the kernel, somewhat like root access, where by default this switch
is on, but a program like a game could suggest that it be turned off for that
game to get better performance (And a program like sudo could pop-up with a
request box and ask if you want to turn it off, and then ask for the root
password to do so). What would be even better is if it could be accomplished
on a per-program basis, so only that game's memory isn't zero'd (Which
probably isn't that important of memory anyway) and the rest of your program's
using the GPU are still zero'd.

------
to3m
I wonder why the drivers don't do this on startup. Judging by the specs, most
GPUs have enough memory bandwidth to clear all of their memory in less than
one second.

------
lnanek2
Amusingly, BIOS features to test/clear memory are usually disabled due to boot
performance impact.

------
hello_there
Could this possibly work with WebGL?

~~~
brtmr
see _wmd's comment below[0]

[0][https://news.ycombinator.com/item?id=9246244](https://news.ycombinator.com/item?id=9246244)

