The concept seems clever (rapidly alternate between illuminating the front-panel, and illuminating the contents of the enclosure while displaying a transparency negative on the front panel) but the requirement for an enclosure and controllable backlight limits the scope of the approach. Still, those results look pretty good for mostly off the shelf hardware.
To make it concrete, let's say we want to set a pixel to red, with 20% transparency.
During the transparent phase we light up the object with white light (always). We set this particular pixel to 20% transparency for red green and blue channels.
During the opaque phase we light up the pixel directly (how? I didn't understand this? just a strong spotlight aimed at the display?) and set red to 80% transparency (100% red * 80% opaque) and the green and blue channels we set to block all light.
In the paper, they're effectively using two backlight configurations. When displaying the solid object, they shine the backlight on the object, which scatters some of that light back towards the viewer. During this phase, they mask the image the LCD wants to display by having it render a greyscale image, where black pixels should hide the physical object, and white pixels should let the physical object shine through. In this mode, the physical object becomes the backlight, effectively.
In the image display mode, they point the backlights towards the LCD panel, and display their super-imposed image. In this mode, the object is not being illuminated, and so its contribution to the image is minimal. This is also why an enclosure is required, because the display needs to be able to throw the physical object into shadow during this second phase.
All of this is, of course, happening extremely quickly. The panel alternates between these two modes so fast that the human eye can't perceive the flicker, and persistence of vision simply combines the light contributions from both modes. The result is what you see in the images and video example.
However, they're effectively blocking 50% of the light coming from the object, so an enclosure with additional illumination might compensate that.
It looks like several layers of transparent LCD screens and combined they give a 3D look.
I managed to read it by going here: https://dl.acm.org/citation.cfm?id=3327984 and then clicking the PDF link. Apparently it drops a cookie or something.