
1000fps image projection on deforming non-rigid surface - hongzi
http://www.k2.t.u-tokyo.ac.jp/vision/DPM/
======
timdorr
The second generation of the projection tech does color:
[http://www.k2.t.u-tokyo.ac.jp/vision/dynaflashv2/index-e.htm...](http://www.k2.t.u-tokyo.ac.jp/vision/dynaflashv2/index-e.html)

And they can now do the tracking without the infrared paint:
[http://www.k2.t.u-tokyo.ac.jp/vision/MIDAS/index-e.html](http://www.k2.t.u-tokyo.ac.jp/vision/MIDAS/index-e.html)

~~~
SiempreViernes
Very cool, a sort of live-action cgi capability.

Hope it doesn't easily scale out to larger spaces and crowds, or the current
tech industry would soon have public spaces filled with ads projected on
peoples belongings.

~~~
throwaway744678
The target needs to be marked with infrared ink: you can project only on
specific surfaces prepared in advance.

Edit: discard this comment, I misread the parent comment.

~~~
JoshTriplett
Not according to the comment that started this thread, which suggests that
they no longer need the infrared paint.

------
daenz
Very cool. I do live projection work[0] and latency is always the killer with
immersion. Anything higher than a 1000/90ms latency breaks integration at
normal dancer movement speeds. 1000fps seems like overkill but it allows for
very fast movement.

0\. [https://youtu.be/ggRcDQZWD_8?t=1281](https://youtu.be/ggRcDQZWD_8?t=1281)

~~~
goblin89
Is this real-time 3D taking into account dancer’s motions? I imagine it takes
a crazy GPU rig?

In any case, very impressive.

~~~
daenz
It is, but I cheat! The dancer has a small android device with a custom
gyroscope app, mounted in the middle of her back. I can get her general
orientation accurately this way (more accurately, and faster, than state of
the art pose estimation).

I am in the process of bringing together a community around art making like
this. Let me know if this is something that interests you.

~~~
goblin89
It is of interest! I am in the process of learning TouchDesigner, looking to
integrate it with Ableton Live. I doubt I’ll have enough GPU power for real-
time 3D renders based on changing sound or visual input though…

Currently I am considering pre-rendering scenes to given BPMs (where
applicable) and doing only limited realtime alterations with TD nodes.

------
cheschire
This seems like an extremely advanced version of those sand tables at science
centers that kids play with, where digging a trench in sand with your hand
modifies the projection to affect virtual water flow.

The youtube video[0] from that page is especially interesting.

0:
[https://www.youtube.com/watch?v=-bh1MHuA5jU](https://www.youtube.com/watch?v=-bh1MHuA5jU)

~~~
raihansaputra
Yeah, and for the sandboxes, latency is the part that breaks the immersion.
But it is straightforward to set up with a kinect and a projector.

------
gambler
You want a high-bandwidth interface to computing? Combine this with
Dynamicland technology ([https://dynamicland.org/](https://dynamicland.org/)).
This makes _way_ more sense than creepy neural interfaces. Best of all, it
would allow people to freely collaborate on things _in real life_.

~~~
airstrike
This here is the real winner. The thread is full of amazing ideas but
successfully executing this one combination would be a complete paradigm
shift.

------
pvinis
This is crazy awesome! And look at this one! I don't know if it comes from the
same project, but looks cool. [https://www.youtube.com/watch?v=c40cxE-
dfPg](https://www.youtube.com/watch?v=c40cxE-dfPg)

~~~
timdorr
Yes, same research group:
[http://www.k2.t.u-tokyo.ac.jp/vision/MIDAS/index-e.html](http://www.k2.t.u-tokyo.ac.jp/vision/MIDAS/index-e.html)

------
bsenftner
In 1899, H.G. Wells wrote "A Story of Things to Come", which later was adapted
become the 1936 film "Things To Come". In the original story, the main
characters mention being irritated by the advertisements projected onto the
backs of the people they walk behind. Old idea, only now possible without
image distortion.

------
ridaj
The latency is way more impressive than the fps

Edit: the video gets the point across more effectively IMO:
[https://youtu.be/-bh1MHuA5jU](https://youtu.be/-bh1MHuA5jU)

~~~
vernie
Well you can't really achieve low latency if the end-to-end system has low
FPS.

~~~
ridaj
Yes but you can achieve high fps very easily if you have a day to compute each
frame! :)

------
phkahler
Another use of a high speed projector would be to create real 3d display
anywhere in a volume swept out by a moving surface. Objects will be
translucent, but otherwise real 3d with wide viewing angles.

~~~
jhallenworld
It's been done-

[https://thefutureofthings.com/3029-the-return-of-
the-3d-crys...](https://thefutureofthings.com/3029-the-return-of-
the-3d-crystal-ball/)

[https://www.youtube.com/watch?v=9U80TK6PAVc](https://www.youtube.com/watch?v=9U80TK6PAVc)

~~~
phkahler
The first I heard of it in the 1980s. TI used a laser to project point on a
rotating surface. Then someone used a flexible mirror in front of a speaker.
The mirror would flex convex/concave changing the apparent distance to a
vector display screen.

Just saying a modern 1000fps display could do this much better.

~~~
leni536
That flexing mirror idea is really smart.

------
Nelkins
My first thought is that this could provide some awesome effects for live
theater.

~~~
rhizome
The Mandalorian on Ice!

Iron Man: The Musical

Benjamin and the Buttons in Concert

~~~
Jamwinner
Ruining film was bad enough.

~~~
EvRev
Ruining comics was bad enough

------
smabie
This is just too much. My mind is completely blown. And makes me think of how
many things I think are impossible but actually are or will be in the near
future.

~~~
baroffoos
Similar feelings. The video doesn't go easy on the tech either. He is shaking
that paper violently and I can't see any faults. The part where he has 2 bits
of paper as well as when he stretches his shirt are mindblowing.

------
ohadron
This is from 2016, but still very cool.

~~~
qnsi
this makes me even happier. Means SOTA is even more advanced?

------
elif
I was fortunate to see some of the 2019 projection mapping competition on
odawara castle. It is absolutely mind bending technology.

This is the real beginning of augmented reality, not VR or cell phones

~~~
Frost1x
One downside with this approach is that you need something to bounce the light
off of (e.g., a surface), so adding virtual objects to AR is difficult if the
objects aren't positioned at the surface of real world objects. That's an
effect you often want to achieve in AR applications.

~~~
elif
Nah if you see it in person you understand. No matter the surface, you can
create a virtual 3d space from the perspective of the observer.

[https://youtu.be/deJnHFQtbiE](https://youtu.be/deJnHFQtbiE)

This gives you an idea but in person the illusion is stronger.

------
hongzi
This lab is doing a whole series of amazing stuff---

500ps hand gesture recognition system:
[http://www.k2.t.u-tokyo.ac.jp/perception/zSpace/index-e.html](http://www.k2.t.u-tokyo.ac.jp/perception/zSpace/index-e.html)

Robust tracking for moving objects:
[https://www.youtube.com/watch?v=p7IL0Gvux7U](https://www.youtube.com/watch?v=p7IL0Gvux7U)

------
balabaster
Having just watched a bunch of videos on Deep Fakes after the revelation of
the Deep Fake video of Nixon's Moon Landing Disaster speech on here this
morning, I can't help but feel like this is something else that will make
fakes more and more difficult to distinguish from genuine.

------
qnsi
I think it would be very interesting idea for a music show. Tell everyone to
dress using white tshirts and create this kind of projection from multiple
beacons standing spread in the venue. Not sure if that would be possible but
very cool to see IRL and talk about

~~~
onion2k
My initial reaction was that it would be brilliant to use in theatre
productions.

~~~
icebraining
The idea of a stage version of A Scanner Darkly's scramble suit seems amazing.

------
Dunedan
Reminds me of Jeff Han's original multi-touch demo. Slightly different topic,
but also mind-boggling back then:
[https://www.youtube.com/watch?v=EiS-W9aeG0s](https://www.youtube.com/watch?v=EiS-W9aeG0s)

------
moron4hire
Back in 2012, my wife and I attended a stage show called "The Animals and
Children Took To The Streets" [0]. It was done with "dumb" projectors, with
choreographed movement of different screens on the stage, but created a highly
dynamic show.

Remembering that show, and seeing these videos, it makes me giddy to think
what could be done with the latest tech today.

[0] [https://www.theguardian.com/stage/2011/dec/11/animals-
childr...](https://www.theguardian.com/stage/2011/dec/11/animals-children-
took-streets-review)

------
jeffnappi
I'd love to see Cirque du Soleil leverage this type of technology in a show...

------
raxxorrax
That is pretty impressive. Tried something similar with a DLP projector stolen
from texas instruments (not stolen, but they tend to be picky about selling
them. Probably because these are awesome devices).

Was slow as hell, since I used a raster with multiple images to measure the
topology. The resulting heightmap was awesome, but even with synchronized
camera and projector, I needed a pretty long illumination time per image. So
it would be interesting what camera(s) they used too. I doubt you would need
multiple projectors, because the available ones are extremely fast.

------
ajflores1604
First saw the on the prosthetic knowledge Tumblr account years ago. Really
miss that account. Whoever was behind it did a phenomenal job of curating
incredibly interesting technology developments.

------
joaodelgado
Here's a very cool art project done with this tecnology:
[https://www.youtube.com/watch?v=3Aos1Z2htDU](https://www.youtube.com/watch?v=3Aos1Z2htDU)

------
fortran77
They can project what's behind an object onto an object to make it "invisible"

~~~
mayli
How could you demo that?

~~~
tqkxzugoaupvwqr
Turn the projection on and off to see the cloaking in action?

------
cameron_b
This is so cool. Now I want to take this ( and so many more things like it
that have come up recently ) and show the students at the art school I
attended that projection mapping can be so much more than lining up all the
parallel lines.

------
abledon
There are going to be clubs/bars/venues that will market themselves as "foo"
enabled light spaces for your "bar" apparel.

we really don't need this stuff to overload our already maxed out sense organs
but here we go!

------
devicetray0
The demo video answered my questions, is short and the tech is impressive. Not
entirely sure of the business model though. The non-rigid tracking might be
more useful than the projection -- perhaps a Defense application?

~~~
Frost1x
Skimming the description, they used a structured light approach (active) for
geometry deformation tracking. This is still probably useful for certain
defense applications but an ideal goal is often to use passive tracking
systems. It's impressive either way.

Lots of theater, performing, and visual art uses come to mind.

------
jcims
Reminds me of this high speed tracker video from 2013:

[https://www.youtube.com/watch?v=qn5YQVvW-
hQ](https://www.youtube.com/watch?v=qn5YQVvW-hQ)

Much easier problem to solve at 1k fps but still cool.

------
chrischen
This would go great with the teamLab borderless exhibit, which currently seems
primitive in comparison (it’s an art exhibit where they project images
everywhere).

------
b0rsuk
Advertising, advertising, advertising and dance floors.

~~~
ebcode
> dance floors.

Holy shit.
[https://www.youtube.com/watch?v=XkXrLZmnQ_M](https://www.youtube.com/watch?v=XkXrLZmnQ_M)

------
proc0
Could this be applied to motion capture to remove all the unnecessary gear
actors have to wear? This could revolutionize digital animation.

------
abetusk
What kind of specs would someone need to reproduce this on a Linux system,
say?

Are there any FOSS projects that are doing this?

------
binthere
I can see some big usage for this in medicine. Projecting stuff to help
doctors performing complicated surgeries.

~~~
Frost1x
I was thinking the same thing. Stryker already has a system that has a
tracking device for surgeons that shows pre-OP CT scans around the area and
shifts sliced around depending on the position (also performs a 3D rendering).

Projecting this directly on the surface may be useful but you'd have to be
careful not to skew information surgeons may find useful. Seems great for
training on cadavers and stuff though.

------
mixologic
Sure seems like a lot of science and tech related awesomeness is introduced to
the world on a website from 1998.

------
ChrisMarshallNY
This is very cool. I suspect that it's only a matter of time before all
projectors work like this.

------
peterhil
This could be cool at techno parties.

------
janimo
Something like this should play a role in the future of more sustainable
clothing/fashion.

------
jes5199
if I want to do object tracking for a low-grade version of this kind of thing,
what’s the best off-the-shelf solution right now? Is it ARKit?

------
jedisct1
That could be big for the fashion industry.

~~~
icebraining
I'm working on 3D for clothes design, and we're already exploring VR; seems
like this could be really interesting too, yes.

------
tlarkworthy
Does anyone know what camera they use is?

------
kuu
Simple but effective. Really cool.

------
newshorts
Military camouflage applications?

------
mrlala
That demo looked incredible!

------
johnflan
Advertisers will love this.

------
deedubaya
This is pretty neat.

------
jbverschoor
Very cool!

------
mattio
This is awesome!

------
ericls
wow

------
slim
invisibility cloak is not far away

~~~
jessaustin
Invisibility cloaks need to work from multiple points of view?

