
Trillion-frame-per-second video - xtacy
http://web.mit.edu/newsoffice/2011/trillion-fps-camera-1213.html
======
nkurz
Interesting story, but flagged as blogspam.

Could an admin please change the link to a primary source like this one:
[http://web.mit.edu/newsoffice/2011/trillion-fps-
camera-1213....](http://web.mit.edu/newsoffice/2011/trillion-fps-
camera-1213.html)

Or even this one: [http://www.nytimes.com/2011/12/13/science/speed-of-light-
lin...](http://www.nytimes.com/2011/12/13/science/speed-of-light-lingers-in-
face-of-mit-media-lab-camera.html)

[Edit]

gmaslov's comment provides an even better link, although not as 'newsy':
<http://web.media.mit.edu/~raskar/trillionfps/>

------
jonah
My friend Steve Silverman worked on a system which they commercialized back in
2006 [1] that unlike the MIT one captures full-frames so you can get 3D at
video frame rates.

It uses a 5ns long pulsed laser "photon torpedos" at 30hz to illuminate the
scene and then captures the at a much higher sampling rate where the MIT
system just scans a line at a time. So, unlike the MIT one, it's a small,
hand-hold-able, system that captures full motion. [2]

You get full scene 3D without the drawbacks of scanning.

They flew one of their cameras on the last Discovery Mission.

[1] <http://asc3D.com>

[2] <http://www.youtube.com/watch?v=3L91F9o600E>

<http://video.google.com/videoplay?docid=-3656494784112768834>

I had the chance to play with it a couple years ago - quite amazing.

~~~
jonah
Parallel vs. Serial.

The Google Tech Talk goes into a little bit of detail about how they capture a
stack of frames (slices in Z) into a buffer right behind the sensor and then
dump that out for each snapshot.

------
nix
This works by sampling a static scene a very large number of times with a
laser flash and a "streak tube" camera that records a picosecond-long movie of
the light arriving at a single scanline.

A normal video camera records a frame at a time, this one records a scanline-
sized movie at a time. The raw data is noisy but the scene is static so they
can sample the same line over many flashes.

After a few minutes of scanning they have a trillion fps video where you can
see a wavefront propagate at the speed of light. Amazing.

------
jcr
The relevant video from MIT Media Lab is here:

<http://www.youtube.com/watch?v=EtsXgODHMWk>

> MIT researchers have created a new imaging system that can acquire visual
> data at a rate of one trillion exposures per second. That's fast enough to
> produce a slow-motion video of light traveling through objects.

> <http://www.media.mit.edu/~raskar/trillionfps/>

------
nitrogen
Ever since I started playing with PWM control of LEDs for lighting, I've
wanted a visualization of the spherical pulses of light traveling through a
room (inspired by the moving-mirror cameras used to analyze high-speed
explosions). I calculated the PWM frequency I would need to reach before the
light arriving from the far walls of a room could be distinguished from the
original pulse. Thank-you, MIT, for actually making this happen, on an even
cooler scale.

I wonder if the in-room impulse response of an LED light source could be
exploited for ultra-high-bandwidth data transmission through open air.

~~~
jonah
Yep, Fraunhofer is doing it.

[http://www.kurzweilai.net/transmitting-high-speed-data-
via-l...](http://www.kurzweilai.net/transmitting-high-speed-data-via-led-room-
lights)

[http://www.eurekalert.org/pub_releases/2011-08/f-dat080211.p...](http://www.eurekalert.org/pub_releases/2011-08/f-dat080211.php)

------
gmaslov
The actual website for the project is here
<http://web.media.mit.edu/~raskar/trillionfps/> , and has a great FAQ section
and more videos.

------
click170
I heard the word virtual used to describe the camera, and couldn't figure out
if they were, for sure, talking about taking pictures of a single photon in
real life -- ergo not simulated. I suspect I misunderstood something.

I'm skeptical because.. how would you see a photon? Unless photons themselves
give off light as they travel, but that would mean photons emit photons...

~~~
jessriedel
Photons do not emit photons. In fact, photons don't interact with each other
at all. But if you send a very short pulse of light (visualized as the thin
spherical shell in the video) at an object, the photons in that pulse will hit
the object at different times depending on how far away the scattering surface
is from the source of the light. After they reflect from the surface, these
photons will also take different amounts of time to travel to the camera (your
eye). If you can measure the time the photons enter the camera very
accurately, you'll see different parts of the scene light up at different
times.

This would be very close to watching the light pulse travel across the
scene... _except_ for that second effect: that the parts of the scene are
different distances from the camera.

~~~
16BitTons
>In fact, photons don't interact with each other at all.

They can. It's a non-classical concequence of Quantum Electrodynamics. If you
are interested in the cross-sections, check out Berestetskii et al., 1982.

------
gregburek
More informative link: [http://web.mit.edu/newsoffice/2011/trillion-fps-
camera-1213....](http://web.mit.edu/newsoffice/2011/trillion-fps-
camera-1213.html)

Imaging Systems Applications Paper "Picosecond Camera for Time-of-Flight
Imaging": <http://www.opticsinfobase.org/abstract.cfm?URI=IS-2011-IMB4>

ACM paper "Slow art with a trillion frames per second camera":
<http://dl.acm.org/citation.cfm?doid=2037715.2037730>

~~~
_delirium
Also, some of the math behind reconstructing the 4d light-propagation is in
this tech report from some of their collaborators:
<http://users.soe.ucsc.edu/~amsmith/papers/ucsc-soe-08-26.pdf>

------
jbri
For anyone wondering how the "capturing light in motion" works - the
researchers use a very short pulse of light, so when playing back the footage
in slow motion, you can see the light pulse moving through the scene.

They're not directly observing the photons in motion, they're observing what
parts of the scene they're scattering off at a given point.

------
brain5ide
A (virtual)photon race with a photo finish. It would be totally amazing to see
two different visible light waves (red vs. blue) crossing a glass prism and
how their velocity difference looks. Also, wondering if it would be possible
to capture light traveling through fiber optic cable (total reflection).

------
JonnieCache
They can't be capturing photons in motion, or capturing anything "moving at
the speed of light." That doesn't make any sense. According to relativity,
photons don't actually "move" at all. (As I understand it.)

~~~
teraflop
Photons do in fact "move" by any reasonable definitions of the word. In fact,
unlike massive objects, photons are moving in all inertial reference frames.
They travel along "light-like" paths through spacetime:

<http://en.wikipedia.org/wiki/Spacetime#Light-like_interval>

You might be thinking of the fact that no proper time elapses along such a
path, which is true but not relevant from an observer's perspective.

~~~
theon144
Wait, shouldn't no proper time pass from the photon's perspective, not from
the observer's perspective?

------
nivertech
We developed a middleware, which helps to capture and process all these image
frames in Realtime running on blade clusters and GPUs. Think about it as
Hadoop for Realtime Image Processing. Our original application was
Semiconductor Inspection machines with large arrays of camera sensors.

For more info contact:

<http://CLASTR.com>

Email: info AT CLASTR DOT com

------
mike_ivanov
Interesting. The process is basically like a convolution of a flat wavefront
step function over a 3D scene.

Now imagine this: instead of registering images, camera _emits_ them in the
reverse sequence, effectively making the surrounding environment send
concentrated coherent impulses to the point where the laser initially was.

Pew-pew.

~~~
burgerbrain
The problem with that idea of course is that the camera is only catching a
very small amount of the light scattered by the scene.

------
brudgers
_> "Because all of our pulses look the same"_

Don't misunderstand me, this is really impressive and potentially has some
important applications...

...but, because the final product showing the plastic bottle is a series of
similar scenes, the video seems more akin to a cell or stop motion animation
rather than to what is typically considered high speed photography which
captures a single event and expands time rather than compressing it. Ten
seconds of traditional high speed film contains images captured in a fraction
of a second. In this video, ten seconds was captured over the course of many
minutes.

In other words, there is a significant degree of editorial decision making
regarding the manner in which events are depicted - even if that decision
making is now handled by software.

But cool nonetheless.

------
planckscnst
I would love to see two parallel mirrors in a scene. I would expect the
objects between the mirrors to stay illuminated longer than the other objects
and that they would fade out gradually.

------
razzaj
"you can see photons moving through space..." I think of this statement and
cant help but think it is fundamentally flawed. One cant see the photon moving
through space. You might be able to see an electron moving through space
because e can emit a photon. But photons cannot emit photons while moving. In
essence one only sees the photon when it hits the detector. Physics majors
feel free to correct if i am mistaken.

------
tlb
I think they're showing less than 1 trillion fps. Light travels 0.3 mm in a
trillionth of a second, so when they play it at 30 fps it should be 9 mm /
second. But it passes an apple in about 2 seconds, suggesting more like 30 mm
/ second.

------
vinkelhake
This reminds me of Searle's relativistic ray tracer:
<http://www.anu.edu.au/Physics/Searle>

In particular, check out "Flash" example in the downloads section.

------
ibuildthings
A more elaborate talk about this work by one of the authors appears at:
<http://www.youtube.com/watch?v=aKu20y1f_RU>

------
sharmajai
They should use this to demystify the double-slit experiment.

~~~
aiscott
That would be a lot more interesting to see than an apple or bottle of water.

------
ORioN63
Does this even make sense?

I am not doubting it,that they've made a superb machine, but which photons
does the camera catch to see photons that travel parallel to that same camera?

~~~
fletchowns
Yes, it makes sense. What do you mean which photons? The photons that the
camera picks up are the photons from laser pulse, after they have reflected
off the scene. Pause it at 2:20 and look at the shape of the light as it
moves.

Imagine that you are in super slow-mo mode for a second. You hold up your hand
and shine a flashlight on it. Since everything is in super slow-mo, you can
watch as the photons strike your hand first, and it lights up. The photons
that don't strike your hand continue on towards the wall next to you, and then
some time after your hand lit up, the wall lights up, with a shadow of your
hand.

~~~
JabavuAdams
I think the parent's point was that we can't see or image anything until those
photons enter our sensor.

> you can watch as the photons strike your hand first, and it lights up. The
> photons that don't strike your hand continue on towards the wall next to

This analogy doesn't work. In order to watch, some photons have to enter our
eyes, but they haven't got there yet, as they're just now interacting with our
hand, etc. I.e until sufficient photons enter our eyes (sensors), there's
nothing to watch.

This points to the fact that the technique is not imaging the whole scene at
once, but doing some kind of reconstruction on an assumed static scene.

------
wayzie
Are the frames viewed consecutively or are they combined?

~~~
gregburek
The MIT News Office article has more details than the PR video and a link to
the paper: [http://web.mit.edu/newsoffice/2011/trillion-fps-
camera-1213....](http://web.mit.edu/newsoffice/2011/trillion-fps-
camera-1213.html)

The actual imaging device is just a long line of photosensors. The camera
aperture uses a varying electric field to deflect photons that arrive later to
sensors further down the line, producing an image in effective 2D - 1D of
space and 1D of time. By repeating the scene and slowly scanning the camera's
mirror, a composite video is built that shows diffusion of a picosecond laser
pulse.

~~~
archgoon
> The camera aperture uses a varying electric field to deflect photons

 _blink_

They're using photon-photon scattering? Wow. I thought in order to pull that
off you needed very high powered lasers.

Edit: "But while both systems use ultrashort bursts of laser light"

Depending on how they define "ultrashort" this might be the key.

~~~
gregburek
The press release (edit: and I) skipped a step: "A portion of the laser beam
is split off with a glass plate and directed to a photodetector that provides
the syn- chronization signal for the camera. The camera images the incoming
light on a slit and this slit on a photocathode where photons excite
electrons. These electrons are accelerated in a vacuum tube and deflected by
an electric field perpendicular to their direction of motion and perpendicular
to the direction of the slit. The signal generating this field is obtained by
filtering and amplifying the synchronization signal from the laser. The image
is thus spread out over time in a “streak” that hits a micro channel plate at
the far end of the vacuum tube that further amplifies the electrons and
directs them to a phosphor screen where they are converted back to photons.
The screen is than imaged on a conventional low noise CCD camera. The complete
time interval captured in this way spans about 1 nanosecond." [Picosecond
Camera for Time-of-Flight Imaging]

So a photo cathode generates electrons which are deflected.

~~~
archgoon
Thank you. That makes a lot more sense.

------
daimyoyo
This will be a very interesting technology once it's perfected. It seems to me
to be like the led was when it was first invented. A solution in search of a
problem. And that can create wonderful innovation.

~~~
einhverfr
In particular I wonder what applications the TSA might find for this
technology......

~~~
jonah
A similar tech - with uses:

[http://www.advancedscientificconcepts.com/technology/applica...](http://www.advancedscientificconcepts.com/technology/applications.html)

------
Sahebi
wonderful camera

------
rorrr
It sounds like they are taking different phases of the light spreading, and it
just looks like the light is traveling.

While cool, it's not a true 1 trillion fps camera.

Here's true 1 million fps footage:

<http://www.youtube.com/watch?v=QfDoQwIAaXg>

~~~
Groxx
I think you described it correctly, but in all honesty, what's the difference?
They're able to capture, very precisely, a trillionth of a second worth of a
line of reflected light. They then repeat it until they get a video. And,
since a line isn't very interesting, they repeat it in a bunch of lines
parallel to each other to get a rectangular video.

Sounds a lot like a CRT. Or some of the oldest video capturing techniques
(scanning line by line into a photo diode, and then using that signal to vary
light on the output following the same scanning pattern).

The only thing that this camera is missing from other high-speed cameras is
1000 more lines in parallel, and a faster cool-down between frames. And many
high-speed cameras in the past got around the cool-down by using multiple
cameras, each taking a different slice of the action, and stitching them back
together afterwards, but they're still considered high-speed cameras.

This could be 100% identical if they built a million of them and took a video
in one shot, but it isn't currently feasible or cost-effective. And, since
their 'bullet' is non-destructive, there's no reason to not simply repeat it
with a cheaper technique.

~~~
Tichy
Well before reading the article I was struggling to imagine how to film light
- after all either a photon hits the sensor or it doesn't. How would you
photograph a photon in mid-flight? You can't, because if it is in mid-flight,
it is obviously not on your sensor. So this (presumably correct) explanation
is a relief for me.

~~~
Groxx
It did sound like that from the description and some of the earlier footage in
the video, but no, they're still capturing reflected light :)

