
Magical, 3D-Warping Techniques Steadies Your Videos - teej
http://createdigitalmotion.com/2009/06/30/magical-3d-warping-techniques-steadies-your-videos/
======
gojomo
Stabilization into a smaller rectangle discards material around the edges of
the original larger video.

So in a sense, this is yet another case of 'more data' (the oversized source
video), plus software/CPU batch analysis, replacing the specialized equipment
and expertise that used to be required for a 'stable' shot.

Taken to its logical extreme: might future casual-use 'cameras' be compact
omnidirectional arrays of high-resolution, almost-always-on photo sensors --
from which idealized photos and videos can be reassembled later, by post-
processing and editing at leisure?

~~~
mhb
Sure. But noise imposes a fundamental limit on resolution.

~~~
iamwil
more sensitive equipment developed in the future would lower the noise floor,
to the fundamental limit of resolution.

~~~
mhb
Will that equipment enable us to know exactly where a photon from the subject
has hit the detector?

------
rufo
Is it just me, or is the warping sort of noticeable?

It feels like there's some sort of other-worldly quality... I think it's parts
of the mesh moving around out-of-sync with other, but it's really hard to get
a sense as it's one of those things where it's the image as a whole, not any
one point you can look at.

Honestly, it makes me slightly nauseous - I'm not sure I could deal with it
for a significant period of time.

~~~
jerf
I think a lot of the otherworldly quality is simply that they've put the
camera on a line. Not "sort of a line", not "a steadi-cam pretty-close-to-
line", but a _line_. Very few shots ever are actually on a rigid line,
especially at a single rigid speed. Even track shots will do some panning, or
travel at different speeds throughout the shot.

This is easy to fix, of course. For demonstration purposes, it really drives
the point home that they have _stabilized_ this picture, no ifs ands or buts.
For real use, I'd probably stick with the lowpass filter. Also, in real use,
you'd be able to configure the algorithm selectively; line here, lowpass
there, etc.

------
ajb
This reminds me of another thing I've never got round to writing - a simple
program that just presents your video as a picture within a larger area, which
moves around to compensate for your hand wobble. All the solutions I've seen
either crop or warp, trying to hide the wobble, but losing or distorting part
of the image. (Admittedly I haven't checked this one out yet, because I'm at
work. Maybe there really is no distortion). I suspect that the simple solution
would be just as good, if you didn't mind letting people know that your hand
wobbled, which most people don't.

------
GR8K
They make this sound like new but this 3D tracking technology has been
available for years & used on feature films all the time for stabilizing &
placing CG elements in to live-action film. These are a few software companies
that sell this technology & some of the vfx companies have their own in-house
developed software:

Syntheyes- <http://www.ssontech.com/>

Boujou 3D- <http://www.2d3.com/>

PFHoe- <http://www.pfhoe.com/>

~~~
ktharavaad
I think you missed the point, their key innovation is to use a content
preserving warp on a frame-by-frame basis to recreate almost ANY motion path
they like ( provided that the path is close enough to the original path of
course ).

The software you listed are simply a part of the system which they developed
use, the tracking/scene analysis part of it. And you are right, that part of
the software is not overly difficult to write, in fact you can throw together
a Harris Corner detector + RANSAC to find correlation between the points and
it'll do quite ok. In their case, they decided to use the Voodoo camera
tracking system available here:

<http://www.digilab.uni-hannover.de/docs/manual.html>

which is comparable to the software you have listed

What I'm most interested in is the speed of processing their warps. From my
experience of playing with ITK/VTK, generating non-rigid warps using b-spline,
MFFD or any of the current algorithms takes a LONG time if you take the image
content into account. I need to read the paper more carefully to find out.

------
TrevorJ
This looks like novel approach and a good one. It is worth pointing out that
it looks like it will only work for tracking shots and not static shots that
have some shake. I'm not sure if it will work for nodal pans or not either.

Either way, this could be a nice tool for independent filmmakers who don't
have access to Steadicam gear. Big budget films don't rely on post processing
to fix this sort of thing unless they have no other choice becasue of the
constraints of the shot.

------
anigbrowl
The writer only mention's Adobe's involvement in passing. Personally, I think
it's a lot better than Final Cut's motion smoothing filter (although the
latter has the advantage of being available now). Bad news for Apple. It's a
subjective impression, but I feel that FCP is steadily losing ground to adobe
in this market.

------
kingkawn
It's an odd effect that the output, despite playing in real time, has a slow-
motion quality to it.

------
joshu
I'm always amazed by this stuff, but I wish more of it turned into available
implementations...

~~~
timdorr
Me too, considering how shaky my 5D2 video usually comes out. I'd be happy
with a cheap, handheld stabilization system and some decent software. Any
recommendations?

~~~
chriskelley
Jonny Lee, the brain behind many of the wii-mote hacks, has a $14 "Poor Man's
Steadicam" that he came up with:

<http://steadycam.org/>

------
liuliu
I think the key of this algorithm is that it utilizes the standard SFM
algorithm. If structure from motion is so mature, I can expect the 3D movie
without 3D camera in very nearly future.

------
jrbedard
Nice, that tech is getting close to render steadicams obsolete.

~~~
greendestiny
I doubt it will for professionals, at least in lots of shots, the distortions
would be considered fairly unacceptable in most movies. Those shots are
dominated by static nicely edged structures, with the dynamic parts making up
only a small part of the frame.

Still going to be a great tool and looks like a small enough iteration on
current stabilisation that it will actually be implemented.

~~~
anigbrowl
It's quite reasonable to assume that with further engineering and sufficient
computer power, many of those distortions will be ironed out. The cost of
digital post-processing is in constant decline, whereas the cost of shooting
with a steadicam/ dolly/ crane on set is more or less constant. All of these
require expensive, bulky, and highly visible equipment for one thing; for
another, any time you take the camera off a tripod the cost goes up because
moving the camera adds 3 new vector variables to the shot calculus, requiring
more takes for a good result.

I see this being an instant hit with low-budget shooters who are prepared to
lavish time on getting it right and are more concerned with energy than
perfect fidelity. Later there will probably be rough-quality realtime preview
on set followed by near-perfect post rendering.

