
Video Stabilization on YouTube - Garbage
http://googleresearch.blogspot.in/2012/05/video-stabilization-on-youtube.html
======
ajb
Seems to work well- I just tried it on a tiger cub vid, which I've had for
years and was taken with an el-cheapo digital camera: Shaky version:
<http://youtu.be/g5P9WwHdOSI> Stabilised version <http://youtu.be/wXfbRUk_1Bg>

There is some motion blur (around 24 seconds, when the momma tiger lies down)
which is a bit puzzling once the image has been stabilised. But hey, it's
better than it was before.

~~~
drewyeaton
The motion blur was there in the first place, you just don't notice it because
there's also _motion_. When you remove the motion (shaking, jerking the
camera), you're left with just the blur.

~~~
ajb
Yes, I know. But the viewer won't know that, because they don't know where the
motion was before it was stabilised.

------
apu
As mentioned in the blogpost, the rolling-shutter version of this won the best
paper prize at the International Conference on Computation Photography (ICCP),
which was held last weekend in Seattle. This is a fairly new but very high
quality conference. In many respects, I prefer it to the standard-bearing
vision conferences like CVPR, ICCV, or ECCV -- although of course, ICCP is
more narrowly focused on computational imaging and photography applications.

In their talk, the authors of this work showed many more video results and
they were all quite impressive. In fact, they were good enough to fall into an
"uncanny valley of motion", similar to the "uncanny valley" of faces or humans
[1] that most people are familiar with. I.e., the motion correction was
_almost_ perfect, but just enough off that something felt vaguely surreal
about the results. Nevertheless, it's a nice step forward.

Also, as others have pointed out, this is a fully uncalibrated method --
requiring no knowledge of how the video was captured. If you do have some
knowledge, then you can often exploit it to do better. But the authors
mentioned that most videos uploaded to youtube have either no calibration
information, or if present, it's often incorrect. As such, it made sense for
them to focus on the uncalibrated case.

Finally, I should point out that rolling shutter, standard on most mobile
cameras, is causing all sorts of problems for traditional image and video
analysis algorithms, which often make the assumption, sometimes implicitly,
that the entire frame was captured at a single instance in time. This is not
true anymore, and can lead to gross errors in many methods. Hence the many
recent papers on correcting for, and in some cases exploiting, rolling shutter
effects.

A recent interesting work along these lines from my former lab at Columbia
University is "coded rolling shutter photography: flexible space-time
photography" [2]. This paper takes advantage of the fact that different rows
in an image are seeing the world at slightly different instances in time to do
things like high-speed photography, HDR imaging, etc.

[1] <http://en.wikipedia.org/wiki/Uncanny_valley>

[2] <http://www.cs.columbia.edu/CAVE/projects/crsp/>

~~~
elithrar
> Finally, I should point out that rolling shutter, standard on most mobile
> cameras, is causing all sorts of problems for traditional image and video
> analysis algorithms, which often make the assumption, sometimes implicitly,
> that the entire frame was captured at a single instance in time. This is not
> true anymore, and can lead to gross errors in many methods. Hence the many
> recent papers on correcting for, and in some cases exploiting, rolling
> shutter effects.

This is not just true of mobile phones, but of any current CMOS-sensored
imaging device (most of them on the market). Compact cameras and SLR's
included.

------
lith7
You can try something similar on linux using transcode with these two lines:

    
    
        transcode -J stabilize --mplayer_probe -i $infile
        transcode -J transform --mplayer_probe -i $infile -y xvid4 -o $outfile
    

I found this blog post with more info:
<http://kevin.deldycke.com/tag/transcode/>

~~~
hahainternet
Just as a note for anyone using this, you'll want to read some of the options
as well. The amount of smoothing is hard to get right automatically, and you
will want different values for different effects.

Having said that, my results with this tool have been excellent in the past.

------
vasco
I really hope this keeps being optional because otherwise a lot of the
authentic value of the videos will be lost. Also, the demo they showed looked
like it was algorithmically deteriorated to make the change more noticeable.
There's shaky hands and then there's parkinsons-level shaking which was what
the demo showed...

~~~
delinka
"...algorithmically deteriorated..."

Or the person holding the camera was shaking like mad to demonstrate the
abilities of the algorithm. Artistically speaking, shake removal is also
authenticity removal. But most times, personal videos are shot by people with
no eye for framing and stability. And most times, artistic (and professionals)
videos _are_ shot with an eye toward these things.

I can't foresee a reason not to keep this feature as an option rather than
enforcing it on all uploads.

~~~
leviathan
If you look carefully, the shakes appear too uniform to be a person shaking
the camera. It looks like the camera always "shakes" in a circular motion
around the center of the original camera location.

~~~
agumonkey
Probably a staged setup, inducing a regular shaking pattern, but I think
generating rolling shutter algorithmicly would be a stretch for them, so I
call physical but maybe voluntary shake.

~~~
platz
Or maybe it's due to the 'rolling' shutter causing shear stretching as
mentioned in the article

~~~
agumonkey
But the point is that if they faked shaking algorithmically then the original
signal would be shear-free right ?

------
ck2
That video demo was quite impressive.

I've used video stabilizer filters on virtualdub but I doubt they could fix as
much as was done in that demo.

Another interesting stabilization demo
<http://www.youtube.com/watch?v=_Pr_fpbAok8>

------
nextparadigms
I'd like to see this come in Android so you can automatically shoot and record
videos without the shaking in them. It would be a great selling point in my
opinion.

~~~
objclxt
You should already start to see that working its way out: the iPhone 4S does
video stabilisation, I am sure high-end Android phones do or will start to do
the same.

The algorithm being discussed here is specifically designed for when
information about the camera or environment is _not_ available: there are much
better ways of carrying out digital image stabilisation on the device itself,
such as using the accelerometer data to compensate, or in significantly
advanced cameras (DSLRs, for example) compensating by moving the lens itself.

~~~
baddox
The iPhone 4S algorithm is far less aggressive than YouTube's.

------
hamoid
I wonder when will we get this quality
<http://pages.cs.wisc.edu/~fliu/project/3dstab.htm>

~~~
modeless
Have you tried it? I have and I'd say the quality is pretty close if not the
same. The much bigger problem to solve now is that shaky videos shot in less-
than-perfect lighting contain motion blur, which is extremely hard to remove.
You'll notice that all of these demo videos were conveniently shot outside in
direct sunlight and contain no motion blur at all.

~~~
hamoid
I have only tried the one offered by YouTube, and not recently. I don't know
if they have improved the algorithm in this respect, but for what I've seen in
the past, the filter often creates a very eerie wobbly effect on the video, an
effect that makes it look fake, like being underwater or drunk. It is slightly
visible in the demo video if you observe the borders. This strange effect is
totally absent on the link I posted, which I believe it's on a different level
of quality. But I imagine it's computationally very expensive and can't be
offered to millions of users for free.

~~~
modeless
An eerie wobbly effect is definitely visible to me in the videos from that
paper.

------
Pelayo
They should apply this during movie fighting scenes. Then we might actually
see the fight instead of the blur caused by "exciting cameras".

~~~
ck2
I wish my tv/movie playback devices had a "de-lensflare" filter.

~~~
toemetoch
Are you referring to JJ Abrams recent flirts with anisotropic lenses?

------
rabidsnail
Protip: add ?m=1 to the end of blogger urls to remove the pointless javascript
bloat.

------
adrianwaj
Isn't quality inherently lost because the same video has to be reencoded again
but without the shakes? Also, I just tested on a video and it looked slightly
smudgy. OK, so if I am filming driving down a dirt road or after half a bottle
of Jack Daniel's (or both) then it'd be good, otherwise it does more harm than
good.

~~~
obtu
As I understand it the motion blur is a product of lossy compression (CCD has
very short pixel-local exposure times; the shearing the article refers to
appears when sweeping the whole image); which means that yes, stabilisation
algorithms would work best with source data that hasn't been compressed using
a perceptual model of motion blur.

------
Too
What's the next step? Include accelerometer data from the camera, synced with
the video, to use as support for stabilizing algorithms.

~~~
est
that's a really good idea actually. Maybe the video container format could add
that metadata support.

------
tambourine_man
Submitting video to a streaming service makes your content look _better_.

Shows how far we are in this whole cloud era.

------
CubicleNinjas
Automatically?

 _sigh_

