
Hyperlapse, Instagram's new video stabilization app - bpierre
http://www.wired.com/2014/08/hyperlapse-instagrams-new-app-is-like-a-15000-video-setup-in-your-hand
======
davidu
It's weird that they called it Hyperlapse two weeks after Microsoft Research
published a paper on the exact same topic, with the exact same name:

PDF here: [https://research.microsoft.com/en-
us/um/redmond/projects/hyp...](https://research.microsoft.com/en-
us/um/redmond/projects/hyperlapse/paper/hyperlapse.pdf)

Examples here: [http://research.microsoft.com/en-
us/um/redmond/projects/hype...](http://research.microsoft.com/en-
us/um/redmond/projects/hyperlapse/)

It's even more weird that the WIRED writer didn't mention this. It was major
news all over the place two weeks ago. Good PR folks at Instagram / FB.

~~~
higherpurpose
I'm confused. So is this the same technology/project, or are they completely
different?

~~~
stdgy
Very different.

Microsoft Research was manually computing a motion map using heavy image-
processing techniques that they then used to decide which frames to use and
how to align them. Their work can do things like throw away a series of
'noisy' frames that wouldn't fit the overall scene motion. For instance, if
you had a head mounted camera and were riding around on your bike, it could
throw away the moments where you look both ways before crossing the street.

Instagram's technology utilizes extra sensor data to make their frame
corrections. As far as I can tell from the brief article, the technology
doesn't do much besides stabilization. That's not to belittle it, as it's a
really neat application demonstrating the benefits of mobile sensor
integration. And you might be able to create a similar motion map using the
gyroscope data that the folks at Microsoft create through their image
processing techniques!

~~~
lyndonh
A "motion map" ? I think you'd better read the paper again. They compute a
full 3D reconstruction. Much much more computationally expensive. You would
not "be able to create a similar motion map using the gyroscope data".

~~~
stdgy
Oops, good point! I forgot that they calculated the geometry and stitched the
images to create novel frames.

The gyroscope data most certainly would not allow one to do the same.

------
mikeyk
Mike from IG here. Here's the direct App Store link while it propagates:
[https://itunes.apple.com/app/id740146917](https://itunes.apple.com/app/id740146917)

~~~
heyheyhey
I was wondering why you didn't link an Android version until I read the
article. :(

Apps coming in iPhone first is by far my most frustrating experience with
Android.

~~~
sp332
The article was pretty hand-wavy about the API issues. Why aren't the Android
gyroscope or camera APIs good enough?

~~~
Someone1234
They're getting better. But unfortunately due to the way Android phones
receive updates many phones have different, older, API levels which don't
support many of the features.

So while they could likely make and release an Android app, expect only the
flagship phones from the last 12-24 months to be supported, nothing a day
older (and even a lot of phones within that window still lack the latest API
updates).

~~~
sp332
I'm pretty sure it's not a fragmentation issue. They said the APIs required
changes.

~~~
swamp40
Phone Mfg's will sometime release manufacturer-specific API's for Android that
will only work on _their_ phones.

I've always been disappointed at how/why Google allows fragmentation.

It certainly seems to me that they have the power to force a _reunification_
in Android (and that if they did, the world would cheer).

Why not tell Samsung that Google must determine the update schedule, not
Samsung? And if Samsung adds non-standard hardware, they must provide working
updated code by _this_ schedule.

And if Samsung doesn't like it, they are welcome to move everything over to
their Tizen OS.

Even if Google cannot fix the past, why not do this going forward?

~~~
sp332
Google doesn't control Android after they release it. It's open software
(which is why Samsung gets to "skin" it the way they do) and Samsung could add
anything they want or even rip bits out. It's not Google's decision.

Edit: this is a lot of downvotes for no comments. What's wrong with my
comment?

~~~
nindalf
The downvotes might be because you're slightly mistaken about the level of
control that Google possesses. While the code base of Android itself is open
source, that means little because Google apps such as the Gmail, Maps and the
all important Play Store are not part of AOSP code base. If a manufacturer
wants to include those (and lets face it, they have to) they need to agree to
several onerous terms and conditions that Google stipulates.

An example of such a condition is that if a manufacturer makes any phone that
includes Google apps, they must not manufacture a phone that runs without it.
This is why Amazon had such a hard time finding a manufacturer for their Fire
tablet/phone. As for skinning, Samsung can only do that because Google
explicitly allows it.

Lastly, I feel that someone should have pointed this out instead of downvoting
you.

------
netcan
I have a category in the back of my mind labeled 'Stuff Apple should do to
prove that they can do it without Jobs." Top of that list is an iOS laptop.
But just below that is getting back to their tradition of being a popular
easier to use option for professional and prosumer media/art.

Stuff like this should be coming from Apple. For example, it would be awesome
if they created prosumer software for filming with a wireless multi-camera
setup and edit it all live on a macbook. would love it if upstart young-Turks
like web shows could get closer to the production quality of TV talk shows.

Something that makes the right people go wow.

~~~
ericras
Apple seems to not want to get into the prosumer market because it is a tiny
fraction of the size of the general iPhone/iPad/Beats consumer market. The
buzz about future Apple products seems to be about things that could achieve
that sort of scale. (TV, watch, something video game related, etc)

~~~
dublinben
Their prosumer computers essentially only exist so that developers can enrich
their mobile platform. Growth is essentially flat on the Mac line, whereas
iPhone sales have grown steadily since launch.

------
nkozyra
Getting a bit of hyperbole out of this (shocking, I know).

Stabilization is not something that costs $15k, at least at this quality. Yes,
it's interesting on a technical level (both with Instagram and MS), but there
are low-tech solutions that handle this aptly.

So in essence we're talking about a video accelerator (and presumably
decelerator)? This doesn't exactly wow me, but then again Instagram was
founded on applying filters to photos, so I should recognize the value of hype
(and subsequent audience size).

~~~
theworst
I think the $15k references a gyro stabilized camera rig. The stabilization is
the hard/novel/neat part, not the playback speed.

Playback speed is the UI feature people notice, but it wouldn't be interesting
if the image weren't stabilized.

~~~
nkozyra
Um, that's what I was saying. Stabilization (at least at this level) is still
not a $15,000 investment. Prosumer ones start in the $500 range.

Software stabilization obviously also supplants hardware cost, which makes
even that # moot. Then there's the acceleration, which can be done by low-
level video prod software already.

I'm seriously not trying to just poo-poo this, it's just very underwhelming as
a total package.

~~~
anentropic
but with this you don't have to fit your phone into a rig, you can just keep
it in your pocket

~~~
nkozyra
Sure, but this is sort of like saying adaptive blurring replaces a $2,000 SLR
lens.

------
DEinspanjer
I think that the MS Research project probably has a few advantages over this
although I've only looked at Instagram's example videos on Vimeo so far.

The main advantage I see for MS Research is that they enable the full dropping
of multiple frames to remove large camera motions such as having a head
mounted camera and briefly looking from side to side.

In the Instagram examples, the camera stays on a fixed subject and their
Hyperlapse algorithm reduces the shake. I suspect that if there were large
camera motions, they would be translated into the final product which could
detract a lot from the appeal of that video.

~~~
colinbartlett
For me the biggest advantage of the Microsoft project is that it has the
potential to be stand-alone software.

I'm not interested in using something that's a part of Facebook's ecosystem or
sharing it with my friends on Facebook.

~~~
mikeyk
Hi Colin, all Hyperlapse videos get saved out to your Camera Roll, and no FB
or IG account is needed to use the app (there are sharing options for those
two in the app, but they aren't a requirement to using it)

~~~
Kayou
Wow, so does all the processing happen on the phone itself? That's impressive!
I thought it was sent on a server to be processed.

------
giovannibajo1
For the curious, this is a video rendered with my iPhone 4S, and the road was
very very bumpy. This is the quality you get with an old phone and with zero
setup / art direction.

[https://www.dropbox.com/s/i2w18xh31aixmhv/Video%2026-08-14%2...](https://www.dropbox.com/s/i2w18xh31aixmhv/Video%2026-08-14%2019%2000%2046.mov?dl=0)

~~~
dntrkv
[https://www.youtube.com/watch?v=Bt9zSfinwFA](https://www.youtube.com/watch?v=Bt9zSfinwFA)

~~~
giovannibajo1
Lol sorry i totally didn't think of it while shooting

------
chrischen
"What was once only possible with a Steadicam or a $15,000 tracking rig is now
possible on your iPhone, for free."

This statement is undeniably false.

------
jrlocke
Cool project, but if you want the same cinematic smoothing without the time
lapse effect, I highly recommend Steady[1].

1\.
[https://news.ycombinator.com/item?id=8048413](https://news.ycombinator.com/item?id=8048413)

Edit: Hyperlapse can do this too. Thank you Wang.

~~~
wangg
Hyperlapse can be used to only steady but not accelerate videos as well.

~~~
pwnna
I don't see any example video on 1x though. Will it be as good as their
accelerated ones?

------
omarish
It looks like the Midnox team (YC W12) has been busy at Facebook!
[http://midlux.us/](http://midlux.us/). Great work.

------
dpflan
Hm, Instagram a little quicker on the draw than Microsoft?
[http://research.microsoft.com/en-
us/um/redmond/projects/hype...](http://research.microsoft.com/en-
us/um/redmond/projects/hyperlapse/)

~~~
simonw
The approaches are completely different.

Microsoft:

"Our algorithm first reconstructs the 3D input camera path as well as dense,
per-frame proxy geometries. We then optimize a novel camera path for the
output video (shown in red) that is smooth and passes near the input cameras
while ensuring that the virtual camera looks in directions that can be
rendered well from the input."

Instagram:

"... Smartphones didn’t have nearly enough power to replicate video-editing
software, but they did have built-in gyroscopes. On a smartphone, instead of
using power-hungry algorithms to model the camera’s movement, he could measure
it directly. And he could funnel those measurements through a simpler
algorithm that could map one frame to the next, giving the illusion that the
camera was being held steady"

~~~
phirschybar
Wow. I would never have guessed that this would be possible to do by analyzing
the gyroscope data. It must be sensitive enough to detect small / tiny jerks
in your phone as those seemingly tiny jerks can often translate into enormous
motion in your camera lens especially if a subject is far away.

~~~
StavrosK
Well, you can do stabilization as well for those, I guess.

------
djfumberger
Great app.

On a side not it's kind of depressing / scary / crazy that it's now just
common place to release these apps for free.

Indie development if not suffocated already on the App Store has little hope
competing going forward.

------
nkg
I just tried it. The result is really better than an un-stabilized video. But
I found it a bit blurry.

I can't wait until someone copies it on Android.

------
dharma1
Nice!

Non-timelapse but related - I wonder how long until mobile phone manufacturers
start using in something like this
[https://www.youtube.com/watch?v=VphxfCXu984](https://www.youtube.com/watch?v=VphxfCXu984)

Post stabilisation is good but it's so much better to get it right in-camera

~~~
smackfu
Nokia's higher end phones include optical image stabilization. It adds
thickness which is probably why iPhones don't have it yet.

~~~
dharma1
Ah of course, Nokia, should have guessed. Interesting video on their
stabilisation technology - on the 925 it doesn't seem to add that much
thickness, that one is only 8.8mm thick, vs 7.7mm for iPhone 5. They should
have licensed their camera tech to all manufacturers. Wonder how much of it
will pop up in iPhone 6 now that they snagged one of the main PureView
engineers.

[http://conversations.nokia.com/2013/09/25/how-nokia-lumia-
op...](http://conversations.nokia.com/2013/09/25/how-nokia-lumia-optical-
image-stabilisation-is-revolutionising-imaging/)

I think the earlier 1020 etc were thick because of huge sensor size.

I have OIS dSLR lenses but none of them are quite as good as that Sony's
Optical Steady Shot linked above - it's almost as good as some brushless
gimbals I'm using on drones, which are rock steady. Sony claim it's 13 times
more effective than their previous stabilisation.

------
thisjepisje
The tech demo from a few weeks ago was much more impressive.

~~~
ceejayoz
The ability for the general public to _use_ this _today_ without _weeks_ of
rendering time is pretty impressive in my book.

~~~
bobbyi_settv
"soon, but that will require changes to the camera and gyroscope APIs on
Android phones" is not nearly the same as "can use this today".

~~~
ceejayoz
You probably can't use it on a flip phone or Nintendo DS, either, but it _is_
available for use to the general public.

------
vardump
Interesting it can be done real time on the phone (I assume).

Personally I've been using free video stabilizer Deshaker for 10 years with
similar (or better) results. The downside is two-pass rendering. Upside is
that it is fairly configurable and can stabilize existing videos. You can get
ok results with fairly terrible source video.

[http://www.guthspot.se/video/deshaker.htm](http://www.guthspot.se/video/deshaker.htm)

------
hagope
I was excited to download and try it, but honestly it didn't seem like it was
stabilizing the frame at all...the camera shaking was still in the video...

------
duncans
So it's basically a very simplified Luma Camera (acquired by Instagram a year
ago) with a speed slider?

[http://vimeo.com/lumacam](http://vimeo.com/lumacam)

------
seeingfurther
It's available in the app store. You find it by searching Instagram then
clicking 'related'. I couldn't find it with normal search.

~~~
giarc
[https://itunes.apple.com/app/id740146917](https://itunes.apple.com/app/id740146917)

------
seivan
I might be wrong here but I thought 'Hyperlapse' (functionality) was builtin
on the iOS 8 Camera app?

Am I missing something here?

~~~
mattkirman
iOS 8 has timelapse, implemented as a series of photos taken a few seconds
apart and then stitched together. This appears to be a video stream that is
then smoothed and then (optionally) sped up.

------
qzervaas
Looks like it only has native share options for Instagram and Facebook. Seems
a bit petty not to offer Twitter. How about let me decide where I want to
share?

From an implementation point of view, this app doesn't use
UIActivityViewController, but instead one they've rolled themselves.

They could in fact have added Instagram to the built-in share view controller
but decided not to. This also takes away options of Twitter, SMS, email.

~~~
corobo
It saves to your video roll, use any number of apps to upload from your video
roll. The decision is yours!

------
kazinator
I would be surprised if this couldn't be done with FFMpeg.

Google for "ffmpeg" "image stabilization".

------
prawn
Wonder if they considered charging for this? Looks impressive enough that
people would pay for a quality app.

------
ghshephard
This is one of the first major reasons why the 128 GB iPhone will be
important.

------
mark2456
It's 2014, can we get Android and iOS versions at the same time? Starting to
feel tired of all this "Android soon".

~~~
cwyers
What does the year have to do with it? 2014 isn't the year Google fixed
Android fragmentation. It isn't the year it managed to get OEMs and carriers
to start shipping timely OS updates. It isn't the year Android users started
actually buying stuff. I'm an Android user, so I'm just as frustrated by it as
you, but it's a problem inherent to Android, not to the developers.

~~~
simonh
I think a significant part of this problem is that it's hard to know when
choosing an Android device exactly what tradeoffs you're making. How many
reviews include information about the quality and calibration of the
gyroscope, motion sensor and magnetometer systems? Other than perhaps some
test shots, how much do you really know about the camera quality? Are all the
relevant APIs implemented, and how good a job are the drivers doing?

It's great the Android offers so much choice in terms of features and price,
but the upshot is that you're not buying a device with an android sized slice
of market share from the point of view of app support, for many purposes
you're buying a device with a that-device sized slice of market share.

In the old PC days the exact spec of a machine didn't usually matter much.
However for mobile devices, the plethora of advanced sensors they come with
and the complexity of software capabilities that take advantage of them has
made fragmentation a much bigger issue, but really only if you're interested
in the most advanced features.

For most phone users it doesn't make that much difference, but if you're
interested in cutting edge features the quality of hardware and software
integration and application support has become critical. Samsung almost has
enough critical mass to establish a stable platform for advanced Android
applications, but the problem is it just doesn't have the vision and
discipline to do so by actually establishing a stable baseline for the
features and software in it's phones.

