
Lytro announces light field VR video camera - ryandamm
http://uploadvr.com/lytro-immerge-vr-light-field-video-camera/
======
soylentcola
Site won't load for me but I read about this elsewhere. While this first gen
product will surely be expensive and somewhat limited (capturing a cubic meter
of light field) it's incredibly interesting to me.

I've been messing with VR stuff for the past year or so and I still think that
the "killer app" in 5-10 years won't be fancier video games but more like
immersive "movies" or VR telepresence. We're at that stage of the hype curve
where expectations are high as gen-one products come out but while these will
surely be underwhelming to many early users, I think that once depth cameras
and light field cameras are developed and improved (along with the ergonomics
of head-mounted displays and the speed of processors) we'll start to see more
in the way of non-game applications.

Just from personal experience, playing video games in VR is cool and all but
when people can watch a movie from the perspective of someone inside the scene
or meet up with people in a live-scanned remote environment, I think they'll
be hooked.

~~~
pythondz
> Just from personal experience, playing video games in VR is cool and all but
> when people can watch a movie from the perspective of someone inside the
> scene or meet up with people in a live-scanned remote environment, I think
> they'll be hooked.

Exactly, I definitely want to feel this in the first fps movie.

Hot pursuit, Steven Seagal's fights, gunfights... It should be freaking
immersive.

~~~
soylentcola
Well, no. That would be pretty awful in my experience. First person in VR is
already dodgy if you're watching/playing from a chair and your avatar is
standing or walking. Still, the only thing worse than a mismatch between your
position and the avatar's position is a first-person viewpoint where you don't
control the camera or movement.

Watching a movie from the first-person perspective of an actor would be one of
the barfiest things you could do in VR. Ideally you're observing things from a
vantage point but everything is in 3d with full 360-degree spherical range of
motion. You can lean around or closer to look at something.

Now, even a mostly fixed viewpoint can be disorienting when you don't have a
"body" to look down at (you feel like you're floating and disembodied) but
it's probably second to the motion issues. Not sure if people will develop
conventions that become generally accepted (like some sort of
placeholder/amorphous body for reference) but overall, think more "fly on the
wall" than "I'm Duke Nukem!"

~~~
damon_c
Exactly.

What is going to be cool is stuff like standing right on or slightly above the
field in the middle of a sports event, or right on stage at a concert next to
Mick Jagger and being able to move around a bit.

Unfortunately there will need to be a globe roughly the size of the range of
motion somewhere to capture the light rays... at least for now until future
physics are invented.

~~~
ethbro
If you dropped once of these in place of the wire cam over {insert field sport
here}, millions of dollars.

You wouldn't have more than limited head tracking (or live motion without
turning off tracking and turning it into a virtual window), but being able to
have _presence_ from 10' above the QB in American football for each snap?

$$$s where you start counting the zeros instead of the digit in front.

~~~
Natanael_L
Imagine every roller-coaster video clip you've seen be filmed with this thing,
every 360° video on youtube, putting these on drones, etc...

Oh, and every high security facility in the world would love these as they're
capable of mapping out 3D positioning, enabling very precise face detection,
etc...

And imagine live concerts sent to be displayed on VR headsets!

Also, in many cases you can likely use 4-6 in a grid to capture the entire
scene.

~~~
ethbro
Remember: desync'd head motion w/ virtual camera perspective motion = violent
nausea.

So limit the applicable use cases to where that isn't going to happen.

The excitement about this is that it makes that no longer a concern for
sufficiently small values of head motion. Read: the motion that occurs when
the best of your body is not moving.

------
amluto
I hope for Lytro's sake that they release real open-source playback tools.
AFAICT, if you have a first-generation Lytro camera right now, you're pretty
much stuck viewing the pictures you take in a Flash app, and I'd be very
hesitant to buy a camera of any sort that gives no real assurance that the
picture format will still be usable in ten (or twenty or fifty) years.

Everyone has access to a JPEG decoder. Lytro, not so much.

~~~
frankiejr
Lytro switched away from Flash to a JPEG/JavaScript player in 2013, then to a
WebGL player in late 2014. There were some open source player controls
floating around last year but I can't recall the name. There were also some
open source tools to extract the JPEG stack from the .LFP file as well, which
you could use to build a player of your own. I worked at Lytro from 2012-2015
and we'd see occasional updates from those tools.

~~~
amluto
Huh, guess I'm out of date here.

Is it really a JPEG stack? I had assumed there was more magic (i.e. math and
clever compression) than that. Or is a JPEG stack with extra fanciness?

I'd love to read up on how this stuff actually works. I know how to calculate
the theoretical information capacity in a light field (hint: very very dense)
but basically nothing about how people manage to munge it into something
compact and useful.

~~~
dwaxe
At its core, the viewer works by just averaging the JPEG stack at varying
offsets as described in this student project:
[https://inst.eecs.berkeley.edu/~cs194-26/fa14/upload/files/p...](https://inst.eecs.berkeley.edu/~cs194-26/fa14/upload/files/proj6/cs194-dj/faizullabhoy_riyaz_proj6/)

I haven't looked into how they make everything efficient, but the source code
for their flash viewer is here
[http://lightfield.stanford.edu/aperture.html](http://lightfield.stanford.edu/aperture.html)

------
mortenjorck
This is pretty exciting - it looks like Lytro finally found a good use for
light-field photography. Trying to shorehorn the concept into a camera product
was never going to take off - as admirably as the company tried, refocusing an
image in post was ultimately proven a novelty that the market was never going
to consider worth all the trade-offs in resolution and convenience.

This has the look of a perfect pivot: Capturing light fields could turn out to
be _the_ key to VR video, and Lytro is poised to make the ARRI Alexa of the
industry.

~~~
GhotiFish
this is the first time I have ever heard of "light-field photography" and I am
at a loss as to why this is exciting or a critical technology for VR. I don't
get it. Why is having a grid of slight perspective shifts better than just
have two perspectives for each eye?

~~~
CarVac
The viewer's head isn't stationary, so you have to have a perspective for each
possible eye location.

------
theschwa
There seem to be a lot of questions on what a light field is. Lytro just put
up a great blog post that gives a good introduction to them
[http://blog.lytro.com/post/132599659620/what-is-light-
field](http://blog.lytro.com/post/132599659620/what-is-light-field)

------
akgerber
Product page: [https://www.lytro.com/immerge](https://www.lytro.com/immerge)

------
jjhale
Site is giving me a 503.

Video here [https://vimeo.com/144034085](https://vimeo.com/144034085)

~~~
tantalor
Bullshit ends and content starts at 1:10.

------
JonLim
This looks and sounds incredibly cool, but I have no real idea what it does.

Could someone with domain knowledge give me a bit of a high level explanation?

I have inferred that it's something you use while shooting footage to capture
how light is present in the room at all of the different possible angles. Is
this then used to be able to generate new scenes that VR headsets can walk
around and interact with? Or is this more for being able to add computer
generated images into the room and light it properly?

~~~
ryandamm
If you want a dense, technical explanation, I wrote about this hypothetically
back in April, for the same site:

[http://uploadvr.com/what-the-bleep-is-lytro-doing-in-
vr/](http://uploadvr.com/what-the-bleep-is-lytro-doing-in-vr/)

The editor informs me that HN has crushed his site, though. Should be back up
soon, I hope. (Good argument for mirroring my own articles on my otherwise-
defunct blog, I suppose.)

~~~
wlesieutre
uploadvr_will, you've been marked as [dead], probably due to spam filtering on
new accounts. Most users won't be able to see your posts, but you should be
able to contact the admins to get it fixed.

~~~
DanBC
If you click the timestamp you should be taken to a page that has a "vouch"
link. So non-mods can now vouch for dead posts.

Here's the announcement:
[https://news.ycombinator.com/item?id=10223645](https://news.ycombinator.com/item?id=10223645)

And here's a follow up:
[https://news.ycombinator.com/item?id=10478886](https://news.ycombinator.com/item?id=10478886)

~~~
wlesieutre
Neat, I missed that! Thanks for the heads up.

~~~
dang
I think DanBC inadvertently gave the wrong (first) link for the original
announcement of this feature. It was
[https://news.ycombinator.com/item?id=10298512](https://news.ycombinator.com/item?id=10298512).
The link Dan gave was when we announced some other stuff, that you also should
know about. :)

------
oroup
This is very cool technology. The challenge it highlights is that there is no
opportunity to have lighting or grips or special effects "out of frame"
because the entire visible space is "in-frame". All those shots that of
untouched wilderness or of actors pretending to drive or getting yanked around
on wire harnesses become much much harder since you'll need to create a
believable 3d representation of whatever traditionally would not be visible.

------
tsunamifury
Where does the camera man sit and how does he/she move the camera?

This basically forces you into stable shots only which are far less
compelling. Ring type VR cameras allow the cameraman to sit in the middle of
the ring while moving the scene for far more interesting results.

EX:
[http://www.nytimes.com/newsgraphics/2015/nytvr/](http://www.nytimes.com/newsgraphics/2015/nytvr/)

~~~
thescriptkiddie
You do _not_ want a moving camera for VR cinema, especially not if you are
using a lightfield camera. If camera moves, the user is struck with the
nauseating illusion that space is moving around them.

------
Keyframe
If this opens up a possibility to (easily) isolate objects per depth, as in
deep compositing, then it would be worth the price of admission alone.

------
dharma1
Really cool. I guess none of the current display tech (aside from Magic Leap)
are not able to display light fields properly though? I mean natural DOF
depending on where your eyes are focused on

~~~
corysama
Nvidia had a prototype a while back [http://lightfield-forum.com/light-field-
camera-prototypes/nv...](http://lightfield-forum.com/light-field-camera-
prototypes/nvidia-near-eye-light-field-display/)

That hardware adds variable focal length to each pixel. But, even without
that, simply being able to move your head around inside the captured scene
with the current tech is 90% of the effect.

------
jeffchuber
can you only move your head "within" the bounds of the sphere?

~~~
uploadvr_will
You can extend beyond, but it will either fade to black or show up with much,
much lower resolution

~~~
jeffchuber
thanks!

------
achalkley
So that's why their older cameras are selling at a heavy discount on Amazon. I
thought something was coming around the corner.

------
damon_c
Here is a "handmade" implementation of this concept made with normal cameras.

[https://home.otoy.com/otoy-demonstrates-first-ever-light-
fie...](https://home.otoy.com/otoy-demonstrates-first-ever-light-field-
capture-for-vr/)

edit: it's actually mentioned in the article which I couldn't see at the
time...

~~~
mrfusion
That's cool. Do you know if they have a version I could try out in the oculus?
It just looks like a utube video on that site.

------
rasz_pl
"demo video" is defined in small print as artistic impression and conceptual
rendering. Dont get too hyped.

------
IshKebab
Hmm I'll believe this when I see it (and not the ridiculously faked demos in
this video).

------
w_t_payne
I want one.

------
gcb0
lytro is and always will be novelty shenanigans.

until they open up their file standards for real and get some open source
project going, it will continue to be ignored by every pro and enthusiast.

~~~
mastax
Yes, no pro works with RED or ARRI cameras due to their closed file formats
and lack of open source projects.

~~~
serge2k
Aren't proprietary RAW formats pretty much standard for cameras?

The difference at this point is that I can use the RED format with industry
standard software.

