
How Magic Leap Is Creating an Alternate Reality - ghosh
http://gizmodo.com/how-magic-leap-is-secretly-creating-a-new-alternate-rea-1660441103
======
tb100
I'm co-founder and research director of a mobile AR software company, Zappar.

I blogged on Magic Leap a few weeks ago, and came to some of the same
conclusions as the author of the gizmodo piece - namely that a device that can
be like Google Glass and Oculus and everything in between could be a real game
changer.

I chose to call the concept "Controllable Reality" (I make no claim that I've
invented the concept or the term!): [http://www.zappar.com/blog/google-glass-
magic-leap-and-the-i...](http://www.zappar.com/blog/google-glass-magic-leap-
and-the-ideal-ar-wearable-display/)

Edit: I also talk a bit about the requirements for an AR experience, existing
approaches to AR HMDs and the big problems to be tackled on the road to
consumer adoption. I've got a PhD on the software side of AR but am not an
expert on the hardware, but have sufficient knowledge to hopefully offer an
interesting perspective.

I'm new to HN too, so let me know if I should be posting this as a separate
item or not!

Simon

~~~
angersock
Erm, what's wrong with the well-understood term "Augmented Reality"?

~~~
tb100
It's more about capturing the spectrum from AR to VR in a single device. With
controllable opacity (which I argue is required for a truly useful AR display)
you also have the option to deliver purely VR experiences by blocking out the
real world completely.

~~~
tfinniga
I bet some of it is also about expectations. If you just say AR, people think
of low-poly characters in a webcam on top of a black and white checkerboard.
They're not interested.

~~~
tb100
And from the video linked in the Gizmodo article, Graeme Devine claims they
don't call it AR because "that is just a 2D HUD", and that's why they insist
on the weird marketing phrase "Cinematic Reality".

To me and most people who have been around AR for years it is very clear that
registration of objects in 3D is a core part of a lot of AR experiences. That
was the part of the video that struck me as most weird; that Magic Leap would
make a lot of noise around AR and yet have a Creative Content VP seemingly so
unaware of the existing AR field.

------
arethuza
By far the best novel I've seen that looks at a near future (2025) world where
most people spend most of their time using augmented reality is Vernor Vinge's
_Rainbows End_ :

[http://en.wikipedia.org/wiki/Rainbows_End](http://en.wikipedia.org/wiki/Rainbows_End)

~~~
neuromancer2701
I loved Rainbows End. Recently I read Daemon/FreedomTM, and it paints a very
different picture. But this technology could be used to create the Darknet the
alternative web/economy in the books.

[http://en.wikipedia.org/wiki/Daemon_(book_series)](http://en.wikipedia.org/wiki/Daemon_\(book_series\))

~~~
freehunter
Daemon was a game changer for me. Ever since I've read the book (recommended
by a colleague), I've dreamed of what could be possible.

That book (along with a small handful of others) made near-future scifi my
favorite genre. Any other suggestions?

~~~
ehsanu1
> _That book (along with a small handful of others) made near-future scifi my
> favorite genre. Any other suggestions?_

Which others?

If you've read _Daemon_ and _Freedom (TM)_ , maybe you've already read _Kill
Decision_ , also by Daniel Suarez. It's about autonomous, swarming drones used
in the military. _Daemon_ is a better book though IMO.

There's _Hieroglyph: Stories & Visions for a Better Future_, which is a
collection of short stores in the near-future sci-fi genre. Neal Stephenson
found Project Hieroglyph with the intent of advancing positive (non-dystopian)
sci fi in the near future, and thus spurring on the next generation of
scientists and engineers to go ahead and create it.

Peter Watt's _Blindsight_ and _Echopraxia_ are also good (currently still
reading the latter). It has scientifically plausible zombies and vampires, so
that's something.

Ramez Naam's _Nexus_ has very interesting ideas on what happens to the world
with very good brain-computer interfaces. It's set in the near-future, but I
do think the tech is implausible here for so soon.

William Hertling's _Avogadro Corp_ is about a paperclip maximizer built by
mistake by a Google-like company, with a Gmail-like service that is
manipulated by said paperclip maximizer.

~~~
freehunter
> _Which others?_

The Martian (Andy Weir), Mind's Eye (Douglas Richards), and Wool/Shift/Dust
(Hugh Howey, set quite a bit in the future, but the story started
(chronologically) in with near-future sci-fi)

I haven't read Kill Decision yet, I've heard mixed reviews. The thing I loved
most about Daemon and The Martian is that it's all completely plausible that
these things could happen in my lifetime. Hell, they could happen right now if
someone was willing to put forth the money. Nothing is particularly outlandish
or impossible.

I'll have to check out your suggestions, thanks. It's my favorite genre, but I
struggle to find worthwhile books in that genre.

------
coldcode
People often talked about Steve Job's reality distortion field but Apple
shipped real products that did change the world. I will believe in a magic
leap when I see it appear, not when I hear all the hype machine output.

~~~
Tloewald
I think it's odd that they're doing this kind of PR push when their product is
so tenuous. If you look at Carmack's keynote and the low level detail of how
Occulus is struggling with latency and precision in head tracking, the idea
that these guys will tackle a whole swag bag of deep problems in some kind of
predictable time frame is pretty ridiculous. My guess is they've made
promising inroads in the core display technology (lcd occlusion plus
fiberoptic scan projection) that is very impressive on its own, but everything
else they've got is merely state of the art.

~~~
tfinniga
PR push? This article doesn't seem like a PR push at all. It seems like
someone scraping as much information out of as many sources as possible.

If it was a PR piece, they'd have animations and pretty renders about how all
the things work and where they're going. Instead there are patent diagrams and
quotes from job listings.

This seems more like journalism.

~~~
Tloewald
Did you see the video of the Unity 2014 presentation? That's a PR push aimed
at developers where they have literally nothing concrete to show.

~~~
josephpmay
It's a push to get developers to work for the company, not to get excited
about the platform

------
hyp0
It sounds like they've got some trick that gives _some experience_ of real
objects - probably far from perfect (hi-latency, lo-res, etc); but that's a
good thing for a company. It means they have somewhere to go when competitors
catch up.

Maybe opaque and depth of field really fool your eyes?

As a comparison, consider how surprising it is that animation works: line-
drawn cartoons look like real objects, animals etc; secondly, that if you just
change the images at a mere 24 fps, they come to life (the meaning of
_animation_ at the time).

~~~
egypturnash
The threshold is lower - a hell of a lot of hand-drawn animation is done 'on
twos', with each frame shot twice, for an effective framerate of 12fps. 'Going
on ones' is mostly reserved for motion that has to match a panning background
(panning the whole screen at 12fps looks terrible, and so does someone whose
feet are siding back and forth with respect to the ground), or for really fast
motion. Or occasionally for really smooth, subtle motions.

In my experience 10fps is when cartoon images start to break down into "a
sequence of drawings" rather than "moving images". This is right on the line
for me; I'm not sure how common that is for people who haven't trained their
eyes to the point where they can pick up on a single frame being shot 3x
instead of 2x.

I mean, yeah, 24fps looks better, 30 or 60 fps is even better, and the maximum
frame rate of the human eye seems to be anywhere between 255fps to 1kfps (I'm
getting a lot of conflicting info from a quick google for that), but 12fps is
pretty solidly acceptable for moving drawings.

(I'm a former animator.)

~~~
hyp0
Maybe very low-quality AR can be convincing, provided you're stationary (i.e.
no panning-like effect)?

Relatedly, Carmack found some serious latency problems with Oculus Rift, but
only with some particular combinations of object movement and changing viewing
direction. He's thinking of fast video games with lots of movement. Just
having a dragon floating in front of you is quite a lot more constrained.

Though I meant it as a general observation, i.e. animation is surprisingly
effective, if you do it the right way; perhaps AR will be too, if it's done in
(so

------
seren
I am a bit puzzled by the names of Austin Grossman, Dave Gibbons and Andy
Lanning being cited as employee (whatever that means).

Are they so confident in the technology that they have started to heavily
invest in the content? Is this a way to position themselves as an
entertainment company rather than a tech company?

~~~
rondon2
Two things.

1\. An elephant on a flower isn't going to get someone to pay $500-$1000 for
this hardware. You need very high quality content to get lots of consumers on
board.

2\. Moving from current cinema production to VR production is going to be a
huge learning curve for a lot of artists. You need to create best practices
and examples of how it is done by artists.

~~~
_almosnow
Those sound like real obstacles. Content is always a big problem, but because
of the amount of money that has been poured into this and the people/companies
backing it up, I'm pretty sure that they already have that one solved.

------
tegeek
This November is pretty heavy for geeks & Singularity lovers (I'm one of
them).

First we saw a man-made machine landing/crash-landing 500 millions KM from
Earth.

Then HN discussed this [1] which is IMO one of the biggest machine-learning
demo I've ever seen (other than Self-driving cars).

And now this futuristic tech which is aiming to eliminate the screens
altogether. (As some of its use cases)

If Magic Leap delivers her promise within next 2-3 years, then we are already
ahead than Singularity timeline [2].

1\.
[http://cs.stanford.edu/people/karpathy/deepimagesent/](http://cs.stanford.edu/people/karpathy/deepimagesent/)
2\.
[http://en.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzwei...](http://en.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzweil#2019)

~~~
melling
How can we be "ahead of schedule" when we still don't have the first couple
tasks from a decade ago? Dictation, for example, is pretty good but it's not a
solved problem.

* Translating telephones allow people to speak to each other in different languages.

* Machines designed to transcribe speech into computer text allow deaf people to understand spoken words

[http://en.m.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzw...](http://en.m.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzweil#Early_2000s)

~~~
tegeek
>> Translating telephones allow people to speak to each other in different
languages.

The technology is there to build this kind of device, I agree not 100%
accurate but still practical enough to be used in real life.
[http://en.wikipedia.org/wiki/IraqComm](http://en.wikipedia.org/wiki/IraqComm)

>> Machines designed to transcribe speech into computer text allow deaf people
to understand spoken words

This too, the technology is there to build a device. I would say we've much
more powerful devices in form of smart phones than just doing 1 or 2 things.
The talko app on iPhone is actually much more than just speech to text
converter. [http://www.talko.com/](http://www.talko.com/)

I would also like to say that Singularity predictions doesn't employ that the
technology will be so mainstream that every person will be using that. But
even if we can build a technology in some lab, it still becomes an event on
Singularity timetable. I've yet to see any off the mark prediction from
Singularity timeline.

------
sixQuarks
Magic Leap has Brian Cox and the team behind the movie Gravity working with
them as well. This is impressive if they can pull it off:
[http://www.bbc.com/news/entertainment-
arts-30056232](http://www.bbc.com/news/entertainment-arts-30056232)

~~~
Tangokat
Pretty interesting quotes from a guy like Brian Cox:

"It's the premiere of a technology that allows you to put digital images into
your field of vision directly," he said. "I saw the prototype in Miami a few
months ago and it's stunning."

"It is going to be transformative technology, there's no doubt about that."

The experience will "disturb" audiences and put them "off balance", he
predicted. "That's what it did when I saw it demonstrated."

The article also says that a demonstration will be made as a part of the
Manchester International Festival next July.

------
thewarrior
Imagine going to the Eiffel Tower in Paris.

On top of all the real people walking there , you'll have virtual people as
well who are all in their living rooms yet still interacting and conversing
with everyone as if they were really there.

Everyones Magic Leaps would be synced to show the same composite reality.

You could have virtual assistants to show you around the city , explaining the
various attractions or just to give you some company.

Ofcourse you'd need a gigantic network of overlapping 3D cameras to support
such a feat but we won't let concerns like feasibility get in our way. I
dreamt of this once and now I'm really excited that I might get to see this
within my lifetime.

------
josefresco
$542 million! I'm used to big numbers from following the tech startup world
but this seems huge. How many other half-a-billion dollar investments in tech
are there?

Doing some Google research, I found some lists of top VC deals:

[http://www.entrepreneur.com/article/233632](http://www.entrepreneur.com/article/233632)
[http://venturebeat.com/2012/12/28/top-venture-capital-
deals-...](http://venturebeat.com/2012/12/28/top-venture-capital-deals-2012/)

Looking quickly at the numbers, it does seem that this deal was relatively
large.

~~~
trishume
If this technology is as powerful as it claims to be, I could easily see Magic
Leap eclipsing Apple as the most valuable company in the world as long as they
hold those patents.

I suspect their strategy for finding investors is like this: 1\. Potential
investor is incredulous about the company's value 2\. Investor gets a demo of
their technology, and it's mind-blowingly amazing. 3\. Investor asks them to
name any price and gets out their checkbook, knowing that this has the
potential to be the biggest invention of the century.

------
bake
This seems like the future for Google Glass -- those articles a few days back
about how the program is waning / losing steam seem to overlook Magic Leap,
which feels more like Google doubling down.

~~~
baldfat
I think Google Glass (Aka notifications on your eye glass with a camera and
augmented reality are apples and oranges.

For notifications and wearables glasses was an idea that has proven to be a
bad choice for social as well as practical reasons.

Augmented reality is well adding to our reality and I believe what people
really want and NOT virtual reality. I can image Amazon Echo with a butler
image would be a HUGE hit as opposed to the audio only option.

------
hemancuso
By all accounts, they are working on some sort of device that projects a light
field onto the retina. But that is strictly additive. You can't darken
anything. This image you see of the elephant in your hand - no way to do that.
At best it'd be rather transparent.

Unless there is some aspect of vision that somebody here would like to chime
in on [locality on the retinal sensor allowing to perceive darkness by
lightening up other areas?], color me deeply skeptical. It seems like
"fullscreen" Google Glass, at best.

~~~
bri3d
[https://www.google.com/patents/US20130128230?dq=inassignee:%...](https://www.google.com/patents/US20130128230?dq=inassignee:%22Magic+Leap,+Inc.%22&ei=VpdaVI-
RMM6oogSW3YKADQ&cl=en)

This patent describes the "occlusion mask device" which is supposed to do
exactly this - darken light coming from certain angles to grant the perception
of blackness.

~~~
hemancuso
Seems like a screen in front of your eye that blocks out light.

~~~
oh_sigh
Presumably it is a screen that is an array of pixels which are controllably
transparent.

------
hyp0
\tangent I've always loved the idea of eye-tracking. It helps the whole
graphics stack, because don't need to render the whole "screen", just the
exact part you're looking at. And you only need render the center of it in
sharp detail (what impinges on your hi-res fovea); the surroundings can be
blurry and even monochrome (no cones outside the fovea, only rods!).

Unfortunately, because eyes saccade (twitch) extremely fast, latency must be
ridiculously low or it would send you insane (figuratively).

~~~
xorcist
It would also be impossible to use for more than one person at a time.

------
mgirdley
The Seqway had similar amounts of mystery and hype years ago.

------
cardigan
"projects light onto your eyes"

Pretty sure my laptop does this :P

~~~
kordless
It's the inverse of the camera demo on the page though, so that means they
have figured out how to project light with the lasers down an optical fiber.
Stick that nearly transparent fiber pointing straight at your eye, in your
eyeglasses, say, and you have a way to paint on your visual input areas.

------
bake
If it works, Magic Leap will mark an entirely new era in machine-human
interface. It could eliminate the need for screens, keyboards, and even other
input devices like light switches and thermostats by turning any surface into
a tactile interface. Why actually attach a piece of hardware to the wall when
I can just render it there whenever I look in that direction? Why carry along
a physical keyboard when I can just 'project' one on any desktop at hand?.

There's also an interesting health & wellness aspect to this -- could this be
a way to free people working digitally from sitting at desks in front of
screens all day?

This is, in my opinion, the most exciting technology in the popular press
right now.

~~~
7952
A lot of old technologies tend to hang on within a niche that they are optimal
within. We still have radio, cinema, and telephones even when each of those
technologies has a better more modern equivalent. I don't doubt that AR will
find a niche but I doubt it will be replacing 10$ keyboards or 50c dials on
thermostats. It is fantastic that Magic Leap has moved beyond the horribly
mundane "helping people find a restraunt in San Fransisco" model that Glass
seemed to adopt. Fantasy AR overlays sound ridiculosly addictive as it would
allow contsant enveloping escapism.

~~~
orbifold
I'm not sure if Fantasy AR overlays are really all that compelling. Unless
they make significant progress in AI as well, most likely those will be
entirely predictable and boring after a while. Although who knows maybe there
are enough people around who want to have a virtual pet dragon. Most of the
real world outside is entirely unsuitable as a canvas for any kind of
interesting movie like experience. Traditional movie story telling artfully
uses both cuts in time and transitions in space to tell a compelling story,
neither of which is readily available.

------
aaron695
One assumes the big players are seeing through the gimmicky whales and
seahorses and can see a wider more functional future.

Pretty things cost huge amounts of money to create. Making them 3d and
interact with your environment just adds to the cost.

------
androidb
Can't believe anyone here didn't mention this but I'll be the immature one.
Imagine porn on this device. I mean just, you know, all the Scarletts and
Jessicas.

~~~
anigbrowl
It's not immature to make the observation. Porn has been a driver of new media
technology throughout history - Magic Leap reminds me of nothing so much as a
portable kinema viewer.

------
jcfrei
I wonder how such a "display" of the Magic Leap is controlled. Could it be
that it's no longer a rasterized display? Maybe rather than coloring pixels on
a plane, the developer is required to send display information of a volumetric
object like in a volumetric display:
[http://en.wikipedia.org/wiki/Volumetric_display](http://en.wikipedia.org/wiki/Volumetric_display)

~~~
tb100
In general you just to provide the depth map too. You're still only rendering
for one viewpoint (well OK, one for each eye) so the display just needs to
simulate the depth and colour of the point nearest to you along any ray.

~~~
tb100
...distinct from volumetric displays which support multiple simultaneous
viewers so need to actually render in terms of voxels.

Combining that with tracking depth cameras and sharing the map of the
environment over the network would let multiple co-located users see and
interact with the same 3D object (much like with a volumetric display) but the
actual rendering of each user's view is decoupled.

Combining this with eye-tracking, as \hyp0 mentions, takes this to the limit -
you don't need to construct a "holographic display" that renders super high
resolution light fields of rays in controllable directions to all the infinite
possible viewpoints around the display. Instead just stick it on your head,
locate the users in a shared depth map, and just render the slice of that
their eye needs to see right now. In terms of CPU/GPU/bandwidth issues it's
much more plausible with today's technologies. The display side is still
challenging!

------
waterlesscloud
I think Gizmodo is a source HN's algorithms penalizes, which is a shame in
this case. It's an interesting article with some good details.

~~~
syoc
This. I refuse to click on gawker links but I would love to read the article.
It would be nice with a pastebin / imgur link when people post things from
gawker.

------
thinkstorm
If a "lightfield" is projected into your eye, how do you cope with (additive)
light input from the real world?

if it is real, will there be a law to tag 'virtual' things so i don't
accidentally kill a real person when I thought that was part of my game? You
know how congress or the senate would design that one ;)

------
ohsnap
Sounds awesome, but if it is as gizmodo describes it feels like one of
Google's moon shots. We are just barely starting to see good VR... combining a
real and virtual world together is another order of magnitude of difficulty.

I'd love to be proven wrong but don't see this for many years.

------
peterwwillis
Maybe I should have stayed at MAKO a little longer for that payout... derp.

------
tempestn
That sounds _awesome_. Also much less tangible than the Rift, obviously. But
with that kind of funding, obviously some smart people believe they've got
real potential. Colour me excited.

------
angersock
Meanwhile, in the land of not vaporware:

[https://www.spaceglasses.com/](https://www.spaceglasses.com/)

~~~
tb100
In the land of "not really good enough". Additively mixed at a single focus
depth plane, not dissimilar to the hacks people have been using in research
for AR for a long time.

It does however really annoy me the amount of press Magic Leap have managed to
get without saying anything public at all. For me it's an opportunity to think
and talk about the properties of an ideal AR HMD and why it might be an
interesting device. I have no idea how close Magic Leap are going to get to
that ideal.

------
tragomaskhalos
Day N: this tech goes to market.

Day N+1: the first pr0n application using it appears.

