
Apple is betting on augmented reality - iKenshu
https://www.bloomberg.com/news/articles/2017-03-20/apple-s-next-big-thing
======
nxc18
I'm impressed by the ability of the press to take any new tech idea and spin
that into Apple being the latent dark horse that will win the category.

Off the top of my head I can think of streaming TV, a physical TV set, AI, and
self-driving cars.

Is there any evidence Apple has any advantage over the already quite
competent, well-funded players who have already demonstrated significant
progress in this area?

~~~
saycheese
AR will become mainstream at the point people see it as the norm in public.

Apple's core advantage is that they set the standard for mobile smart devices
and consumer will follow them if they're able to gain momentum.

Question at that point swings back to how they'll try to do it, which becomes
highly speculative.

If I had to guess, they'll morph the iPhone to a AR device like the morphed
the iPod to a phone; meaning the first step is to add AR to the iPhone and
gain traction.

~~~
Cthulhu_
The iPhone already is an AR device though, and one of its first major apps was
an AR application - Layar, back in 2009. One of its major apps has an AR
component (Pokemon Go).

I could see them making a Google Cardboard-like mounting bracket / dock for
the iphone I guess, maybe with better cameras and sensors.

~~~
mulletbum
I think Google Glass is more where they will go. Except, just like the first
iPhone, they will make it actually look like a normal pair of sunglasses.
Apple has NEVER made new products, but they have refined existing products at
key times to come out with something amazing. iPod, iPad, iPhone were not the
first, but they were breakthrough staples that rules their area. Apple Glasses
would do the same if done right.

~~~
root_axis
> _Except, just like the first iPhone, they will make it actually look like a
> normal pair of sunglasses._

Not going to happen (any time soon). That's an insurmountable engineering
challenge for the foreseeable future.

~~~
MarkMc
Let's get specific: How long do you think it will be before Apple release
iGlasses? 10 years?

~~~
root_axis
I'm not sure when they will release a product in the vein of "iGlasses", but
I'm certain they won't release an AR device that "looks like a normal pair of
sunglasses" within 10 years. This isn't a situation where incremental progress
will make this possible, there would need to be a fundamental breakthrough in
battery and GPU efficiency that, while possible, is not necessary inevitable
and definitely not on track within a predictable timeline.

~~~
probablybanned
What do you think about solving this with wireless power transfer? There has
been some promising research recently into extending the range of resonant
inductive coupling.

It's nobody's ideal, but a clip-on/necklace style power solution might be
sellable to the public if the value proposition of the glasses is good enough.

~~~
root_axis
A future where resonance charging equipment is safe, functionally practical
and ubiquitous is sort of a breakthrough scenario in and of itself, but even
then, you're still talking about a technology that is somewhat out of reach.
Even if we can squeeze the necessary GPU power into the form factor of "a
normal pair of sunglasses", consider the additional heat dissipation
requirements of a device as thin as normal glasses that is also in a state of
perpetual charge, especially when we take into account the human face's
particular sensitivity to heat.

Maybe very thick oversized (by today's standards) frames will come into style
and something like that will be on the border of possibility, but it seems
like a real stretch within the next 10 years.

~~~
probablybanned
After thinking it over a bit, I'm going to take the other side of the bet.

We agree that power consumption is a limiting factor no matter what. In my
imagined system, only something on the order of 1W would be delivered to the
glasses. Delivering 1W over 10cm with large-ish coil area at >50% efficiency
using NFMR isn't really a breakthrough, but it's _only_ a way of eliminating
the LiPo bulk and nothing more.

The thing is, I don't think we're near the boundary of power efficiency even
with today's silicon. The way forward is to push all the feature tracking and
other low-level vision tasks down to the ASIC level, while accepting some
strict limitations on the rendering side. So a lot of it comes down to scope
of features. Its most immediately useful applications are little more than a
glorified HUD -- project some GPS data here or an info overlay there. Blit a
video source on top of a marker, that sort of thing.

This sort of practical product, by 2027, I'll say yes. Mindblowing, transform-
your-world stuff? No. But let's see what happens...

------
CPLX
I love reading paragraphs like this:

 _The global market for AR products will surge 80 percent to $165 billion by
2024, according to researcher Global Market Insights. But Apple really has no
choice, says Gene Munster, a founding partner at Loup Ventures who covered the
company for many years as an analyst. Over time, Munster says, AR devices will
replace the iPhone._

Oh, OK. Glad we've got all that sorted.

~~~
colmvp
It's 2017, and e-books didn't replaces paper books, tablets didn't replace
textbooks or laptops, gamers still use their PCs (it's not dead!), and
consoles are still a thing despite the popularity of mobile games.

~~~
seanmcdirmid
It's 2017, e-books have hugely affected the paper book market, tablets have
hugely affected the textbook and laptop markets. PCs are no longer a growing
as a market and are actually shrinking a bit. Consoles aren't as lucrative as
they used to be because of the popularity of mobile gaming.

Anyone in this industry who doesn't try to look ahead will be caught with
their pants down. The byline:

> CEO Tim Cook is betting on augmented reality, a cousin of VR that he
> believes will keep his company on top and may even supplant the iPhone.

This is a completely reasonable statement in that context, it isn't as strong
as Munster's quote above, but it probably wasn't meant to be that strong
anyways.

~~~
CPLX
Gene Munster, of course, is uniquely talented at predicting what Apple is
going to do:

[https://9to5mac.com/2011/10/24/analysts-apple-prototyping-
te...](https://9to5mac.com/2011/10/24/analysts-apple-prototyping-television-
set-for-a-2012-launch-but-it-wont-come-cheap/)

[https://9to5mac.com/2015/05/19/munster-apple-television-
retr...](https://9to5mac.com/2015/05/19/munster-apple-television-retraction/)

------
Shank
There are a lot of skeptical comments in this thread, but remember: Apple has
revolutionized past industries like phones and tablets where other competitors
failed miserably. A compelling AR product can and will exist eventually, and
all this article is saying is that Apple will be competing in the AR field.

An aside, but I can imagine AR done right being extremely useful. I love the
idea of being able to instantly have access to information on my phone, but it
always requires typing or voice input. I could totally see just looking at
something and tapping a button triggering a card that says "that thing you're
looking at is this."

While that's just my dream, I'd much rather see people experimenting with new
technologies rather than forgoing it with "Apple can't do X right, how can
they do Y?" If anything, it could be because Apple is working on Y that X is
taking a backseat for a while.

~~~
sebleon
Before, Apple used to be led by a product visionary. Now it's led by a supply
chain expert.

I seriously doubt Apple will develop another revolutionary product with the
new leadership. My prediction is that paradigm shifting AR products will come
from a product-driven organization, though Apple may buy them out and scale
it...

~~~
johansch
I also have some very serious doubts about Tim Cook's abilities as a CEO.

However, it's quite obvious that AR is the next UX frontier:

Historic UX frontiers:

#1: text/console/terminal

#2: Graphical UI with a mouse (Apple sorta owned this at first)

#3: Mobile graphical UI with a capacitive touch screen (Apple owned this, at
first)

#3.5 VR (but putting on a helmet is quite a bit too hardcore for mass
adoption)

#4: AR

This progression is because of the 3.5 VR step quite obvious even to Tim Cook.
So, I don't really doubt the claims that he's been doubling down on AR @
Apple. It all makes sense.

And once he has given the command, I have no doubt that the Apple company has
enough talented people in each of the required competency areas to pull it
off.

Perhaps the most important thing here is that a) Apple has lots of money, b)
Apple/Cook has lots and lots of pressure to come up with the new thing.

I think we currently are in that particular place that doesn't require lots
and lots of executive vision and taste.

~~~
sebleon
Right - identifying AR as a hot area is easy, but formulating what that looks
like requires vision. Money and talent will deliver the Palm Pilot (1997),
vision will deliver an iPhone (2007).

My take on UX frontiers:

#1 text console #2 mouse & keyboard #3 touchscreen #4 camera & microphone

I think we're at the beginning of #4, and starting to see this in categories
like messaging, social networks, dating, and e-commerce. I consider overlaying
AR elements to a live camera feed to be a subset of this shift, but by no
means the defining aspect of this new method to interact with machines.

~~~
johansch
> Right - identifying AR as a hot area is easy, but formulating what that
> looks like requires vision.

Good point. And an relevant parallel between Sculley's Newton and Cook's
potential "AR innovation project".

However - I get the feeling that it's a bit different from the mobile/touch UI
revolution. That required lots of vision in terms of software and design. In
AR I think there's really a _lot_ of low-hanging fruit in terms of UX/design -
once the trick part; the AR hardware has been solved.

~~~
sebleon
Thanks!

What low-hanging fruit do you see for AR?

------
rcarmo
I just don't see this happening, for a number of reasons related to the way
Apple designs hardware - as unobtrusive, natural extensions of what we already
do that you can hold but also toss aside as required.

And also because most of the "evidence" listed in the article has many other,
simpler, uses in Apple's "incremental improvement" juggernaut. They can keep
adding minor popular (and less risky) features without building in-your-face
solutions that have uncertain acceptance.

Bloomberg might well be becoming the new Gizmodo as far as Apple coverage is
concerned.

------
excalibur
> Run by a former Dolby Laboratories executive, the group includes engineers
> who worked on the Oculus and HoloLens virtual reality headsets sold by
> Facebook and Microsoft as well as digital-effects wizards from Hollywood.

The entire focus of the article is AR, yet the author classifies HoloLens as a
"virtual reality headset". This is sure to be embarrassing when he realizes
the size of the gaffe.

------
nothis
I tend to be a skeptic with these types of technology but I must admit I've
seen a video of supposed HoloLens footage (and even if it wasn't, it would do
as a concept), that made me wonder.

This is the video I'm talking about:
[https://www.youtube.com/watch?v=7mtS_224mKU](https://www.youtube.com/watch?v=7mtS_224mKU)

Now it's super flimsy and everything, but I guess those technologies are best
imagined as they would look like if all hardware quirks were solved. I can
definitely see a future where, instead of a desktop/laptop/tablet, you put on
glasses and see an unlimited amount of virtual, arbitrarily-sized, floating
monitors in front of you. Forget the 3D-gimmmicks, even. That alone sounds
cool and useful. It's 2030 tech more than 2020 tech, probably, but I can see a
future where this works.

~~~
moron4hire
There are two things to keep in mind when watching videos recorded through the
HoloLens:

1\. The frame rate and image stability are much better in person. The HoloLens
has to cut the rendering frame rate in half to be able to record video through
its front-facing camera because the camera and the display share a bus. So
what you see is significantly degraded.

2\. The constrained field of view is much more noticeable in person. With the
video, you don't get to see anything out of the bounds of the frame, so there
is no reminder that we can't "see" everything. Also, we are used to watching
videos, so we don't think about the fact that we can't turn our eyes and look
in different directions. What you see is significantly more generous to the
HoloLens than reality.

I think the combination of the two factors generally makes the video an
"accurate" representation of the HoloLens, even if it's not terribly
"precise".

I don't consider the current iteration of the HoloLens to be a usable device.
Comparably, the HTC Vive is a much better experience, one that is capable of
providing to users an experience that is worth the drawbacks and the extra
effort necessary to deal with them. The HoloLens is more effort than it is
worth.

But on the other hand, tethered, completely opaque systems that require
external tracking hardware for head and limbs are clearly a dead-end
technology, even if that dead end is 5+ years from now. The end goal with
mixed-reality technology is clearly in the HoloLen's lineage, not Vive or Rift
or even Meta 2.

------
Geee
I'd rather want a new Mac Pro VR with a $20,000+ price tag. They should make
something cool, and not this crippled stuff. Completely new virtual OS with
built-in virtual programming language. Etc. Go crazy. Make it so that this is
the workspace of the future for every professional. It's been too long that we
have had to glimpse through these tiny windows. Break us free.

~~~
j2bax
Yeah, seriously! Skip the car research. Bring us a virtual alternative that
enables us to vastly reduce our need to travel! This is probably the closest
we can ever get to teleportation. Why should I have to travel to various
cities on either coast every few weeks to meet with clients? We should all be
able to meet in a virtual space that allows us to connect nearly as intimately
as an in person visit. The fact that we can do it far more often, would
probably make for an even better relationship than we currently have with
yearly live visits and lots of conference calls in between.

~~~
azinman2
Probably for the same reason you're having to travel despite the telephone,
FaceTime, secondlife, and email being invented. Telepresence isn't the same as
physical presence, and the act of costing you is an important social signal
itself. VR won't change that.

------
redsummer
And how are people supposed to develop for this new technology if they haven't
released a Mac pro for nearly 4 years, and graphic card support is well behind
Windows?

~~~
jsz0
Probably via ThunderBolt3 connected eGPUs and whatever GPU comes in the next
iMac update. Graphics card support isn't that difficult of a problem to solve.
It'll never be on par with Windows but it doesn't have to be either. There are
already a lot of AR/VR capable GPUs supported on macOS via the included AMD
drivers and NVIDIA's third party drivers. It's probably a moot point though
because they could just support testing on the AR device itself or perhaps
with any other iOS devices with a similar GPU.

~~~
iainmerrick
If only there were some way to get rid of all the cables! Could you, I don't
know, and plug the GPU _directly into a socket_ on your computer?? Then put
everything inside a bigger box to keep it all tidy and clean.

I wonder if Apple is secretly working on something like that.

~~~
tempodox
Apple just freshly created that problem by abandoning tech that wouldn't have
required those cables.

Maybe they'll re-introduce it some day and call it revolutionary.

------
Fricken
If you're counting on the iPhone of AR, expect to wait. The state of the art
is more at the palm pilot stage.

There's too many bleeding edge technologies, both software and hardware
related, that have to come together before a sleek and affordable consumer
grade device can really be a thing.

I look at a company like Magic Leap, and it's disappointing to see how much
money has been dumped on them. Now they have to build a consumer grade device
to justify their valuation, and I don't think it's possible anytime soon. I
think they should be working with a fraction of the capital they've received
and be focusing on a killer enterprise app that some industry might be willing
to pay $5000 per unit for a clunky and unfashionable device.

~~~
erikpukinskis
The _only_ thing iPhone needs to become the best AR device on the market is
rock solid low-latency positional tracking, low latency camera, and custom
silicon for overlaying simple 3d graphics over the camera feed while using low
power.

They don't need a headset.

They don't need stereo.

They don't need lightfield projection.

They just need an iPhone that feels like a handheld portal into augmented
reality. I see no reason why they couldn't do that this year.

Heck, if they add positional tracking to the AirPods, it'll even feel
immersive.

------
marknutter
People have such a fundamental misunderstanding of Apple's business model.
They do not invent—they refine. You won't see Apple release a VR/AR product
because they simply haven't been in the market long enough. Their strategy has
never been about being the first or even an early mover in a new technology.

It has always been about taking technologies which have been on the market for
a substantial amount of time and iterating on a version that gets all the
small details right until it reaches their very high internal standards for
usability. Apple isn't interested in being the first to do something, they're
interested in being the first to do something _right_.

------
iamseiko
Although I am skeptical that Apple will bring out a good, furnished AR
product, I am afraid that they might actually be successful because developers
have often showed a good inclination towards making apps for Apple devices,
and it is one of the primary reasons why Apple has been really successful in
what they do, they have apps. Microsoft's HoloLens uses apps from their
Windows Store and those that they made themselves, which is a very big
limitation for them, and when both devices are out in public and ready for
use, Apple will probably get a much larger influx of apps for their device
than other AR/VR products. Even failed Apple devices are good money-makers for
Apple, and apps are one of the main reasons for it.

The only reason I see Apple failing in the AR and VR field is their lack of
data. Apple refuses to save data from their users, and although it is
something I admire about them, it is what drives Machine Learning and
Artificial Intelligence. Google and Microsoft is successful in this area
because of the data they collect (via their Search engines and other OS
services), which Apple does not. It is one of the reasons why Siri is
competitively weaker than it's counterparts on Android and Windows, because it
does not have a good data-driven Machine Learning backend, and unless Apple
starts changing their policy on the type of data they collect and use from
their devices, their AR/VR devices will go the same way, that is they will be
successful in the amount of sales they make, but will be the weakest in terms
of functionality and services they can provide.

~~~
moron4hire
> Microsoft's HoloLens uses apps from their Windows Store and those that they
> made themselves, which is a very big limitation for them, and when both
> devices are out in public and ready for use, Apple's devices will probably
> be more successful.

What does the current state of the various app stores have to do with AR? By
that, I mean, AR applications are so fundamentally and significantly different
than 2D apps that back catalogs are meaningless. Any new player to the AR
ecosystem is starting with an essentially empty app store, regardless of
whether or on they deploy it on top of a pre-existing, successful app store.
We can effectively say that Microsoft has the largest AR app store in the
world.

And, we can expect Windows Store to grow as they push Windows Holographic and
their tethered, cheaper VR displays (the Hololens is still a $3000 "dev kit").
Apple is not just starting from behind, they haven't even started yet.

~~~
iamseiko
Apple does have more registered developers than any other company right now.
It is why Apple is able to get a larger portion of the consumer market than
other devices even when they arrive late. Just take Apple's Watch and it's
competitors for instance (Android wear and Microsoft Band).

------
1_2__3
I suppose this is as good a thread as any to again make the claim that I don't
think AI will be anything but a toy before AR is mainstream. For whatever
reason right now the fashionable thing in tech is to pretend we don't have
incredibly sophisticated and powerful meat brains in our heads and the best,
most economical, and most feasible way forward is to completely eliminate it
from every equation and replace it with silicon. When in truth the melding of
silicon (binary) computers is easier and results in something much more
powerful (and much sooner) than just silicon.

I'm fully aware computers will eventually outpace us. But the economic and
human potential of human/computer augmentation is enormous and will likely
last decades or even centuries in the "near" term. At some point we'll shed
our obsessive fascination with denigrating humans as "old'n'busted" and
remember just how incredible people can be.

------
intrasight
> While the smartphone will do the heavy lifting, beaming 3D content to the
> glasses will consume a lot of power, so prolonging battery life will be
> crucial.

The compute required for the AR or VR experience that would satisfy Apple
isn't going to happen on battery power. Perhaps the headset can be battery
powered and tetherless. I think the base unit would have to be a new Mac Pro.

> Content is key too.

Stating the obvious. But less obvious is to define exactly what one means by
"Content" in this new realm. Simple things like "games" and "movies" aren't
going to attract new customers to the new realm. What I expect from Apple here
is a paradigm shift like was the iPhone.

------
bitwize
"Baidu goes all in on AI."

"Apple is betting on augmented reality."

What's next? "Facebook doubles down on human cybernetic enhancement"?

~~~
sosborn
> What's next?

Uber folds.

------
jlebrech
when Apple looks at a technology it means the tech is ripe for mass
production.

They can see profit here, i'm sure it will be a similar price to existing
products, but the price has gone down enough for Apple's type of profit
margin.

~~~
jlebrech
also apple are not desperate to make a quick buck, they'll spend 10 years on
R&D on something till they think Starbucksers will be prepared to part with
their cash.

~~~
Cthulhu_
And they're willing to let go of an idea - like the car, TV, etc that they
have probably spent a nontrivial amount of money into. Or postpone it for a
few years, like the ipad which was conceived well before the iphone.

------
jamesrom
As an Apple fan, this smells like the Apple car rumours. Moonshot projects to
retain talent perhaps?

Either way, AR is more suited to Apple than VR, even if more of a technical
challenge.

------
gigatexal
My BMW had a rather rudimentary HUD (they've since gotten a lot better). I
think something like that for personal use would be really cool: the ability
to look via some glasses or something at an object and see wikipedia snippets
or the like. Or turn by turn directions as I walk without looking at a phone,
pretty cool.

------
nickhalfasleep
I think Apple could leverage it's supply chain in this market very well. The
product is small, intimate, and needs to work well with a minimalist
interface. With the right sort of eye tracking, it could optimize CPU/GPU time
only where the user is actually looking.

------
Overtonwindow
Wait. I thought that was wearables? No wait, the TV, or was it cars? Oh gosh
I've lost track.

------
Someone
_" The global market for AR products will surge 80 percent to $165 billion by
2024"_

So, current market is $32 billion a year? Where is that market? Mostly
military and for professionals, I would guess?

------
dclowd9901
I think Apple's play here should be around making iPhones amazing online
clothes shopping devices. I would kill for a way to see how shoes or a shirt
would fit before buying them online.

~~~
PKop
[https://www.mtailor.com](https://www.mtailor.com)

I've used this for a few flannel and dress shirts. It works pretty well.

I initially thought Amazon has to buy this. They would dominate retail if they
can flip those last remaining shoppers for clothes who need to buy in person
in order to try on clothes.

This app has incrementally improved over time, and added products. Initially
you could only buy shirts. Now they have pants, suits and just recently added
jeans. There's a lot of potential there...

~~~
emehrkay
That tagline perfectly targets a man with certain feelings on masculinity:
"Stop Wearing Another Man's Clothing." I get that it is another way to present
the idea of having clothes tailored, but it makes me think of how I used to
trick my son into eating his all of his dinner by asking if he was man enough
to eat it all or not.

------
jordache
lets have apple get their bread n' butter stuff right first. Like how
unplugging a USBC -> Displayport cable causes my macbook to crash requiring a
hard boot, several times a day.

------
vlunkr
I'd honestly be more interested in them focusing on existing products. I'm
still skeptical of VR/AR really becoming mainstream. The best AR app so far is
what? Pokemon Go?

------
synaesthesisx
Apple will likely ship an AR device this year using the tech they've acquired
from metaio/primeSense. All signs are indicating an AR-enabled iPhone 8.

------
leecarraher
i'm excited to see what comes next for AR/VR, but without any prototype or
demonstration, it seems more like investor fodder than real tech.

------
vorticalbox
Worked so well for Goggle and Microsoft so far.

~~~
MikusR
Like with smartphones. Google and Microsoft do the hard work and then Apple
swoops in and claims to have invented stuff.

~~~
seanmcdirmid
> Like with smartphones. Google and Microsoft do the hard work and then Apple
> swoops in with something that consumers actually want to buy.

FTFY.

------
forgotmyacc
I have friends in VR and apple has tried recruiter them last year. It wasn't
for their development experience. Only their VR work.

------
dandare
Could this be just a decoy strategy from Apple while they work on something
more tangible? AR is not a smaller disc or faster processor, AR does not exist
- yet - and may very well never exist due to physical constraints (you can not
paint black etc).

~~~
gilgoomesh
I'm not sure what you're talking about. AR _already_ exists:

[https://www.microsoft.com/microsoft-hololens/en-
us](https://www.microsoft.com/microsoft-hololens/en-us)

It isn't necessarily useful but it exists.

As for whether this is a decoy strategy... Apple aren't even promoting this so
it's not a strategy on their part; it's just information pieced together from
hirings, anecdotes and guesses.

~~~
ctdonath
Makes sense for Apple to spend some resources investigating potential large &
new technologies, even if they don't produce a product. The field _will_ shift
when AR and self-driving cars and [insert intriguing vaporware here] arrive,
so Apple should take a good hard look at the impact, if not the product
potential, before they're too far behind the curve.

------
shadowmint

        One of the features Apple is exploring is the ability to take a picture and then change 
        the depth of the photograph or the depth of specific objects in the picture later; another
        would isolate an object in the image, such as a person's head, and allow it to be tilted 
        180 degrees. A different feature in development would use augmented reality to place 
        virtual effects and objects on a person, much the way Snapchat works.
    

Oh come on.

I'm completely unimpressed to hear Apple working on AR if that's the best they
can do.

It's a fabulously hard problem that no one has come close to solving; and to
be fair, if they poured R&D $$$ into it, maybe they could make something out
of it...

...but you know, since I can't use a Vive or Occulus on a mac, because of the
ancient ass version of opengl they support, and the ridiculous on going split
between vulkan and metal, I'm kind of unimpressed by the weight of their
technical prowess at this point.

I don't believe it. General consumer AR won't come from Apple.

