
How ARKit 2 works, and why Apple is so focused on AR - awat
https://arstechnica.com/gadgets/2018/06/arkit-2-why-apple-keeps-pushing-ar-and-how-it-works-in-ios-12/
======
paul7986
AR I think needs killer apps and utilities. The AR measuring app is a good
example if it works accurately.

Other AR apps/ideas I think would be killer are...

\- AR historic view: wave phone or point glasses at a location and see how it
looked year’s ago via pics & a 3D model or maybe a mix. Make me feel I stepped
back in time and have a Disney like feel/experience to it.

\- AR night time/day time view: At nighttime view in glasses how it looks
during the day

\- AR Linkedin/Facebook: Know the strangers name your talking to and all their
public iNet info via glasses view

~~~
monsieurbanana
> AR Linkedin/Facebook: Know the strangers name your talking to and all their
> public iNet info via glasses view

A killer app indeed; it'd kill AR. At least I like to think we aren't so far
gone that as a society we can't see _how wrong that would be_.

~~~
marssaxman
I am apparently gone because I've always wanted that - in fact it's the only
intelligent-assistant application concept I've heard which is appealing. What
is so wrong about it?

~~~
alasdair_
When you can instantly see the salary data and criminal history of everyone
you meet, we can very quickly spiral into dystopia.

It's also likely inevitable. Every time you tag someone in a Facebook photo
you are helping this happen.

------
giancarlostoro
Seems like AR is a lot more approachable than VR in that you don't have to
strap on so much gear to immerse yourself into a Virtual Based / Enhanced
reality. Even Snap seems to be invested in AR if you think about it, how many
snaps don't feature some augmented visuals? If people go crazy over that to
the point that Facebook, and even iOS steals that type of AR functionality,
there must be more to AR than meets the eye.

To a computer geek it may not seem impressive now, but to the average person
it casually sneaks up into their everyday life.

~~~
ghaff
It seems as if VR is one of those things where getting to 90% isn't very
useful for most people and even if you get to essentially 100% (wireless,
usable with mainstream processing power, comfortable lightweight goggles), it
may not be all that interesting outside of some niches like certain types of
gaming or virtual worlds. A lot of the time people don't want to be immersed.

AR, on the other hand, is very amenable to incremental development. You don't
need a Google Glass 2.0 with all the technical and social challenges
associated with that. A phone that you can point at something and get
information about it is already useful.

~~~
oldcynic
I think VR is forever destined to be a niche sideline. Seems like I've been
hearing that VR is about to take off every few years since the 90s. It
resolutely fails to do so and aside from some "wow" at tech demos and games
there's no killer VR app that makes it a must have.

AR is much more instantly obvious to all with things like Night Sky and
augmented maps.

~~~
ghaff
In some ways, you could almost argue that VR (and immersive experiences
generally) has garnered less interest as it's become closer to reality. The
Kinect is gone. The Wii fad is over. The VR devices that do exist have pretty
much highlighted the difficulty of creating fully immersive experiences,
especially in a typical home environment. And, as you say, there's really been
nothing that makes the typical or even not so typical person go "I gotta get
some of that VR."

I suppose we may get there someday but it may also be one of those things that
it seemed logical to assume people would want (and many thought they did), but
they mostly don't.

------
tootie
The first killer app for AR was probably Word Lens (translating images of
words in real time) and that debuted in 2010. Then Pokemon Go 6 years later. I
can't think of anything else that has caught on. Seems like a really exciting
paradigm, but no one has figured out anything useful to do with it.

~~~
tbassetto
Has World Lens ever caught on though? Honest question, I don’t have any data
but I have never used it nor seen anyone else using it.

~~~
dividuum
Its now built into google translate (at least on my Android, no idea about
iOS). It works offline and I've used it occasionally while staying in
Hongkong. Was pretty useful.

~~~
lathiat
I use it a lot when travelling. (In google translate) it’s great.

~~~
ghaff
Google Translate is a great example of how incremental iterations of AR can
still be useful. It's on a phone and not some fancy glasses and the
translations aren't really very good but it's still really handy especially if
you can't even read the script of the language in question.

------
ForrestN
I get most excited about VR when I think about the physical ways we currently
augment reality with ugly physical objects.

If AR becomes truly ubiquitous, I hope that signs and signals can most fade
away. Imagine walking through a city without street signs or advertisements or
stoplights, or walking through an art museum without any labels on the wall,
or an airport without screens and arrows pointing you around.

I’m also interested (but also concerned) from a consumer perspective—Amazon
etc. will offer Shazaam for literally any object. Like her dress? Buy
instantly. That house is tinted green because you qualify for a loan that
could purchase it, but the newer one next door is tinted red. Follow the
glowing footprints around that corner if you want the slice of cake that
person is eating in the park.

Most of all, I want to take a walk in the woods and be able to identify any
tree, fungus, flower or bird instantly and beautifully. AR will hopefully let
us take intelligent guided tours of almost anywhere. Learn about architecture
as you walk to work, trace important sites out the window of an airplane onto
the ground below, see a bustling Mayan market animated over what is now a set
of rectangular fields.

~~~
JofArnold
In that future world it’s going to be pretty tough getting around if you’re
homeless and/or can’t afford AR...

~~~
ForrestN
We don’t have much in the way of pedestrian navigation right now, apart from
crosswalks and such. But you’re right that perhaps crosswalks that cars must
yield to (California style) should be the last to go.

------
ngngngng
The only thing I want to be able to do in AR is walk around historical sites
like the Colosseum and hold up my phone to see what something looked like
millennia ago as I walk around.

------
thosakwe
What would really take AR to the next level is something like Google glass,
where you’re essentially always in AR.

The only downside is that you’d have to wear extra glasses, which might not be
ideal if you already wear prescription, and that the technology is way too
expensive for most people.

But overall, I don’t think an AR boom is that far away.

~~~
slantyyz
Well, if sci-fi is the model of where the tech is going, we'll probably be
using contact lenses or implants when the tech "crosses the chasm", so to
speak.

------
kumarvvr
It's really interesting Apple has put the overkill face ID sensor in the
iPhone. I'm sure future iPad's will have the same too, possibly with even
better hardware.

I also believe accurate tracking of the eyes of the user will have tremendous
impact on the applications that can exploit AR.

AR is a tremendously promising field and Apple has gone the easy way, to
capture the market and developer mind share. Oculus Rifts' and HoloLens' have
failed to adequately capture developer mindshare as on date.

Just look at the measure app. It's a tremendously useful app. Though I know it
will not have much accuracy, it will satisfy most use cases for normal users.

I am excited about the future of AR apps. I can see a lot of potential
applications in Engineering, Medical, Civil surveying, etc.

------
tsenkov
Not too many AR apps have succeeded, only because the massively-available AR
hardware is not here, yet. The phone with a camera is not it. You can only do
that much within that paradigm.

The moment (hopefully it's coming) we have (relatively) normal-looking glasses
capable to perform like the Magic Leap's headset (haven't tried it - only
judging by the publicly available info and investments), that's when handheld
devices become obsolete and that's when we will see AR pick-up and win.

~~~
ghaff
You can do a lot with a phone and there's no social stigma associated with
walking around with a smartphone. It's an open question for me whether
Glassholes 2.0 become a thing even once the tech is there. (Certainly there's
a role in niches like doing repairs where information can be overlaid but I'm
not convinced it will become a thing when just walking around.)

~~~
coralreef
Viewing the world through your phone's screen/camera lens is just not a great
experience.

Having stuff just "appear" is way better. Remains to be seen how this will
play out in the short run, but the fantasy is something like smart contact
lenses.

------
codesternews
Surely it look very cool in demos and on screen but It was just hype and will
be just hype until there is major breakthrough because of below reason.

\- It is because you can not use AR everytime and results are not consistent
and not work in all environment unlinke touch which can be used everytime
perfectly.

\- Battery and heating problem with AR on phones.

\- Intreaction is major problem.

Intreaction with glasses and all those things is very akward and really not
comfortable.

I think we have to go long way to have enable the AR.

------
edko
What I find disappointing is that Apple is leveling the field against
independent developers. Was there really a need for an official measurement
app, to destroy the market of the ones that are already on the app store?
What's the deal with USDZ? It looks like it will be fundamental for developing
AR content for apps on iOS, and they give the market away to Adobe and their
paid solution, that is not even finished. Sure, it is an open specification so
"anybody can compete", but, if Adobe, with all its resources, gets such a head
start, how can indies compete? These two things, to me, are signals that Apple
intends only the big players to participate, and, as an independent developer,
discourage me from investing on AR development on iOS.

------
nubb
Does anyone know of a decent AR headset for someone to hack around with?

~~~
gnl
Project North Star from Leap Motion seems like an affordable DIY solution if
you don't mind getting your hands dirty.

[http://blog.leapmotion.com/north-star-open-
source/](http://blog.leapmotion.com/north-star-open-source/)

------
mrfusion
Could the arkit functions be useful for robotics?

------
mrfusion
I’m really excited by this stuff but I can’t think of a useful or interesting
project to do with it. Any ideas?

~~~
anonfunction
Translate foreign signs in place, augment plane tickets with realtime flight
details, see products in your house before you buy them... just off top of my
head.

~~~
kitsunesoba
I think the concept of “live” paper via AR will be much more popular than most
expect. Tickets, like you mentioned, are a great example. Other things could
be the ability to flip through different revisions of a printed document or
the ability for multiple people to mark up and annotate the same paper
document.

------
xoa
I think they missed one of the more valuable potential ones that I'd like to
see, and it would be yet another extension of something Apple has made a
pretty core selling point: privacy and security, specifically enhancing Face
ID with a very natural coercion code type system. If a future Face ID 2.0/3.0
system is capable of fast and wide field enough facial and
expression/sentiment recognition, then it could be possible to not merely have
a binary lock/unlock based on face and attention, but add in
expression/sentiment as something else as well, and give it other levels of
transparent unlock. So it'd be possible for users to make a full unlock
require a specific expression, or alternately (or additionally) have it
recognize another expression and trigger other actions. This could at a
minimum include disabling Face ID (similarly to pressing a button 5 times) or
even triggering an erase, but even better could also leverage Apple's
improvements to their granular data encryption system to "unlock like normal"
except that apps and content the user had marked as sensitive would not be
decrypted and simply not show up on the system in any way at all.

Of course this could be done with PIN codes too and I hope that would be an
addition, but it'd particularly natural using faces and having the system be
able to actively look for signs of distress. Apple is in a real position to do
this in an extremely user friendly way and it'd be a huge boon for personal
privacy and security.

In fact leveraged even farther and using more data points (even just GPS and
time of day) I think Apple could make a framework that could go beyond just
privacy/security and into helping everyone better handle their usage of mobile
devices (which is something else they are clearly at least aware of). Imagine
being able to create "views", where underneath granular key release is
utilized to allow a user to make any arbitrary set of apps/data be visible or
unavailable, and then be able to assign arbitrary trigger conditions for what
view the phone shows at any given time. It'd be possible to have a device with
personal entertainment, social media, communications and such and also work
related material and hobby material and create hard barriers between all them.
At work entire or certain times of day entire bits of distraction would simply
vanish, totally transparently and without effort, while at home vice versa
could happen. While traveling everything but a few key travel apps could go.
Users could set it so that financial apps could only be accessed easily at
secure geographic locations. All under the control of the owner, and it'd help
with self control and information overload, allowing owners to pick the right
set of data to take their attention for the right setting. And of course it'd
ensure that even if threatened sensitive material could have been made
unavailable ahead of time.

I'd love that, I think it'd leverage a lot of Apple's strengths and existing
frameworks anyway, and that it'd be a real step forward for getting a handle
on the ever increasing amount of stuff being thrown at us.

~~~
macintux
“How did you accidentally erase your phone?” “I sneezed at the wrong time.”

Apple makes secure devices but doesn’t go for the type of customizable
security you’re interested in, I imagine because the number of people
compelled to unlock their phone under duress is so small as to be negligible.
I hope.

~~~
xoa
> _“How did you accidentally erase your phone?” “I sneezed at the wrong
> time.”_

This seems like an exceptionally and unfairly reductionist, close minded and
bad faith reading. I only gave that as one example option, and if you actually
consider it it's easy to imagine that there could be restrictions like "You
must have Backups on to enable this option". Or it could just not go that far,
it wouldn't subtract from the overall feature at all.

> _I imagine because the number of people compelled to unlock their phone
> under duress is so small as to be negligible._

I don't think that's a safe assumption at all, particularly outside the first
world (or even in the first world considering how borders are hardening).
Apple like anyone (more then many corps in fact given their startup-type org
structure) must prioritize features and build up, and considers new things as
the world changes and as their foundations continue to improve. They didn't
always have FDE at all. They didn't always have the network lockdown to
discourage theft. They didn't have biometric auth, until they did. They spent
a lot of time and effort building up their HSM implementations, it wasn't
there day 1. They are only now in iOS 12 apparently planning to push out a
lock down of the wired port. Etc etc.

Finally, you completely ignored how these same measures would fit into another
major current public issue and just announced Apple effort: allowing people to
get "digital addiction"/information overload under control. There are multiple
compelling lines here of which privacy/security are just one angle.

[EDIT] - Edited to add an example of the above from just this past WWDC: HN
discussion on "iOS 12 introduces new features to reduce interruptions and
manage Screen Time":
[https://news.ycombinator.com/item?id=17230469](https://news.ycombinator.com/item?id=17230469)

"Views" or however Apple chose to convey/brand it would be a great extension
of that. "While I'm doing X, I only want to see Y" with a nice GUI is
something that makes intuitive sense, it's how our lives tend to be ordered
anyway IRL. GUIs got started by leveraging real world systems and symbols, and
while that of course can be overdone and be too restrictive I don't think it's
played out yet either at all. Lots of people don't bring work home with them
or home to work but rather try to maintain a separation. Why shouldn't our
digital devices be capable of automatically supporting that partitioning?

