
How iOS 13 redraws your eyes so you're looking at the camera - bobsil1
https://twitter.com/schukin/status/1146359923158089728
======
keiferski
I am reminded of a brief story about video calls in David Foster Wallace's
_Infinite Jest_. Some excerpts:

 _Good old traditional audio-only phone conversations allowed you to presume
that the person on the other end was paying complete attention to you while
also permitting you not to have to pay anything even close to complete
attention to her...Video telephony rendered the fantasy insupportable._

 _The proposed solution to what the telecommunications industry’s
psychological consultants termed Video-Physiognomic Dysphoria (or VPD) was, of
course, the advent of High-Definition Masking;_

 _Mask-wise, the initial option of High-Definition Photographic Imaging — i.e.
taking the most flattering elements of a variety of flattering multi-angle
photos of a given phone-consumer and — thanks to existing image-configuration
equipment already pioneered by the cosmetics and law-enforcement industries —
combining them into a wildly attractive high-def broadcastable composite of a
face wearing an earnest, slightly overintense expression of complete attention
— was quickly supplanted by the more inexpensive and byte-economical option of
(using the exact same cosmetic-and-FBI software) actually casting the enhanced
facial image in a form-fitting polybutylene-resin mask,_

[http://declineofscarcity.com/?page_id=2527](http://declineofscarcity.com/?page_id=2527)

~~~
tictoc
Yeah this whole snapchat phenomenon with filters is literally out of Infinite
Jest. It's almost hilarious how exact he got the fact that people will be
talking with each other on the camera, and it will literally be a visually
enhanced version of themselves communicating with each other. I mean people
have no shame in photoshopping their entire form and passing it off as
themselves. We are a sad bunch. Except for the ones who don't comply.

~~~
aloer
I was very surprised when I downloaded zoom yesterday and noticed that an
“appearance enhancement filter” is enabled by default. This is something that
I am not used to outside of social and camera apps from (mostly) Asian
countries

Sad state of the world where the _default_ in a business related App is
essentially telling you that this is necessary. Can’t be healthy for society

~~~
fjp
Eh, it really doesn't do much but smooth skin.

I'd rather a client/vendor/coworker not be distracted if someone on the call
has big nasty pimples, and I'd rather have a coworker with big nasty pimples
feel more comfortable getting on a video call.

~~~
aloer
The existence of the feature is not a problem, that’s just where technology is
heading. For individual use it could be enabled, if desired

But it being made default and thereby de facto saying that everyone needs it
is absurd. There is no way that something like this is not damaging to society
in the long run

~~~
zimmund
I wonder, however, if smoothing skin makes for a lighter (compressed) image,
resulting in bandwidth savings. We don't need to see skin details in a work
conversation, just expressions.

Perhaps the feature should just be renamed/rebranded to something like
"smartly reduce image details to save bandwidth"?

------
userbinator
I can't help but think all this image recognition/manipulation tech being
silently applied is a tad creepy. IMHO going beyond things like automatic
focus/white balance or colour adjustments, and identifying more specific
things to modify, crosses the line from useful to creepy.

~~~
viraptor
It looks like there's a toggle
[https://twitter.com/flyosity/status/1146136279647772673](https://twitter.com/flyosity/status/1146136279647772673)

But yes, it crosses the line where it's no longer a camera (as a feature, it
does not capture reality)

~~~
sjwright
What is truth?

When the person on the other end of the call is looking at the image of me on
their screen, the physical truth is that they're looking away from the camera,
but the emotional truth is that their eyes are looking at me.

When I'm chatting with loved ones, which truth is more important? Physical
truth, or emotional truth?

Consider for a moment that an offset front-facing camera might be just as
serious a distortion of truth.

~~~
viraptor
You've lost me at "emotional truth". The camera captures a specific scene. The
result is processed to correct for lens distortions / colour / lighting
issues. But none of those changes the content.

I don't think an offset camera is distorting anything. It captures the image
it's expected to capture.

~~~
LocalH
But it doesn't capture the image the _user_ expects it to capture - when on a
Facetime call, they expect (or at least _want_ , if they think about it at
all) to be looking _directly at_ the other parties to the call. All this is
doing is correcting for _that_ , hence "emotional truth". Reality is always
_shaped_ by our emotions and our mental state, and thus there is merit to the
idea that a representation of reality that isn't _100% physically accurate_
can still be closer to what we perceive as reality.

~~~
viraptor
I see where you're going, but I disagree with the "emotional truth" idea. It
crosses the line for me: fixing eyes is on the same level as a gender swap or
cat face filter. It modifies content and meaning and I'm against this being
normalised in normal communication.

~~~
sjwright
I don't see how the eye correction is any less of a "modification of content
and meaning" than the use of an offset camera angle which it's correcting for.
Just because the offset angle is currently a technical necessity and an
"analogue" distortion, doesn't make it any less of a manipulation.

~~~
viraptor
> doesn't make it any less of a manipulation.

I think we've got a fundamental disagreement here. We'll just have to agree to
disagree :-)

~~~
sjwright
In an effort to be more agreeable, I really do understand where you're coming
from. I'm just openly wondering if there is a technology bias at play making
stuff we're already accustomed to and which feels normal to us (e.g. cameras
with rectilinear lenses capturing images from arbitrary angles) feel less
manipulative than stuff which is new and technically sophisticated.

~~~
brootstrap
what up SJ. I have to comment on you again my friend. Why are you so
passionate about this eyeball re-drawing technology? Do you video call your
family and friends all the time because you live far away?

Reading these conversations i'm going a bit crazy as you try to rationalize
why this eyeball redrawing business is a good and necessary tech. I think it's
complete trash. Is this the tech that makes apple worth billions of dollars?

Not trying to stir shit up with you. just generally curious as I have an
extremely opposite viewpoint. I am 28, prime millenial love to use apps and
phone age. I am a huge nerd and geek (clearly as we are commenting on HN lol).
I have literally never face-timed anyone except when other people are doing it
and I am nearby.

~~~
sjwright
I first used a Macintosh 512k when I was around 5 years old. I had my own PC
by the time I was 18 (a dual celeron 300A at one point) and I now spend my
days doing programming and web development.

I’m now 38 and I’ve seen my two year old son’s face light up when he sees his
mum through the phone screen.

My wife’s parents are on the other side of the planet and their
intercontinental video chats connect them to our family.

I love science, math, philosophy and photography. I should care about pixel
perfect truth. But to me the value of FaceTime is entirely emotional. I say
that when my wife looks into my eyes, I want to see her eyes looking at mine.
The fact that this isn’t how it works now is because of optical distortion.
While this isn’t a true holistic fix for the underlying distortion—that would
be amazing—it looks like a neat solution.

I’ve not seen it live and I’ll reserve final judgment till then. If it’s even
slightly noticeable I’ll absolutely hate it. But if I can see no sign of
trickery when staring into my wife’s eyes, it might just make the world a
happier place, even if only a tiny bit.

------
LocalH
I fail to see how this is creepy (outside of potential uncanny valley issues
in edge cases). There is a toggle to disable it, and this is something that
most average non-savvy users would either want by default, or wouldn't even
notice happening (because the end result will look natural to most).

I don't buy the slippery slope argument here, _yet_. All they're doing is
shifting eyes and nose to compensate for physical offset from the camera.
Seems like Animoji would be _more_ creepy, as it's capturing your entire face
movement.

~~~
phalangion
> I don't buy the slippery slope argument here, yet.

That's how the slippery slopes get you. They look all gentle and inviting at
the beginning like you can totally stand on them no problem, then the slippery
gets you and you're sliding right on down.

~~~
moneytalks
The real problem with talking about slippery slopes in tech is that we're
already on an unrecoverable slide into the unknown. So the question is just
"does this new thing make the slide even faster?" Not, "we're on stable ground
and we need to be careful about beginning a slide."

There's no going back and we have no idea where we're headed.

------
bobsil1
Before/after images:
[https://twitter.com/WSig/status/1146146914985009154](https://twitter.com/WSig/status/1146146914985009154)

~~~
aetherspawn
I really want to see this, but I opt out of having a twitter account :(

~~~
3JPLW
Hunh? You can view tweets just fine without twitter accounts.

~~~
kabwj
This also happens with posts from Facebook often. People just want to show the
world how virtuous they are by not having accounts in popular networks, so
they don’t even click the link but complain they can’t read the content.
Reminds me of this:

[https://twitter.com/1990sLinuxUser/status/97350916902105089](https://twitter.com/1990sLinuxUser/status/97350916902105089)

------
crazygringo
Huh. I was on a team that experimented with this a few years ago, and the
results were so uncanny-valley that we quickly stopped going down the path --
despite it looking "real", people wound up looking like slightly possessed
demons because there was something "just off" about their eyes. I mean, it
seriously freaked us out.

I'm really wondering if/how iOS avoids this, or if it took a ton of extra
work.

~~~
moftz
I'm just hoping that it will correct my eyes so I can watch TV in the
background while my girlfriend thinks I'm looking at her during the videochat.

~~~
valine
I don't think it works like that. iOS is adjusting your eyes by a specific
angle to account for the distance from the screen to the camera. If you look
away it will appear to the person on the other end of the call that you are
looking away.

------
jakejarvis
I understand why this feels creepy in our tech bubble, but I think it's worth
noting how popular apps like FaceTune have become the past few years [0] [1].

I'm certainly not defending this trend and I think it's incredibly unhealthy —
especially for the average teenager who's already naturally self-conscious
about their appearances. But a minor eye correction will be peanuts in the
eyes of this crowd (no pun intended) compared to the amount of processing that
most of their Instagram and Snapchat photos go through before being uploaded.

[0] [https://www.theguardian.com/media/2018/mar/09/facetune-
photo...](https://www.theguardian.com/media/2018/mar/09/facetune-
photoshopping-app-instagram-body-image-debate)

[1] [https://abcnews.go.com/GMA/Living/photo-retouching-apps-
affe...](https://abcnews.go.com/GMA/Living/photo-retouching-apps-affect-
impressionable-teens-social-media/story?id=56928554)

------
tomxor
Weird, what happens when you are actually trying to look at something else?

~~~
syllable_studio
Insert image of Clockwork Orange film scene where a machine holds your eyes
open and forces you to look.

------
scotty79
I was wondering why skype and any other video chat don't do this five years
ago, or more.

Probably we needed deep learning to do this well.

~~~
ktpsns
Correct. Apple certainly "reused" the infrastructure they established with
realtime facial recognition and manipulation, all encapsuled in ARKit
([https://developer.apple.com/augmented-
reality/arkit/](https://developer.apple.com/augmented-reality/arkit/)).
Furthermore, these compute intensive codes are accelerated with special chips
("AI chip", "Neural engine", etc.). That's actually quite novel stuff which
wasn't there five years ago.

------
suyash
Sorry I'm not seeing it, can someone explain in bit more detail what is going
on?

~~~
rcv
One of the drawbacks of video chat is that the camera on your device is offset
from the face of the person that you're chatting with (typically your camera
is on top of your screen, rather than embedded in the middle of it). This
leaves people chatting with the impression that the person they're chatting
with is staring at their neck.

This new iOS video chat software modifies your video stream to correct for
this camera/screen offset by _redrawing your eyes_. The end effect is that the
person you're chatting has the impression that you're looking into their eyes
rather than at their neck.

The effect is subtle, but can be made more obvious by putting a straight wire
in front of your eyes. With the software feature disabled, the wire appears
straight. With the software enabled, the wire appears bent.

~~~
suyash
thanks, I get it now and see it now :)

------
lquist
This is such an elegant solution! Previous discussions about how to solve this
issue revolved around putting a camera in the center of the phone behind the
screen!

------
bartcobain
The Simpsons did it again:
[https://imgur.com/a/oGSkSgN](https://imgur.com/a/oGSkSgN)

------
ben_w
Nice to see this feature. A few months ago I made a demo app for my own
personal use which did this exact thing, just to see if it would work. As I
was still learning AR at the time, I didn’t fully understand the API well
enough and I just had a point cloud with loads of holes in it, but it still
showed it worked, and made me wonder why Apple hadn’t already done it.

------
scarejunba
I love this. What a casual use of futuristic technology to make what people
want to express real. Great work from Apple.

------
jokoon
I had a video call interview and I was quite impressed how that woman always
seemed to look at the camera, because I really thought she was looking at me
and not me on the screen.

I told her, but she answered that she was not really staring at the camera. I
guess it makes sense now.

------
Waterluvian
Off topic: anyone know why 100% of the time I load a Twitter post from HN it
loads "something has gone wrong" and I must refresh the page once to see the
tweet?

------
kazinator
I asked for this:
[https://news.ycombinator.com/item?id=12165812](https://news.ycombinator.com/item?id=12165812)

:)

