
Making Blurry Faces Photorealistic Goes Only So Far - MindGods
https://spectrum.ieee.org/tech-talk/computing/software/making-blurry-faces-photorealistic-goes-only-so-far
======
PragmaticPulp
Everyone in the ML field understands that ML-assisted upscaling doesn't
produce output that accurately represents the original full-resolution image.
It produces output that humans perceive to be realistic-looking, or free of
traditional upscaling artifacts.

While this is obvious to anyone familiar with the technology, it's difficult
to explain to casual observers. The image output looks real. It feels like it
could be real. It's free of traditional scaling artifacts that would trigger
suspicion. Without additional explanation, it's easy to see why casual
observers would assume the hallucinated upscaled version is an accurate
representation of the original image.

Historically, police sketches and blurry surveillance images are obviously low
quality enough that people inherently know they're approximations. The problem
with these ML hallucinated upscaled images is that they look and feel real
enough that they bypass people's suspicions. We can try to present them as
"Here's what the suspect might look like", but when they look like a full-
resolution photograph, people will simply assume that it's _exactly_ what the
suspect looks like.

~~~
wmf
One obvious problem is creating only one output face when there are many
possible faces that match the low-res version. A tool that turns a low-res
face into, say, 16 maximally-different upscaled versions doesn't suffer from
the same level of false certainty.

~~~
notdonspaulding
Doesn't it?

You arbitrarily picked 16 possible photorealistic faces out of a total
solution space of what? Millions?

Wouldn't the balance of probability be on someone in the general population of
humans more closely resembling one of your 16 candidate images than any of
your candidates resembling the Ground Truth image?

Doesn't the problem get both better and worse as you scale your N up from 16?
That is, it would be better because _one of your candidates_ is more likely to
match the Ground Truth, but it would be worse because you've also widened your
net for catching false positives?

~~~
rabidrat
I saw the blurry picture of Obama and my brain thought "I think that's Obama".
When it was 'enhanced' it became a (white) person that I would never recognize
as Obama. In fact I did not recognize them at all. The image may have been a
more probable person overall, but clearly not a more probably famous person.

Which makes me wonder, can we train a model on only famous people, weighted by
their relative famousness?

~~~
ben_w
> can we train a model on only famous people, weighted by their relative
> famousness?

Well, yes, but why would you want to?

~~~
rabidrat
So that we can take a picture of ourselves and have it tell us which famous
person we most look like when blurry?

~~~
bryanrasmussen
The hit kids app of the summer.

------
speedgoose
I would say the technology has some issues. For example when you don't include
enough black people in your training dataset, probably by simply not thinking
about it, it can make the algorithm a bit racist. Example :
[https://twitter.com/Chicken3gg/status/1274314622447820801](https://twitter.com/Chicken3gg/status/1274314622447820801)

~~~
scrooched_moose
The Oprah one is probably worse:

[https://twitter.com/Truttle1/status/1274361095285886982](https://twitter.com/Truttle1/status/1274361095285886982)

~~~
cycrutchfield
I mean, it's amusing, but the face is not aligned the same as the training
data so what would you expect?

~~~
GuiA
As a computer scientist? Garbage.

As a layperson? Magic, because that’s how these things are marketed by
startups and journalists.

------
stabbles
Inverting a non-invertible operator impossible, study finds.

~~~
jaredtn
Sure the theory is nice, but can I see some empirical evidence? :)

------
ttul
With a large enough database of potential faces (see VKontakte), you might be
able to use this tool to upscale blurry images and match them to a short list
of candidates. Other intelligence could then lead you to the actual person in
the blurred image. Scary implications.

------
_trampeltier
It might be less true for pixeled videos. I know from some IR camera companys,
they use very small movements from the camera to calculate the picture in a
higher resolution. And I think, the first picture from a black hole use also a
kind of this technologie

------
owenshen24
This seems pretty reasonable; I don't know if anyone was claiming otherwise
throughout these upsampling results?

~~~
rasz
this is not upsampling

~~~
owenshen24
That's true; I was too loose with my terms.

------
robertlagrant
> For starters, Rudin said, “We kind of proved that you can’t do facial
> recognition from blurry images because there are so many possibilities. So
> zoom and enhance, beyond a certain threshold level, cannot possibly exist.”

Least needed "proof" ever.

~~~
steerablesafe
It's also not a proof.

------
hirundo
Instead of blurring faces in photos of protests, Google Street View, etc.,
blur them and then upscale them, so they don't have the jarring blur effect
but still anonymize.

------
m3kw9
Try a grid of 2x2. Is called ambiguity the more you hide it. The idea is so
simple on why it wouldn’t work even on trained models.

------
elwell
Waiting for the CSI episode:

"Ok, zoom in... alright, now enhance that... now upsample using AI trained on
criminal database... "

~~~
Chazprime
They already did that in _No Way Out_ , way back in 1987:

[https://youtu.be/0zLL_XdqxmQ](https://youtu.be/0zLL_XdqxmQ)

------
tyingq
Depends on the blur used, right? Straight pixelation is very lossy, but
gaussian is less so.

~~~
A_No_Name_Mouse
Right. As long as the number of pixels remains the same it should be possible
to remove gaussian blur almost completely. The information isn't lost, it is
still there, but smeared out over a larger area.

Also I don't understand why the first Mona Lisa results in a picture that when
pixelated again, wouldn't produce the original pixelated picture. It is as if
the create a face inspirated by the original, but not one that could ever be
the original.

~~~
hatsunearu
You actually do lose quite a lot of information, particularly so if you have a
limited bit depth/other quantization noise.

------
aantix
Slightly offtopic - who has the best video upscaling algorithm out there?

------
BurningFrog
So since this algorithm can't be used to reliably recover faces from blurry
photos, it won't be.

It can still be useful for entertainment purposes.

------
ithkuil
Can such a model be used to compare a real picture of a suspect with a low res
picture and produce an objective measure of similarity (i.e. not only raw
pixel similarity but a similarity measure that actually takes into account how
actual human faces downscale, possibly resilient to minor rotations?)

------
etaioinshrdlu
The increasing politicization of the machine learning field makes me grow
tired. It makes me want to work on things without political implications.
Something like electrical engineering. Does anyone else feel this way?

