
Hacking the Brain with Adversarial Images - imartin2k
https://spectrum.ieee.org/the-human-os/robotics/artificial-intelligence/hacking-the-brain-with-adversarial-images
======
debatem1
Seems like an odd approach. The interesting thing about robust adversarial
examples is that they fool computers but not humans. Something that fooled
humans but not computers would seem to be the right next step. If you fool
both it seems more likely that you just made an image that looks like what it
classifies as.

~~~
stanfordkid
Yeah my first thought was - well dogs and cats look kind of similar under
fairly simple transformations.

What would be interesting is if you could establish a correspondence between
visual and genetic similarity... neural networks provide a distance metric.

------
lambada
This reminds me strongly of the BLIT series of short stories, in which a
category of images is discovered that cause processing problems ‘glitches’ in
the human brain with fatal results.

[http://www.infinityplus.co.uk/stories/blit.htm](http://www.infinityplus.co.uk/stories/blit.htm)

That’s the first one, others are available online too.

~~~
taneq
The other work of popular fiction that gets referenced in these cases is, of
course, Snow Crash.

------
ccvannorman
TLDR: If you modify pixels enough, you confuse humans, especially when you
only let the humans look at the tiny blurry image for 70 ms.

 _rolls eyes_ I feel like this article totally misses the point about why
adversarial ML tactics are interesting. The fundamental reason they work is
that _computers don 't have any abstract notion of a cat, a toaster, a banana,
or even gravity or matter._ THAT is why they're easily fooled.

To stack up two adversarial images and say, "Look we fooled humans too, OMG
humans are just like computers _! " is like saying, "We erased the lines on
the road and both self-driving cars AND humans got confused, OMG humans ==
computers!"

_I know the article isn't saying humans == computers, but .. come on .. I'm
not seeing any merit to this investigation

------
YeGoblynQueenne
Woooah. Huuge assumptions here.

First of all, when we say that a classifier "misclassifies" an image, we have
an agreed-upon definition of what that means: it means we have an image, we
have a target label for it and the classifier assigns it a different label.

What exactly does it mean that a human misclassifies an image? More
specifically, when the article says that "humans think that they're looking at
something they aren't", what in the world does that mean? I mean, I have to
assume a confusion between "looking", and "seeing". I can't really believe the
article is saying that when I'm looking at the top right image I'm not
actually, you know _looking_ at it; but instead ...looking outside the window?
Or what?

On the other hand- seeing? Really? Who knows what it is that anyone else is
seeing? Who knows what _is there to see_? Especially when it comes to an image
specifically manipulated to be confusing, as opposed to a real-world image?
What is the ground truth here, for the image on the top-right? Is it a cat,
just because Google reserachers say it's a cat and it's fooling my senses into
thinking it's a dog ish? Is it fooling my senses just because the Google
researchers say it is? Google researchers are the final arbiters of objective
visible reality, now?

To be more precise, who is to say that that image can just be a "cat" or a
"dog" and nothineg else? Because, you know, the first thing I thought when I
saw that image on the top right was "that looks like a jumbled mess made with
bits of an image of a cat and an image of a dog", or something along those
lines.

Of course, if you sit me in front of a computer with two buttons, one for
"dog" and one for "cat" like in the researchers' setup... well, then you can
force me to misclassify the image all you like. But what does that _prove_?
Besides the fact that if you force me to choose one of two things, without
knowing what you think is the "right" thing to choose, I'll choose the "wrong"
thing a lot of the time?

------
arkades
The subsequent images and conclusion that adverserial attacks against multiple
models would be effective against humans seems tautological. Individual models
may be unique, but multiple models are related in that -human- classifiers are
the final source of truth. Anything that works against all of them is
indirectly targeting the human classifiers that unite them.

------
lowracle
How far fetched would it be to assume that in the future, facebook would start
building a neural network for each of its users, training it to like the same
content as you ? With the amount of data generated by the endless scrolling,
you could build a fairly accurate neural network of which content produce
which emotion, and then generate adversarial images for targeted advertising.

~~~
AnIdiotOnTheNet
At some point that neural network could just take over your social media
presence and no one would ever know, because it's essentially your clone.

------
yters
So if I make a cat look like a dog, people think it's a dog. That's amazing
and totally unexpected.

~~~
saulrh
The point is that the human brain is just as vulnerable to the adversarial
attacks that people are claiming will be the downfall of ML. It's like the
people that scream about people shining lasers into the cameras of autonomous
vehicles - yeah, sure, if you did the same thing to a human driver you'd get
exactly the same crash, why are you suddenly worried about this?

And it's more than just "making it look like a dog". If you look at it for any
length of time it's obvious that it's a picture of a cat with some noise on
it. But if you only have 100ms to respond to it you are _guaranteed_ (>95%
misidentification) to say that it's a cat. Now what happens if you take this
and make your car look like a tree? Someone could crash and that would be
terrible! Cancel driving, tear up the roads, ban wheels, cars were a mistake!
/s

~~~
matte_black
It’s not the same. Shining lasers into eyes is not scalable. However, placing
a little sign at the side of a road that causes self driving vehicles to
swerve out and crash violently, killing the occupants, can be done in mass.

~~~
saulrh
If you make the sign red and octagonal you could even kill human drivers too!
Or maybe if you slapped an illusory-motion pattern over something that looked
like a dog, then people might swerve into the opposite lane to avoid it,
double kill! Shock, horror!

Human vision is slightly more robust solely because it's had more time to go
back and forth with adversaries. Nothing prevents ML from reaching the same
levels of safety. Nothing prevents you from deploying attacks against humans
that're identical to to the attacks against artificial systems.

------
m3kw9
How is this news? It’s wraping AI terminology over what we would call over
since dawn of human, a disguise.

------
trevyn
Article subheading:

> _Researchers from Google Brain show that adversarial images can trick both
> humans and computers, and the implications are scary_

So the IEEE is telling me how to feel about this article in addition to
presenting the facts. One might even consider that "hacking the brain with
adversarial text". :-)

> _A worrying possibility is that supernormal stimuli designed to influence
> human behavior or emotions..._

Sooooo, like clickbait headlines? :-)

~~~
TheOtherHobbes
Exactly like clickbait headlines, political posturing, advertising, and
propaganda - all of which are misleading stimuli designed to elicit a specific
behavioural and cognitive response.

Brains are very easy to hack. The idea that we're reliable exemplars of
rational objectivity and rigorous self-awareness is nonsense.

------
kerng
This is interesting. Could this be a way to watermark images automatically
before uploading to the "cloud" to prevent AI processing by large corporations
(e.g. Facebook)? Main problem seems that the AI can be adjusted down the road
to eventually fix it.

------
ouid
there's a lot of bullshit in this article.

To start, it should be clear to everyone that it is possible to actually
transform a picture of a cat into a picture of a dog. The argument that the
author is trying to make is not, in fact, that the human brain is hackable
with adversarial images, but that the kinds of strategies which allow you to
hack a wide range of deep learning models (to turn cats into dogs) require the
introduction of "features" that a human would recognize as being doglike.

The leading image is not evidence of this, however. We have to keep track of
the what figures correspond to which claims, because the article jumps around.

The first figure shows us a mask which people agree turns a cat into a dog.
This is not evidence of any claim because such masks are guaranteed to exist.

The second figure shows a mask which people don't even detect which causes a
computer to claim that a panda is a gibbon. This is basically the null
hypothesis of the paper, it demonstrates that you can attack a _single_
computer with a subtle mask.

The third figure shows us an example of an attack that is robust across deep
learning models and also to perspective shifts. The argument being defended by
this figure is that humans recognize the mistakes that the machine is making
as being non-superficial features of laptops and toasters. The author
undermines his own argument here, by requiring that the attack be robust to
perspective shifts. The point he/she is trying to make is that in order to be
robust to many neural nets you have to include non-superficial features, but
then convolutes that claim by also requiring the features to be robust to
other types of transformation as well. Does the "featuriness" of the attack
come from the multiple neural network requirement or for the transformational
robustness requirement? The answer is unknown.

Furthermore, I think this argument falls short pretty quickly if you simply
hide the computer's answer. Would you guess that the computer guessed laptop
for the first image, even if you knew it got it wrong? How far down your list
would that have been? Or toaster for the second? The features offered in the
second image definitely look like they have some features that are similar to
the corner of a rounded metal cube, but this _is exactly_ a superficial image
statistic which is robust to change of perspective transformations. My
classifier might as well be looking for that exactly. This is not evidence for
the authors case.

The fourth and fifth figures are more interesting, but still do not provide
any evidence in defense of machine learning.

It's important, for some reason, to point out that the accuracy on image tasks
is NOT measured linearly with error rate. 99.9% accuracy is 10 times better
than 99% accuracy. The difference between 65 and 75% is about 1000 times less
than the difference between 75% and 99% Allow me to submit, without evidence,
that humans can differentiate between a cat and a dog with >99% accuracy when
given as much time as they need to make a decision about whether or not they
are looking at a cat or a dog (with no adversarial mask applied). This means
that the difference between human performance on the task without the time
constraint, versus human performance on the task with the time constraint, is
about 1000 times larger than the difference between human performance on the
task with the time constraint and the task with the time constraint and the
filter.

This means that if you really want people to fail on the cats versus dogs
task, spending time coming up with the filter to apply after you've somehow
managed to present the image that you're trying to get the person to
misclassify for 1/20 of a second, followed by a noise mask, is >100 times less
valuable at getting them to misclassify the picture than figuring out how to
add the time constraint. Not to mention the fact that transformations that
_actually_ take cats to dogs and vice versa are necessarily possible, and that
differentiation between cats and dogs is an image recognition task with a much
clearer limit on human performance than differentiation between cats and
laptops, which we are nevertheless _amazingly_ good at.

Cats are very close to dogs, and it seems like this paper has completely
forgotten this fact, and is deeply surprised to learn it.

Overall this is a very poor defense of deep learning.

------
matte_black
With attacks like this it would never be safe to use image recognition in any
applications where human life depends on accurate results.

~~~
dgacmu
Because my favorite hobby is modifying my liver so that the biopsy picture
confuses the cancer detection algorithm. :)

~~~
matte_black
I’m thinking more along the lines of cars, surveillance, heavy construction
equipment etc.

~~~
Filligree
[https://xkcd.com/1958/](https://xkcd.com/1958/)

~~~
matte_black
If this is the defense, it’s paper thin. Human drivers can easily reason about
fake lanes, or fake pedestrian cut outs, or a sign somewhere that it shouldn’t
be.

You also don’t need an abundance of murderers.

