
Audio Adversarial Examples - moultano
https://nicholas.carlini.com/code/audio_adversarial_examples/
======
aeling
Prior discussion / link to arxiv:
[https://news.ycombinator.com/item?id=16220376](https://news.ycombinator.com/item?id=16220376)

------
bcheung
There is a similar attack against image classifiers.

So is this just a proof of concept or can this be exploited in the wild?

Based on my understanding you need to have access to network itself (weights,
baisses, activation function, architectural topography) to pull off this kind
of attack. Doesn't seem like this could be easily be duplicated as an outside
agent.

~~~
chestervonwinch
To generate adversarials, you really just the Jacobian of the network, which,
without the network architecture, etc., would require estimation via finite
differences which would require running the same signal as input multiple
(lots! for large signals) times with minor perturbations to each component. I
think whether or not this is feasible depends on the circumstances. Also, I
could be wrong about everything because I'm not super current on this stuff.

~~~
whataretensors
You don't need access to the network, you can just build another one.

[https://arxiv.org/abs/1602.02697](https://arxiv.org/abs/1602.02697)

"Our attack strategy consists in training a local model to substitute for the
target DNN, using inputs synthetically generated by an adversary and labeled
by the target DNN"

~~~
chestervonwinch
that's clever and sneaky!

------
jchw
I wonder if it's possible for an attack like this to work against the human
brain.

~~~
colechristensen
Flashy cartoons give some people seizures.

I'd bet they could be engineered to affect a higher proportion of the
population.

~~~
taneq
I'm pretty sure some alarms use a strobe light at a particular frequency
combined with a loud 2kHz tone to discombobulate intruders. That's more
sensory overload than an adversarial signal, though.

I'd expect a malicious adversarial example to be more like the ones discussed
in Snow Crash or BLIT (
[https://en.wikipedia.org/wiki/BLIT_(short_story)](https://en.wikipedia.org/wiki/BLIT_\(short_story\))
)

------
ttul
I can’t wait to try this on my friend’s Google phone.

~~~
aeling
I believe this requires the attacker to have access to the ASR neural net's
weights, so Mozilla's seems like the only popular framework that's vulnerable
right now (not that I'm opposed to them keeping things open).

~~~
mbebenita
This is exactly why we need to keep ASR tech open. These kinds of issues
affect DNNs in general, and other ASR engines based on them are also
vulnerable. Having models and the data used to train them open is a great way
to help academia make progress in this space.

------
debt
Remember WEP encryption? I feel like we're at that stage with NLUI except they
mostly have zero security.

------
svilen_dobrev
would this be useful to "massage" a voice phone conversation, so the wannabe
wiretappers cannot (automaticaly) decode what really is in it?

------
jdalgetty
None of the examples triggered my google home.

~~~
jamesgeck0
See justme22's comment.
[https://news.ycombinator.com/item?id=16267546](https://news.ycombinator.com/item?id=16267546)

