
Ask HN: Deep learning attack vectors - gtirloni
Since the possibility of living in a world full of AIs based on deep learning is almost a fact now, what would be the weak points that could be explored to throw an AI off the tracks? Is deviating from the learning data the best approach?
======
nickbail3y
Noise. Deep learning looks for trends in data. If there's a bunch of noise
(extraneous random data), it's very difficult to put together trends.

For example, lets say you want to throw off Google's targeted advertising
algorithm. Let's pretend you want to buy a stapler. Right now, if you go visit
a bunch of sites that sell staplers, or a stapler manufacturer, then you'll
probably start getting ads for staplers. In order to counter this, you could
visit a bunch of random sites while mixing your stapler research in. This way,
it's difficult to spot the trend.

This is a gross simplification, because a smart algorithm is going to see that
you've visited a bunch of random, non-correlated sites, and eight stapler
websites. So the noise that you're hiding in needs to be specifically crafted.
For example, you still want to do stapler research. But you know now that you
need to be even sneakier. So you visit twenty sites on subject a, twenty on b,
etc, then you do all of your stapler research in a single 'window', and then
continue onto more random subjects. Thus it's more difficult to figure what
you were actually researching. You could probably write a script to do this
pretty easily.

