
The Trollable Self-Driving Car - tptacek
http://www.slate.com/articles/technology/future_tense/2016/03/google_self_driving_cars_lack_a_human_s_intuition_for_what_other_drivers.html
======
centizen
I feel like the author of this article is placing altogether too much trust in
human intuition and being very selective with their examples of which human
qualities are beneficial to safety of the roads.

A self driving car will never distract itself. It can't get drunk. It won't be
able to lose it's consciousness or mental acuity as it ages. It has access to
huge amounts of real time information from sensors more accurate than any
humans intuition ever could be and can act on that data in an instant.

If the only hurdles we have to this becoming a reality are edge cases like the
ones listed in this article, then I am very optimistic for the future of self
driving cars.

~~~
tptacek
I think the whole point of the article is that the strengths and weaknesses of
human and AI drivers are _different_ , and that the differences are
exploitable by adversaries, and may in fact be so easily exploitable that
"adversaries" could include bored teenagers.

~~~
modoc
Bored teenagers can (and sometimes do) "exploit" real drivers. I've seen kids
throwing rocks, and bricks off of overpasses, walking on the sidewalk lunge as
if they are about to jump into the road as a joke, etc... I've also seen
people slam on their brakes and/or swerve into other lanes to avoid empty
plastic shopping bags blowing around, etc...

~~~
brbsix
The invisible rope prank comes to mind.[0] I think there are a whole class of
pranks to which AI will not be susceptible to. I assume the LIDAR can or will
be able to clearly detect no such obstruction in the roadway (perhaps even if
it does exist). However there are certainly many other hacks that will
consistently fool AI, at least for a while.

[0]: [https://youtu.be/G_pAcIjqcuY](https://youtu.be/G_pAcIjqcuY)

------
zeteo
The linked article [1] is actually far more worrying than Slate's concerns
about bored 12-year-olds. If a $60 setup can do a rather precise spoof of
lidar readings, we are open to scenarios such as:

\- Causing the car to collide with other vehicles. E.g. getting into a fender
bender to avoid the illusion of an impending frontal collision.

\- Causing the car to stop where it shouldn't. E.g. imagine armed robbers or
kidnappers waiting on a deserted stretch of the road with a lidar spoofing
device.

\- Causing the car to run people over. E.g. spoofing away a pedestrian on a
crosswalk.

What's even worse is that all of these attacks would be difficult if not
impossible to detect or track down. With zero risk you can acquire valid
insurance claims, put victims in positions where they're easily robbed or
kidnapped, or cause your enemies to be run over.

[1] [http://spectrum.ieee.org/cars-that-
think/transportation/self...](http://spectrum.ieee.org/cars-that-
think/transportation/self-driving/researcher-hacks-selfdriving-car-sensors)

~~~
Houshalter
Couldn't you just shoot human drivers from a distance with a sniper rifle? I
don't see why everyone is scared of new technologies being exploitable. Our
civilization is built in the basis that most people are not assholes most of
the time.

~~~
spdionis
Sniper rifles are not easy to get and use.

~~~
panarky
Laser pointers are easy to get and point at helicopters and airplanes.

The FBI successfully arrests and prosecutes those who do. Why would this be
any different?

[https://www.fbi.gov/newyork/press-
releases/2014/westchester-...](https://www.fbi.gov/newyork/press-
releases/2014/westchester-man-charged-with-aiming-high-powered-laser-pointer-
at-police-helicopter-deferred-prosecution-agreement-reached)

------
cornellwright
I don't know, a lot of these things seem to have analogs to human drivers.

I can cause emergency braking on the road outside my house by behaving like
I'm about to run into the street while never leaving the sidewalk.

I could get on an overpass and shine a bright spotlight in drivers' eyes at
night.

I get what the author is saying, and there will be a continual need to improve
self driving cars. Sure there may be different ways to troll them, but I don't
think he makes a strong case that they will be particularly more troll-able
than humans.

~~~
fixermark
Really, the only question that differs usefully is whether self-driving cars
are deterministic enough to be _reliably_ trollable.

Which if they take off, we'll probably find "Yes, until the machine-learning
algorithm at the top aggregates the metrics from multiple cars on the road,
devises a new rule, and pushes that rule via auto-update."

~~~
cornellwright
It's more than just deterministic, but deterministic and stupid enough to make
do some really undesired behavior. Using the same example as before, if I run
a full speed toward the street, almost every car is going to screech to a
halt. Doing this a rush hour, would be incredibly disruptive and probably
cause a small traffic jam if no one made me stop.

The question with self driving cars will be, for example, can a group of
people "herd" one into a garage to steal it, or perhaps get it to do something
crazy like drive into a lake. My guess is at some point the car will just sit
there, just like you would if a huge group of kids were trying to do this to
you.

------
nostromo
Imagine New York City -- where unless a car _appears as if it is ready and
willing to kill pedestrians_ people will jaywalk directly in front of it.

Now imagine New York City clogged up with self driving cars -- frozen at each
intersection in fear of hitting someone. It'd be a disaster.

Humans will quickly understand that these cars are easily bullied, and many
will take advantage of that. Why wait for your turn when you can just cut off
a driverless car?

~~~
kps
I'd like to work on Artificial Road Rage. There are some great opportunities,
especially for networked vehicles. You cut off one Taxy™? Now there are 50,000
tons of mobile neural net wandering the city looking for ways to make your
life miserable.

~~~
brbsix
This has been a free market dream of mine. There's no need for a state to
enforce minor traffic violations (and who knows what else) when "the network"
is willing to do the enforcement. However this poses a huge problem when the
rules are dictated by a central coercive entity rather than by consensus.

------
zodPod
I think the trollable driver is significantly more difficult/dangerous. Humans
have moods and anger issues and vengeful reactions to things.

As has been said repeatedly, accidents like running into a bus are innocent
and could've happened to anyone. While the car was technically at fault, this
likely wouldn't have happened had the bus driver been driving more defensively
or even if the bus driver had been an AI too... lol

~~~
ultramancool
This is sort of the thing though - people can experiment and test the limits
of a driverless system with little to no consequences. Once they know those
limits, they can abuse them, knowing when it's always safe to cut off a
driverless car for example. I can picturing all the videos on youtube now
about how to fuck with these things.

With human drivers, that unpredictability factor adds to a level of fear which
often prevents this sort of behavior.

I think there's also a primed number of people who would do this out of open
hostility to self-driving cars, "took our jobs" types, etc and a lot of people
who would do it just for fun or to make their own trips quicker.

------
mangeletti
> Still, people have an intuitive fluency with this kind of social
> negotiation.

Apparently, the author has never arrived at a four way stop at the same time
as another car.

~~~
roymurdock
Driving 101. In the US, the car to the right has the right of way.

~~~
DrScump
<Driving 101. In the US, the car to the right has the right of way.>

Law 101: driving laws are state-specific.

Law 101(a): Maybe I'm not understanding what you are trying to say here, but
for CA and taken literally, it's incorrect. See, for example, CVC $21753.

[http://leginfo.legislature.ca.gov/faces/codes_displayText.xh...](http://leginfo.legislature.ca.gov/faces/codes_displayText.xhtml?lawCode=VEH&division=11.&title=&part=&chapter=3.&article=3).

~~~
roymurdock
I think you've misunderstood me. I was responding specifically to the case
pointed out by the OP in which 2 cars get to a stop sign at the same time.
Here's the relevant CA code, and I haven't heard of a state that does it
differently:

21800\. (a) The driver of a vehicle approaching an intersection shall yield
the right-of-way to any vehicle which has entered the intersection from a
different highway. (b) (1) When two vehicles enter an intersection from
different highways at the same time, the driver of the vehicle on the left
shall yield the right-of-way to the vehicle on his or her immediate right,
except that the driver of any vehicle on a terminating highway shall yield the
right-of-way to any vehicle on the intersecting continuing highway.

[http://law.onecle.com/california/vehicle/21800.html](http://law.onecle.com/california/vehicle/21800.html)

------
melling
We should pick an easier problem to solve for now. Let's have a few dedicated
roads where we assist the technology either electronically or physically. We
could automate major transport truck freight routes, for example.

[http://ops.fhwa.dot.gov/freight/freight_analysis/freight_sto...](http://ops.fhwa.dot.gov/freight/freight_analysis/freight_story/major.htm)

This system would pay for itself and provide immediate benefits. Convoys of
trucks traveling at 75 mph, drafting to save fuel.

~~~
fixermark
Arguably, that's already being worked on---semi-automatic cruise control that
factors in signals about local traffic flow and the curve of the road has
already been deployed in some luxury vehicles. Applying it to freight hauling
probably has more to do with the entrenched cheap solutions we have now than
with missing technology.

("Why don't we have both?" as the meme says)

~~~
melling
So, in another decade or two we'll see convoys of 10 or 30 trucks cruising
down the road sat 75mph, driving close enough to aerodynamically draft behind
the other vehicles?

No, what you are saying isn't the same thing. People aren't going to be
comfortable driving next to this. The risk on a separate "driverless" road is
much less. And like I mentioned already, other redundant assistance can be
added.

~~~
detaro
Building useful parallel infrastructure is expensive though and would probably
be to slow. If current Autobahn construction projects here in Germany are any
indication, building a parallel long-distance network would take longer and
cost more money than waiting for automation to be reliable enough to share the
roads. Even if we assume that there is space for that, which in many important
places there is not.

Maybe some changes to make roads more robot-friendly (e.g. by adding robot-
only exits, navigation infrastructure, ...).

I think the first step in that direction we are going to see on public roads
are autopilots for trucks. Follow a human-driven truck, or maybe drive simple
long-distance highway routes automatically. At the end, have a human take over
again and do the city stuff. Either have the human on board but resting (which
means they could at least take over with a few minutes delay if necessary), or
have driver hop-on/off parking places at highway interchanges.

~~~
melling
No one said that you have to build an entire parallel infrastructure. You
simply find a section with a lot of traffic and duplicate that. It could be a
long-haul section or it could connect a port to a train, for example. 68% of
all freight is carried by trucks. The United States is big. Find somewhere
where is makes financial sense.

[http://selectusa.commerce.gov/industry-
snapshots/logistics-a...](http://selectusa.commerce.gov/industry-
snapshots/logistics-and-transportation-industry-united-states.html)

The automation technology can be further developed in a "real system". At some
point it'll be ready for public roads.

------
JulianMorrison
People forget that humans used to own the road. Sure there were horses and
carts, but the horses were self driving and went around you if they weren't
going too fast. Children used to play in the street, or at least the less busy
back streets. It has really only been since the 1920s or so that humans got
shoved over onto the sidewalks, or left out of the road planning entirely. A
return to road traffic that deferred to pedestrians would simply put us back
where everyone was in the zeroth through nineteenth centuries.

------
brbsix
There are some places in the world that I'd hate to have a self-driving car,
e.g. South Africa, Brazil, or West Oakland. In such areas, I'd assume it would
be trivially easy to stop the car and rob the presumably well-to-do occupants.

How would a self-driving car react to situations like these?
[https://youtu.be/2VAlyDXl3LY](https://youtu.be/2VAlyDXl3LY)
[https://youtu.be/9RzeRlhhprw](https://youtu.be/9RzeRlhhprw)

These are situations that you'd presumably want to run through the roadblock.
Whereas in others, e.g. an undercover cop with a gun to your head, that you'd
want to stop for.

~~~
recursive
> presumably well-to-do occupants.

Eventually, it should be cheaper to get a ride in one of these than a taxi.

~~~
brbsix
Absolutely. I was just picturing something more like a Tesla Model S (which
AFAIK is the closest thing to a self-driving car now available for purchase)
but I'm aware of the push by Tesla, Uber, and others to make affordable AI
transport practical.

------
vectorEQ
" if you put a foot out into the road you might trigger a nemergency stop" ..
you might trigger a similar painckey reaction in a human, that might include
but wont be limited to , pressing brake, pressing throttle, steering
suddenly... just let them develop it, and if you dont like it, dont use it. ^^
personally im glad people are working on this, especially for older people who
get their licencces revoked due to bad eye sight etc., it can mean the world
to them to be able to 'drive' somewhere.....

------
fixermark
It's a possibly interesting question with a bad example case. In the specific
crash cited, the human driver stated that they reached the same conclusion the
car did. So to the extent that the car's algorithm was "gameable" here, a
human was gamed the same way.

(Besides, since the "win" here was "Get the car to crash into your vehicle," I
wouldn't call it "winning" precisely ;) ).

------
drewg123
One thing I like about the self driving cards is that you can count on them to
let you merge and not be aggressive jerks like some human drivers. I imagine
this will cause lots of people to just cut them off though..

------
davnn
I really think that we often overestimate human intelligence. We may be able
to infer what another person is going to do next but I don't think there is
any magic going on.

PS: Try trolling humans, it works.

