Autonomous vehicles offer far more benefits than they have drawbacks.
This article is mainly focused on the situation where the criminals will have both hands free. It ignores the fact that the vehicles would just stop if anything was blocking the path. Plus, the vehicles could certainly be turned off remotely during an emergency. And nothing stops law-enforcement from having autonomous vehicles that could be set to follow another vehicle with near perfect accuracy.
This is similar to the Tesla stolen vehicle. WHY would the police engage in a high-speed pursuit, risking the lives of numerous people, to catch a vehicle that has GPS in the car and can probably be shut down remotely. Law-enforcement should be partially responsible for the people injured in that situation, in my opinion.
They can possibly be remotely controlled to lock the doors and drive to a police compound.
The BBC article fails to link to the report. The few details in the BBC report make me want to say "so much derp", because it really does sound dumb. But then maybe the full report is a lot better?
All the police probably have to do to stop an autonomous vehicle is to turn on their lights. Autonomous vehicles have to follow all traffic laws, which would include stopping for emergency vehicles.
That's actually a grey area. Following incidences of police impersonation, it was recommended that concerned people should turn on their blinker to acknowledge the police and safely drive to the nearest populated area before pulling over.
Having your car automatically and uncontrollably pull over every time red & blue lights are observed could lead to traps being set up on any backwoods road.
Edit: I really wish the news article would link to the actual source report so I could confirm it, but I bet that the above scenario is included in the FBI's hypotheticals.
If it's really a concern, (real) emergency vehicles could stick in a wireless beacon with an API the driverless cars could query, that would respond with a {currently_an_emergency: true} signed by a government-CA-issued TLS cert. Then you'd only have to worry about officially sanctioned creepy cops pulling you over on deserted roads.
Is it required that you stop for emergency vehicles? I have always been under the impression that moving out of the way was sufficient. Cops would have no reason to pull over autonomous vehicles unless they were stolen or unsafely damaged.
The vehicles could limit themselves to only drive X miles on a public highway after the registration expires before it alerts the DMV and you receive a ticket.
I just realized something, when the majority of vehicles become autonomous, we will need less police patrolling the streets. I'm not sure if that's a good thing or a bad thing.
Your post seems to ignore the very real possibility that criminals will prepare their cars by customizing the programming prior to the crime. Does your car have a remote disable switch? Hack the car and turn it off. If you have arbitrary physical access to the car, then it will be very difficult to stop you from doing this.
As others have commented, you really do have to take into account the worst-case scenarios. Even if automated cars are a win in 99.9% of the cases, the 0.1% of failures (intentional or not) may be devastating.
It does happen, as is the point of your post. You even left out one of the bigger attacks, 9/11, which deeply impacted the US. You can argue about why, but you can't dispute the fact that they do in fact happen.
The main difference is (and/or will be) that with better automation, an intelligent hacker can execute such an attack remotely, with minimal risk to their person. Just as is the situation today with malware on the internet, we'll be enabling an entirely new class of remote attacks which have the potential to affect many people. We've learned enough from malware to know that this turns into a cat and mouse game with arbitrary depth. The issue is not whether this situation will get away from us, but when and how.
Well, without physical access to the vehicle it seems unlikely that malware could allow unlimited control of a vehicle. Ensuring firmware is genuine and that certain rules are stored on a read-only part of the memory would be helpful, such as don't hit people, turn off vehicle if issued a certain command.
It is good to think about these situations to enable safeguards to be preemptively put in place.
Autonomous vehicles aren't necessarily able to be remotely controlled, the computers and guidance systems are built into the vehicle, waypoints that would be located on roads would be used, which would be unable to cause damage other than the goods not being delivered and the truck being in the wrong place.
Even after 9/11 planes are still somewhat of a threat if the attacker is determined enough. Spend years becoming a pilot or flight crew and you'd have access to the cockpit, after that there's not much limit to how much damage you could cause. There is some protection at certain spots though, the White House is fairly safe: http://cryptome.org/eyeball/wh-missile/pict0.jpg
You are ignoring the very low-tech possibility that criminals may just hire a good driver. Cheaper and more reliable than hacking a very complex system.
Some people apparently don't realize that this is a part of the FBI's job: to dream up and evaluate all the possible scenarios, weigh risks, and plan for as many contingencies as possible. As the article notes, the report is not entirely negative about driverless cars, apparently mentioning as many pros as there are cons.
The point is, when the first mobster/hacker manages to turn a driverless car into an autonomous hit-and-run machine, or the first terrorist uses one to deliver a car bomb, the responsible agencies will be able to react quickly because they've already made plans for that scenario.
> I thought they were supposed to investigate crimes?
When someone manages to figure out how to send a driverless car through a crowd of people in a mall somewhere, do you want the lead investigator to show up and say "What do you mean there was no driver?!"
Some of investigatory science comes from analyzing crimes that have taken place, and some of it comes from imagining potential new crimes. By having imagined the above scenario and being familiar with the type of car in question, they might know to look for computer logs on the car that could lead them to the perpetrator, or that autonomous vehicles rely on GPS which might have left a trail.
The purpose of the research and report is not to determine whether or not driverless cars should be legal, rather to determine what the potential risks are and how they might be mitigated with regulations and/or cooperation with the manufacturers.
That's where the forensic team would come in, examine the evidence and determine the culpability and likely perpetrator.
Pre-crime isn't a thing except in a P. K. Dick novel or a Ghost In the Shell OVA. I expect private vehicle ownership to become mostly a thing of the past once autonomous vehicles are available, and I also expect there to be a phase-change analog where one day most cars are driven by humans and then none are. I can't wait for that day observing the jackasses around me in their cars driving while they: eat, drink, apply make-up, text, talk on the phone, eat, read (!) or watch video on their in-car displays. I lost my brother to a distracted driver and I can't wait until the option to drive becomes obsolete.
It's easy to ridicule the FBI for statements like this, but I'm glad they're considering the downside to some of these technologies. The pie-in-the-sky version of a driverless car society is tremendous, and I hope we get as close as possible to that, but it does open up some other problems.
1. How would you go about stopping an autonomous semi truck that was programmed to drive through crowds?
2. Will terrorists use autonomous cars as delivery mechanisms for what were formerly suicide bombs?
3. If locations with pedestrians are sandboxed in some way, could a sufficiently skilled 'hacker' intentionally break that sandbox without a trace only to blame it on bad technology?
4. How much privacy / 'liberty' would we be willing to give up to enable such a society?
Someone should be considering the worst-case scenarios..
You can remote control a vehicle or truck at the moment with a few electronics and supplies. I don't see anyone using them for weapons, so why would autonomous vehicles be used that way?
As for suicide bombers, a lot of them appear to have other options for achieving the same goal, yet they choose the method that takes their life. They can buy a grenade for $10, and toss it into a crowd, yet they choose to strap a bomb to their chest. I don't think they're doing so because it's the only option, and because autonomous vehicles are not available, I think they're doing it because they want to give their life to their cause.
What about quadcopters? Couldn't anyone just strap a bomb to a quadcopter, fly it over a stadium, and down into the crowd? I think that's even more scary than autonomous vehicles, because they can fly over barriers.
People have countless ways they can cause damage or mass casualties available to them. However, I can walk around a city surrounded by millions of people, and feel completely safe. Most people are quite nice, they're not sitting at home unable to achieve their terrorist goals because Google autonomous vehicles are not available to them.
Nothing will change. Sure, someone will toss explosives in an autonomous vehicle, instead of putting them in a parked car. It's unfortunate, but not a reason to hold back technology that would positively benefit the other 99.99999% of the population.
Agreed. A law enforcement kill switch requirement is probably something that will be discussed as agencies talk through compromises that enable this to happen. (Can't think through all of the pros/cons to this)
That said, if someone is skillful enough to program an autonomous vehicle, they'll probably figure out a way to get around a kill switch as well - like any technology, it can be used for both good and bad. Risk can't be completely removed, but can be sufficiently mitigated. I'm glad the report addressed both sides of the argument, and I'm a bit disappointed that the BBC chose to sensationalize one small conclusion to such an extent.
An autonomous semi-truck that drives through crowds?
I understand that there are maniacs out there, but the technical skill required to do this combined with the cost and the level of insanity would make this scenario pretty unreasonable to consider a valid threat. Not saying it couldn't happen, but there are easier ways to cause a similar amount of damage.
How would I stop it?
- Box it in with dump-trucks, other semis, or armored vehicles.
- Use a tank or other armored vehicle from one of the many armories located everywhere.
- Use one of those MRAPs that the military has been giving law-enforcement recently.
- Shoot out tires.
- Armor piercing or 50 caliber rounds to the engine.
- Remote shutoff that is impossible to disengage without also disabling the GPS.
- I'm very close to a major air-force base, they have plenty of ways to stop a civilian semi-truck.
- Use one of the many armed drones, a military helicopter, or even an armed aircraft. I'd have no problem with the US military using an air-to-ground missile to take out a autonomous vehicle that was programmed to do harm, as long as they were sure that there would be zero civilian casualties.
> 1. How would you go about stopping an autonomous semi truck that was programmed to drive through crowds?
In my opinion that question is no longer important. Back in the 80s and 90s the prevalent thinking was that "bad guys" won't die for their cause. This mandated things like removing the baggage of passengers who aren't flying (which wasn't the case before).
I think we learned quite starkly in the 2000s that suicide attackers are a legitimate threat and therefore we cannot take for granted any longer than a "bad guy" won't put themselves in harms way in order to hurt others.
So if we apply that to your autonomous semi-truck example, how do you stop that NOW? Since presumably someone who wanted to accomplish that could go out right now, get in a truck and then do it. Automation doesn't really change too much.
As far as an answer: Concrete bollards supported by steel mesh (both internally and in the foundation) in pedestrian areas.
This is true, but humans scale a lot worse than automation. Now instead of convincing somebody to kill themselves, all I've gotta do is wire up 30 trucks to do what I need them to do. Didn't work? Do it again. Cars are cheap.
Discussing this keeps reminding me of Suarez's Daemon novels. Swarming changes many, many things.
Surely the universe of "bad guys willing to kill scores of people" is magnitudes larger than the universe of "bad guys willing to die to kill scores of people"?
Stories like this are useful. Not for the details they contain, which are scant and make me think a lot of people in olved are stupid, but because they give "us" a chance to prepare for the inevitable accidents or criminal uses of this good tech. One day a child is going to be oilled by a self-driving car. That will be very sad for that one child, but we need to be able to point to the thousands of children who were not killed or maimed. Getting that message right seems important.
Would a driverless car really be ideal to use as a get away vehicle? You would be stuck in a car that obey's the speed limits and rules of the road. Hardly useful to dodge the cops.
Clearly that section is based on the question 'assuming car software could be modified in a way that made them violate the Laws of Robotic Cars, what new dangers would that pose.' Just like the bit that mentions the trailing car assumes they would have a way to know where a car is going in the future, suggesting a way to query a given car for it's direction.
The first isn't an outlandish question, modifying locked down software isn't exactly a new art. It wouldn't happen for a while likely just due to the small reward to the difficulty of execution.
Wow, the comments here are of an unusually low quality. Let's step it up.
I've personally been thinking of the negative implications of driverless cars as well. I also thought that a car with a driver could take advantage of a highway full of conservative automated cars by driving like a maniac to get to his or her destination faster while all the autonomous cars moved out of the way.
Of course, I still strongly support moving to driverless cars, but I can acknowledge that there are some unanswered questions.
This is scare-mongering, likely for the purpose of reducing privacy and personal rights in using autonomous vehicles for personal transportation.
There is enough open information regarding autonomous vehicles that anyone with disregard for safety could build an autonomous car bomb now. As with other supposed terror threats, the only thing preventing them is that nobody is doing it. But that doesn't sell anti-terrorism brainstorming consulting.
Pretty typical right? And your bank robbery getaway car could be orbiting the bank waiting for you to come out, etc etc. The report boils down to "gee a self driving car should would be useful to crooks." which is sort of obvious.
That said the benefits for non-crooks are a lot higher. And while roads in general benefit crooks getting away in cars, most of the people on them are just trying to get somewhere.
Because any suggestion that driverless cars might present any hint of increased risk in any possible scenario amounts to mindless Luddite paranoia and should be dismissed out of hand?
Who's talking about "stopping progress immediately?" The FBI doesn't appear to be saying that in this article, rather they seem to be speculating on how driverless cars might tactically change the dynamics of a vehicular pursuit, which is a perfectly valid thing for them to do. Because they're right - you can shoot at the cops a lot better when you don't have to worry about a steering wheel.
Also, the Pope is upset because unmarried people could have sex in the back seat while the car drives itself. There is really no end to the infinite possibilities.
>any hint of increased risk in any possible scenario amounts to mindless Luddite paranoia
Not any hint, but this particular one...yes. Dismissed out of hand. A get away driver (an idea that has been around for a very long time) would be far better than a computer that will stop for literally anything in front of it.
The FBI pushing such stupidity is embarrassing to me as an American. Next they'll report that if banks didn't have money they'd never get robbed.
I don't think the report (which is apparently unreleased so i'm speculating) is suggesting that the autonomous car itself presents a greater threat in traffic, but that the driver who no longer has to worry about operating the vehicle is more free to shoot at the cops and bystanders and whatnot - which would be true.
As far as I know, criminals had always a designated driver for exactly those reasons. "bad actors will be able to conduct tasks that require use of both hands or taking one's eyes off the road which would be impossible today"
Looks like the FBI is just pushing to get a real-time interface into the car control. "Algorithms can control the distance that the patrol car is behind the target to avoid detection or intentionally have a patrol car make opposite turns at intersections, yet successfully meet up at later points with the target"
But seriously, FBI analysts get paid to imagine all types of future scenarios let's not hold it against them too much. It wouldn't surprise me though to see more anti-electronic gear, such as EMP, in the field as we go forward. They already have devices that can shut down a car, I imagine they'll work on these new ones just as well and the FBI simply wants to prepare for it.
It doesn't negate the potential use of driverless cars nor is it a good argument against continued development....just things that are worth considering...
My guess? Law enforcement will try to get a kill switch they control installed in all driverless vehicles. I don't know that they should have this, but I bet they try.
I always just assumed that driverless cars would come with a remote kill switch that law enforcement and public safety officials could use.
Even if we ignore this particular threat scenario, what happens if someone has a heart attack, is passed out in an automatic car (as seen by other cars on the road), the cops/ambulance turn up but are unable to get to the person to give them aid?
Or if an automatic car somehow malfunctions and starts driving the wrong way down the freeway/motorway.
That's a good reason not to use this as justification for regulation or nonsense. It's not a good reason for them not to pay attention to developments and figure out possible responses to various situations. I don't know to what degree "warns" was flavor from the reporter versus content of the statements, and I don't know the context in which these statements were made.
lol, in 100 years, it'll look like us saying penicillin should be illegal because if you fired a bucket of it through an uber powerful potato cannon, it can act as a buckshot
This article is mainly focused on the situation where the criminals will have both hands free. It ignores the fact that the vehicles would just stop if anything was blocking the path. Plus, the vehicles could certainly be turned off remotely during an emergency. And nothing stops law-enforcement from having autonomous vehicles that could be set to follow another vehicle with near perfect accuracy.
This is similar to the Tesla stolen vehicle. WHY would the police engage in a high-speed pursuit, risking the lives of numerous people, to catch a vehicle that has GPS in the car and can probably be shut down remotely. Law-enforcement should be partially responsible for the people injured in that situation, in my opinion.