
How Many Miles of Driving to Demonstrate Autonomous Vehicle Reliability? (2016) - quickfox
https://www.rand.org/pubs/research_reports/RR1478.html
======
stefan_
This is just bad math to justify the "imperative to develop adaptive
regulations" aka "lets just let Uber drive without emergency braking".

No one is fooled. It didn't take a billion miles to expose Ubers system as
broken, though it did unfortunately cost someone their life. It is also not
needed to take a graduate class of statistics to look at Waymos tests and
realize none of them involve driving rain in pitch black night to come to the
conclusion that maybe they aren't giving the entire picture and that "driving
miles" are not created equal.

------
8bitsrule
First show me a machine that won't run into a barrier, or a human, or a parked
fire truck, or a parked police car at 40-60mph without braking.

Then ask this (for now ridiculously abstract) question again.

~~~
IanCal
These things seem to happen with humans too.

But what you're asking leads directly to the question being asked here! How
long would something have to drive for before you believe me it's unlikely to
do one of those things? Is it even possible for me to do this?

~~~
diggernet
It would have to drive fully autonomous in _all_ conditions that humans drive
in, for enough miles to produce a statistically valid lower rate of accidents
than humans.

Since it can't drive fully autonomous until it has proved itself, it would be
allowed to have a safety observer onboard. But here is the catch: _Any_
intervention by the observer counts as an accident, since we are simulating
what it would do without a human. Assume worst case.

Oh, and software updates reset the clock.

------
ck2
it's a perfect chain of self-destruction

public thinks technology and computers are "magic" and perfect self-driving
cars are somehow easily possible, not having a clue about the extensive
limitations of radar and other sensors vs software

managers think they'll just fix the code by pushing a new update later, so
just get it out the door because someone higher up is breathing down their
neck

government forces their agencies to work with minimal budgets so testing is
too basic, never updated and never re-reviewed, maybe even corrupted by
lobbyists to do self-regulation without oversight

victims often won't be the drivers who early-adopted or abuse the technology
out of ignorance, laziness or greed but instead innocent pedestrians,
cyclists, or other drives in regular vehicles doing nothing wrong

when the victims sue, juries will be made to easily believe technology can't
be wrong and it was human error by either the victim or the driver

remember toyota spaghetti code for electronic fly-by-wire steering and brakes?
sadly I think it's going to take some lawyers with billion dollar class-action
suits to get all this to slow down

beta testing 2-10 ton vehicles moving fast in the realworld with plenty of
possible victims demands space-grade-level testing (and even then plenty of
rockets blow up on the launch pad or never make it to orbit)

meanwhile I hope independents like consumer-reports come up with an ever-
improving test site of obstacles and events for these cars to deal with and
that they are very thorough and harsh about the tests every year

~~~
brokenmachine
_> sadly I think it's going to take some lawyers with billion dollar class-
action suits to get all this to slow down_

That will come in short order.

Uber was lucky that the person they killed was homeless and not a politician's
son or daughter.

------
rdm_blackhole
We should not make "better" the enemy of perfect or something of the essence.
If we can have self-driving vehicles that are twice as safe as the average
human, then they should be allowed on the streets.

[https://www.rand.org/blog/articles/2017/11/why-waiting-
for-p...](https://www.rand.org/blog/articles/2017/11/why-waiting-for-perfect-
autonomous-vehicles-may-cost-lives.html)

I realize that accepting that a machine may kill you when you cross the street
is a bit unsettling but if you think about it, it is no different than
accepting that the person in the car behind you may have a moment of absence
or not pay attention to the road end up killing you or a pedestrian.

Yet having another human kill another human seems to be accepted by society?

If car manufacturing companies take on the responsibility and financial burden
on them to compensate families that are killed/injured by a self-driving car,
just like insurance companies do nowadays on the behalf of the drivers at
fault, then the problem becomes non-existent unless I am missing something.

~~~
ux-app
The article is saying that it may be difficult or impossible to prove that
self driving cars are actually safe. The Tesla issues and recently released
sensor shortcomings are damning and really put a dent in my faith of this
technology.

An example from a recent road trip I was on: I was travelling down a suburban
street with cars parked on the side of the road. There was a car parked with a
relatively high undercarriage, so that from a distance I could see under the
car through to the other side. I saw a pair of shoes toward the front side of
the car, which I (correctly) deduced were connected to a human being. In
anticipation of driving past a person who might step out in front of me I
slowed down and made sure to pass safely.

Tesla, on the other hand will drive at 70mph into a concrete divider if some
paint on the road is smudged.

We've got a ways to go yet I think.

~~~
imtringued
I'm not sure why you are talking about tesla in a submission about self
driving. Tesla doesn't have self driving technology. They only have adaptive
cruise control and lane assist.

~~~
ux-app
You really think Tesla is not relevant in a discussion about self driving
cars?

------
nmca
This paper consistently assumes a fleet size of 100, and 24/7/365 driving.
These are silly in different directions, but I think the former invalidates a
lot of their claims. As a slightly not-random example, GM produces around
30,000 Chevy Bolts a year. Things start to look a little expensive, but the
paper claims "This analysis shows that for fatalities it is not possible to
test-drive autonomous vehicles to demonstrate their safety to any plausible
standard". I think they are neglecting "without increasing fleet sizes beyond
100 vehicles".

~~~
notahacker
Their assumptions are just one way of reaching the 275 million mile mark
needed to hit that 95% confidence interval. If we're looking at actual
numbers, Waymo's recently boasted of hitting the 7 million mile mark over
several years of testing so if anything, Rand's assumption of 275M miles in
12.5 years for a test programme looks to be erring on the generous side. Sure,
if Chevrolet decided to use their entire production run of Bolts to test
autonomous vehicle performance you'd get 275 million miles pretty quickly, but
they might struggle to get consumer buy in and regulator approval for selling
fully autonomous driving capability to 30,000 consumers unless and until they
can produce evidence that their autonomous mode accident rate is lower than
human drivers...

And there's a much bigger problem, in that the software being tested is
continually evolving, and modifications which result in improvement in general
driving conditions don't necessarily tends towards stable (or monotonically
reducing) numbers of rare accident-inducing bugs.

------
mLuby
>Self-driving vehicles must be driven for hundreds of millions of miles to
demonstrate reliability, which would take 10-100 years.

So we can A) accept higher fatalities for convenience, B) demand much better
performance from self-driving vehicles than we expect of humans, or C) wait.

~~~
thx11389793
If we have a hundred years to burn, how about reorganizing our cities into
forms that obviate the need for cars (self-driving or not) in the first place?

Anecdotally, I've heard people make the point that public transit is
infeasible in the states because the population density is too low, and people
are too spread out.

I'm not sure that it is -- the US Census Bureau stats show the share of
Americans living in urban areas growing steadily throughout the history of the
country, with 80% of the population currently hailing from urban areas.

We are and have been urbanizing since the birth of the country, I reckon if
were smart about building infra now, we can remove the problem SDVs are trying
to solve in approximately as long as it would take for them to become
ubiquitous were we to take no action.

~~~
chiefofgxbxl
Agreed. If car crashes kill so many in the U.S., wouldn't it make sense to
make the country a less car-dependent nation if we want to reduce fatalities?
Walking and biking generally won't get people killed, so why not promote that?

I think part of the problem is that self-driving vehicles is marketed as a
life-saving technology, when it is the novelty and futurism people are
attracted to. (It is indeed an impressive and challenging problem.) The
furthest mainstream press release I can recall making the case for autonomous
vehicles was Google's efforts. This makes me wonder if they had a solution in
search of a problem, so they took road deaths in the U.S to sell the idea.
Maybe they were chasing the next iteration of intense computing application,
e.g. Deep Blue -> Watson -> self-driving cars.

The 40k+ fatalities per year is a solvable problem without autonomous
vehicles: (1) promote equity in other methods of transportation, (2) enhance
driver-assistance technology, (3) design roads to be built for safety rather
than convenience/speed [0]

Regarding point 2 above, note that Chevrolet has "teen-driver technology"
which can set a maximum speed, and provide visual and audible warnings if the
(teen) driver exceeds the speed limit [1]. Presumably the vehicle has cameras
to read speed limit signs. Why isn't this being applied to _all_ drivers? It
seems to be the classic case of "let's blame teens for everyone's bad driving"
(juvenoia).

[0] [https://www.strongtowns.org/journal/2018/1/25/speed-kills-
so...](https://www.strongtowns.org/journal/2018/1/25/speed-kills-so-why-do-we-
keep-designing-for-it) [1] [https://www.chevrolet.com/teen-driver-
technology](https://www.chevrolet.com/teen-driver-technology)

------
mgamache
Totally unsupported by examples, but it seems like the huge economic benefits
will cause society to underestimate to risk of autonomous vehicles. We'll
debate and distort the relative risk and provide plausible deniability for the
right people. In the end, we'll tweak the AI based on real world failures
until it's good enough to win the PR battle. This is assuming it can be
tweaked to work. The key will be protecting (or limiting) the liability of car
manufactures from lawsuits. If AVs are involved in 37,000 fatalities a year
(the 2016 human stats) the lawsuits will be untenable. Maybe there's a plan to
handle this, and I just haven't seen it. Just my opinion...

------
vladislav
How do we test human readiness for driving? We give them a driving test, which
takes all of 40 minutes. This is possible only because we know a lot about
what it means to be a human. To test self-driving technology we need to
understand a lot about the AI that is behind it, which is an active area of
research with much progress. Moreover, there is adversarial testing in
simulation and on raw sensor data, which can tell you a lot about your system
before any large scale real world testing begins, including when it might be
safe to start real-world testing. Real-world testing at such large scales is
neither a practical nor a necessary component to validating self-driving
systems.

~~~
ux-app
>We give them a driving test, which takes all of 40 minutes

40 minutes + 16 years to prime the neural net for

\- object recognition and classification

\- developing a theory of mind so that the NN can anticipate how other NNs
will behave

\- etc

The NN can't sit the 40 minute test unless they prove they can do successful
object recognition and avoidance at low speed (while in bipedal mode). That
they have the sufficient fine/gross motor skills to navigate a complex and
changing 3D environment etc.

It's going to take a while before we've got digital systems that can compare.

