
This car is on Autopilot. What happens next? - otobrglez
https://www.bbc.co.uk/news/av/business-44460980/this-car-is-on-autopilot-what-happens-next
======
mikestew
So the quote at the end is Tesla's "you're holding it wrong" moment, eh? It
reminds me of stubborn developers that blow off bad UI design with "they're
not supposed to do that". Well, fact of the matter is, users _do_ do that, so
account for it. Fact of the matter is, Tesla, your advertising gives users the
impression that they can practically take a nap with Autopilot on (because it
has the hardware for fully-automated driving, amirite?), so don't be pulling
out the fine print about proper use of Autopilot.

To a more general point, when driving a car I should not have to ask, "is it
going to work this time?" I hit the brake pedal, I have a high degree of
confidence that the car will scrub off speed. I hit the gas, I don't worry
that it will go (except in our old VW). I set Autopilot, I expect that it
won't plow into the back of car in front of it. Whoops, scratch that last one.
Instead, I'll constantly be wondering if it's going to do its job.

~~~
notlob
In risk management for the medical devices I’ve worked on, we account for
users and patients not following instructions during the design process.
“You’re holding it wrong” is literally something we account for. From my
perspective Tesla has been baffling, especially so considering the highly
involved CEO is aware that people use his company’s product contra to the
instructions for use.

------
dm319
Tesla keep repeating the 'if you use it as designed you won't have an issue'.

But they need to be honest with themselves - in the public's mindshare these
are 'self-driving' vehicles, and there's no doubt that Tesla has benefited
from the idea that Tesla cars have this technology and the corresponding image
of their cars being a new generation of vehicle.

It isn't helped (or it is, depending on your perspective) by naming the system
'auto-pilot', which implies to the lay person an automatic system.

Really this isn't much more than lane-assist, adaptive cruise control and
emergency braking, terms which do not imply you can delegate control to a
computer.

~~~
vinceguidry
The problem isn't that Tesla isn't being reasonable, the problem is that the
public doesn't have the attention span to try to understand a more nuanced
picture. People just don't want to hear that they're still driving the car,
they want to deal with cars that drive themselves, whether to say it's a great
thing or to say it's a scourge.

There's really only so far Tesla can go in its attempts to educate the public.
In fact, they've been strenuously stating, over and over again, that these
aren't self-driving cars. Autopilot as a name for the technology is in line
with how it's used and expected to perform in commercial airplanes. You still
have to know how to fly the plane and you still need to stay vigilant.

~~~
mikestew
_There 's really only so far Tesla can go in its attempts to educate the
public._

Thankfully, they have a long way to go before they reach the limits of how far
"only so far" is.

 _In fact, they 've been strenuously stating, over and over again, that these
aren't self-driving cars._

The first link I came to when searching "tesla fully automated hardware" was
this: [https://www.tesla.com/blog/all-tesla-cars-being-produced-
now...](https://www.tesla.com/blog/all-tesla-cars-being-produced-now-have-
full-self-driving-hardware) I read the page twice, and didn't see a single
outright statement that the cars aren't self-driving. Oh, I could _infer_ it,
but Tesla never came right out and said it, let alone "strenuously".

EDIT: because my search string was a bit loaded, in that I knew it would find
the link I'm looking for, let's be more fair with:

"tesla fully automated driving" \- the link above is the first listing

"tesla autonomous driving" \- the link above is the 2nd link.

In summary, should I care to know more about Teslas and their autonomous
driving abilities, I will likely be led to a page that never mentions that
these cars can't drive themselves, but strongly implies that they can.

------
simsla
They just really shouldn't have called this autopilot. It creates all the
wrong expectations.

Better names:

\- Advanced cruise control

\- Lane Following

Doesn't sound as sexy, though. :-)

~~~
andrewtbham
The name is technically correct. Autopilot on planes is an assistive
technology. The problem is there is a common misconception among the general
public that it's beyond assistance and is partial automation.

[https://en.wikipedia.org/wiki/Autopilot](https://en.wikipedia.org/wiki/Autopilot)

Furthermore, even if you're totally aware it's only assistance, there is a
tendency to be more easily distracted. That's what a google study found like 5
years ago.

I am a huge Tesla fanboy, and stock holder. But be careful.. if you are
struggling to remain vigilant... turn off autopilot.

~~~
compcoffee
> _The problem is there is a common misconception among the general public
> that it 's beyond assistance and is partial automation._

Again, where do you get that this is a "misconception among the general
public"? Read the marketing. Watch the demos. Listen to the claims.

"Tap your phone and the car will return to you". Where does being vigilant
behind the wheel fit with that?

~~~
crysin
I think the person you're responding to was saying there's a misconception of
what autopilot means in general. People think autopilot is some form of
automation for planes but its not. Though I could be reading the person's
comment wrong.

~~~
andrewtbham
Yes, exactly. If you ask an average person what auto pilot does on a plane
(with no mention of Tesla). They say it's when a plane flies itself. That is
wrong. It's an assistive technology.

------
Animats
That surprised even me. I would have expected Tesla's vision system to
recognize a car rear end and stop. That's the one thing it's supposed to do
well.

~~~
vvanders
It does, the car is changing lanes at the last minute so it doesn't have
enough room to physically stop.

~~~
DanBC
It doesn't even try to stop.

~~~
neolefty
It _does_ slow down and stop, but not in time—it comes to a stop before
hitting a second vehicle directly in front of the first.

The video says it doesn't have room to stop before hitting the first vehicle.

~~~
planteen
I'd assume when you are going 40 mph and hit the brakes as hard as you can
that you'd hear squealing and leave skid marks? Anyone have a guess for how
hard the brakes were applied?

~~~
mikestew
_I 'd assume when you are going 40 mph and hit the brakes as hard as you can
that you'd hear squealing and leave skid marks?_

Not with ABS, you won't. Yes, I've tested this exact scenario. YMMV, based on
the manufacturer.

~~~
marschr
Looking at the Tesla's suspension, it looks like it gradually applied the
brakes.. The accel forces would be way stronger if the car punched the brakes.
Btw, it's probably designed to respond to a confidence level curve and apply
the brakes accordingly. Not to mention that, if the dummy car is NOT made out
of metal (or at least contain some), the ~70GHz radar would have trouble
detecting it.

------
dmode
This issue has been discussed adneuseum on this forum. There is a good article
that was posted here a a few days back that explains the issue:
[https://arstechnica.com/cars/2018/06/why-emergency-
braking-s...](https://arstechnica.com/cars/2018/06/why-emergency-braking-
systems-sometimes-hit-parked-cars-and-lane-dividers/)

~~~
sounds
I think this article is useful because, in addition to the BBC being a pretty
big media outlet, this shows that basically _anyone_ can create the hazardous
conditions.

The Ars article was not as clear-cut.

------
jackhack
What happens next? Nothing, until somebody important dies, at which point
lawyers for the family will swoop in to feed on the company involved.
Meanwhile, lawmakers will preen and posture and create eventual regulations
and restrictions on the concept. The media will portray this as an exaggerated
risk to everyone on the planet.

I am not passing judgement, merely making an observation/prediction.

------
dsfyu404ed
The vast majority of vehicles will have reflective surfaces of particular
colors and sizes in a particular range of locations in accordance with
applicable law and a reflective plate with letters and numbers. Of the ones
that don't most of them will have underside guards, hitch receivers, or one of
several common commercial vehicle body types.

Detecting these patterns is easier than pedestrian and bike detection because
the problem space is so much more bounded. It's hard to give them a pass for
not doing it.

~~~
HeyLaughingBoy
What difference does that make? There are many road obstacles besides cars.
Last Saturday, in broad daylight (3PM), I had a deer run out in front of me on
a freeway onramp, then stop in the middle of the road when she realized her
fawn wasn't following her.

I, and the other drivers behind me, avoided hitting the deer because we're
used to dealing with them. Should we give Tesla a pass if it hits one because
detecting a stationary deer is harder than detecting a stationary car?

------
Osiris
It seems like it would actually be "better" to indicate (audio and/or visual
alert) to the driver actions that should be taken rather than taking them
automatically.

1\. Indicate to the driver that they drifting out of their lane 2\. Warn about
possible slow or stationary object ahead

If the driver knows that car won't take action on it's own, but will help them
but telling them about things that can't see (or when they are distracted),
then we wouldn't be blaming Tesla but the driver.

~~~
tinbad
It always infuriated me that Tesla called the same tech (Mobileye)
"Autopilot", while Audi (and other OEMs) have used the exact same tech but
didn't go as far to call even semi-autonomous driving. Audi & Mercedes systems
do exactly what you mentioned, the tech is used to warn the driver
(audio/visually but even with vibrations in seat and steering wheel) of
impeding danger. It's a good case of Silicon Valley's "break stuff fast,
iterate quickly" mindset that may be acceptable if it's the latest photo share
app, but absolutely despicable business practice when human lives are at
stake.

Disclosure: I currently own an Audi vehicle with this technology that has been
available since 2009.

------
eckmLJE
I don't actually understand how Tesla autopilot detects cars ahead of it--but
does this experiment properly simulate the stationary car, e.g. is it made of
metal (and is that the material that Tesla autopilot is designed to detect)?

~~~
dmode
No, the TL;DR is that all semi autonomous systems ignore stationary object as
it is difficult to precisely measure if the object is in the lane. Highways
have a ton of stationary object such as overhead signs, debris, construction
stuff. Trying to accurately process all of these items will lead to frequent
wrongful hard braking leading to rear ended accidents. That's why they are
ignored.

~~~
diggernet
And yet, Subaru EyeSight can detect and react to stationary objects without
any of the problems you describe. Do they have some magic technology
unavailable to Tesla?

------
sriram_sun
I thought Tesla could detect one car ahead by bouncing off a signal under the
car right in front of you.

~~~
rohit2412
Only a slowing car, and that too would need to be metal.

This is a stationary non metal car. This shows that Tesla's vision system
cannot even recognize a car, much less any random obstruction on the road.

------
GreaterFool
So why didn't Tesla stop?

~~~
nmeofthestate
Because it didn't have the space, and it apparently is not programmed to steer
round obstacles (a much more tricky job to tell where you can steer to as
opposed to staying in between what look like lane markings).

~~~
jackhack
But did it even engage the brakes? I wouldn't be shocked if it could not stop
fully, but for it to not even _attempt_ to stop is very troubling.

~~~
neolefty
Yes, in the video it clearly and immediately brakes, and comes to a stop
before hitting a second dummy car directly in front of the first.

~~~
GreaterFool
So is the presumption that the human driver would do better?

~~~
albertgoeswoof
the car in front did better...

~~~
ovao
The driver in front also knew about the stationary ‘car’ well ahead of time
and was instructed to avoid it by merging into another lane. An actual human
driver in a real scenario may perform better than the Tesla, as well as the
Tesla or worse than the Tesla — it just depends.

~~~
nmeofthestate
Tesla's algorithm is "try to slow down, but nevertheless plow into stuff if
that doesn't work". It is demonstrably worse than the human algorithm of "slow
down, but also steer to avoid obstacles". I can't see how you can objectively
say they're equally good.

~~~
JoeAltmaier
Do we want a car deciding whether to leave the lane? Leave the road? It could
do more harm than good. We'll have to decide this, soon. By law hopefully -
some standards for auto-drive and what it's allowed to do.

Because if we don't, software will have to code up the 'trolly car problem'
solution, starting now. Hit the school bus ahead? or the guy walking beside
the road? or the brick wall (and kill its passenger)? There's no good
(universally defendable) solution.

