
Tesla's 'Full Self-Driving Capability' Falls Short of Its Name - rakkhi
https://www.consumerreports.org/autonomous-driving/tesla-full-self-driving-capability-review-falls-short-of-its-name/
======
etaioinshrdlu
I would find it kind of satisfying if Waymo solidly won the self-driving race.
It would be a triumph of solid engineering over unreliable "good-enough"
solutions.

~~~
nwienert
You’re defining solid-engineering funny.

You need a mass amount of data to train on. How do you get that? Its an
engineering or a business problem, depending how how you look at it - how do
you get a ton of people driving for you?

Tesla has the better business model here. That leads to more data, which is
the most important factor in training.

Now, google is orders of magnitude bigger _and_ had a huge head start, perhaps
they brute force their way to the lead. But being more conservative != better
engineering. You could philosophically disagree with many things Tesla is
doing, but in terms of _solving self driving cars_ I think they have the
better strategy that gives their engineers the data they need.

Tesla is in the real world recording every car they sell through vastly more
dynamic scenario/terrain. That’s smart engineering.

~~~
lern_too_spel
Is Tesla really sending all that camera data off the cars? That doesn't sound
plausible.

~~~
jayd16
In theory they could train the model in a distributed way on the cars
themselves.

~~~
mlindner
No this isn't really possible. The computation required for training is
extensive compared to the computation required for simply running the system.

~~~
etaioinshrdlu
I disagree, federated learning is a very valid approach. The cars have a ton
of computational power especially with HW3. (Although I think they don't have
high precision floating point - the chips are for inference)

Federated learning is a lot like distributed training of a neural network.

The trouble with distributed training is that it's not as fast to converge as
simply running more training steps. It is basically like increasing the batch
size.

Also, it sounds like a general hassle for Tesla to use federated learning. I
believe they need to be carefully auditing and labelling their training data
for almost all their tasks. Perhaps some like depth-estimation don't require
labelled data.

~~~
Gravityloss
You could train and exchange data while charging.

"You are the product."

------
candiddevmike
All of the current gen assisted/self driving tech seems like it's operating in
a legal grey area like Uber. Every car manufacturer assumes people are still
behind the wheel, and the government is (naively?) going along with it. "By
operating this vehicle you agree to..." seems to dissolve them of any
responsibility, including around quality of software, and it honestly scares
the hell out of me.

I don't know what the fix is for this, the genies out of the bottle here. We
won't have reliable self driving cars for years (decades?), and until then
we're stuck in this horrible wild west where an off by one error can cause a
pileup on the highway.

Let's stop the the assisted/self driving stuff until we have a regulatory
framework that can prove the tech works in various conditions, much like
seatbelts and collision testing.

~~~
z6
We have these accidents and pileups constantly without involvement of assisted
driving technology. By your logic we should disallow anything that could ever
hurt anybody.

~~~
candiddevmike
Exactly, why add another way for humans to be negligent? I would absolutely
love to see mass transit replace self driving cars as the panacea to our
transportation issues.

~~~
mlindner
Mass transit doesn't take you to every location and cannot. Cars must exist in
some form or another for access to many locations. Mass transit has not
replaced all cars in any (non-city-state) country on this planet.

~~~
epistasis
Cars and transit are not mutually exclusive. The more people that use mass
transit, the better the experience fire people that still use cars. However we
are hampered from enabling better mass transit in the US by those who refuse
to let the two coexist and want to force everyone into cars.

~~~
Gravityloss
There are indeed lots of ways of getting around. Walk, tram, bike, e-scooter,
escalator, funicular, metro, schwebebahn, car, train, taxi, rickshaw, boat,
bus. The best cities utilize a mix of those that work for them. Rental
solutions have exploded in recent years.

You can't pick just one and say it's the best for everything.

------
mdorazio
This seems right about where I expected their real-world capabilities to be
today based on their sensor setup and the state of ML for self-driving. i.e.
not very good, and nowhere near the Elon hype train or timetable. The real
question is how much of this can actually be fixed by software updates alone.
Personally, I don't think they can even get to 99% reliability on these use
cases, let alone the number of 9s required to be at least as good as the human
drivers it's supposed to replace. My money is still solidly on Waymo being the
frontrunner for years.

~~~
xiphias2
Actually 3D video and pose detection significantly improved this year in
research, but it takes a few years to put it into production, and it needs to
mature a little bit.

I'm sure both Tesla and Waymo are looking at those research advancements.

------
ForHackernews
Wow, this is pretty damning:

> Traffic Light and Stop Sign Control is designed to come to a complete stop
> at all stoplights, even when they are green, unless the driver overrides the
> system. We found several problems with this system, _including the basic
> idea that it goes against normal driving practice for a car to start slowing
> to a stop for a green light_. At times, it also drove through stop signs,
> slammed on the brakes for yield signs even when the merge was clear, and
> stopped at every exit while going around a traffic circle.

(emphasis mine)

Is that really how this feature is intended to work?

~~~
wilg
It will notify you that it is approaching a traffic control, and if you do not
confirm that you want to proceed through, it will slow to a stop. Presumably
the idea is this is safer than blowing through an intersection accidentally,
which seems reasonable.

You do not have to confirm if there is a car in front of you or it sees other
cars proceeding through the intersection.

I think it (stopping at traffic controls) works quite well for a feature that
rolled out only a few months ago. The article is generally accurate, but
almost all these features are still labeled "beta" and work pretty well,
though certainly not perfectly. The idea is they will improve over time and
eventually get there, the article kind of assumes they are all supposed to be
flawless which I don't think is particularly accurate.

If you interpret "Full Self Driving" as "everything works flawlessly" it's
going to be a disappointment. That's fair enough for a naive reading of the
phrase, you can argue that's overselling the potential of the feature, but
perfect driving at this time is not what is actually claimed.

I think on balance all these features are pretty positive and designed smartly
so that the driver can understand what is happening and correct the car's
actions, and it does seem like they are improving over time.

~~~
tsherr
One doesn't interpret "self-driving". The definition of self driving is
something that drives itself. Period. Adding "full" in front of it makes it
emphatic. It's the same as saying something is "free" or "completely free".

There's no nuance here. He's lying in his marketing. Even a Musk fanboy can't
argue that

I'm not saying it isn't good, it just isn't what he claims it is.

~~~
wilg
I just am not that interested in the semantics of the name. I understand your
point, that’s fine, I didn’t name it.

The idea is that you buy an option for the car that they claim will not now
but in the future allow the car to qualify as “Full Self Driving” by some
definition. It is not claimed that it will do that now, it’s fine to be
skeptical that will ever happen to your satisfaction, but the feature has
always been something that will be delivered over time.

------
fnord77
I suspect some aspects of driving a car consumes a majority of a person's
brain power, even if it doesn't feel like it.

Ever get into a situation where there's on-ramps and off-ramps on both sides,
you've never been on that stretch of road before and there's a lot of traffic?
Driver conversation stops. Driver paying attention to the audiobook or podcast
playing stops. It takes all your attention.

If that's the case, then maybe we won't get true self-driving until these
systems have the processing power of a human brain...

~~~
dstaley
> Ever get into a situation where there's on-ramps and off-ramps on both
> sides, you've never been on that stretch of road before and there's a lot of
> traffic?

One advantage cars with large sensor suites have over humans is that they can
see and process information from all around them. In your specific example,
the car can watch both on an off ramps at the same time, isn't phased by the
"newness" of the road since other cars have navigated this section of road
hundreds of times before it has, and can keep tabs on cars obscured by line of
sight due to the car's ability to bounce signals off of vehicles.

~~~
fnord77
ok - here's a better example - skillful drivers on a highway can predict when
another driver might change lanes just from the "body language" of that car.
It's pretty subtle and I don't think all human drivers can recognize it, much
less SOA AI.

Along the same vein, most drivers will become extra vigilant of another car
that's driving in a way that's the least bit erratic, giving that car a wide
berth.

------
thesausageking
In the summer of 2015, Elon Musk said the software upgrade Tesla was rolling
out would allow the Model S to have hands-free autopilot.

5 years later and even for an extra $8k we still don't have hands-free
autopilot.

------
sega_sai
To be honest I just don't understand how what is shown could be legal --
Having a beta software on something that could easily kill, making the
drivers/passengers other road users lab rats and beta-testers. I do admire
what Musk did with SpaceX, but the 'self-driving' aspect of Tesla is just
disgusting.

~~~
cptskippy
I don't like the way Autopilot and FSD are advertised, and they're very clear
that when you buy FSD it's a promise and not a product. What you get are
incremental updates and new features hopefully leading to actual FSD.

You are not a beta tester testing out pre-release software, the features you
received are considered complete.

The stationary object detection problem that has caused fatalities is industry
wide.

The combination of failures that lead to Walter Huang's death however aren't
in my opinion.

I think GM's monitoring system is the right solution until FSD is solved,
Tesla's steering wheel torque monitoring isn't enough.

~~~
rsynnott
> You are not a beta tester testing out pre-release software, the features you
> received are considered complete.

Well, then, why are they labelled 'beta'?

------
swiley
Meh, everyone knew they couldn’t do it. The sensor array is too limited and
they’re leaning too much on ML.

~~~
jayd16
A human could drive the car with just the sensor input so in a very real way
this is not correct. I agree that Tesla still has a long way to go though.

~~~
starfallg
The difference is that human can do true inference, whereas that's not really
attainable even with the best in class ML models. So until we make
breakthroughs in AI reasoning, we still need aides like LIDAR in order to
workaround current limitations.

~~~
objektif
What does true inference mean?

~~~
xsmasher
If I see a motorcycle go behind a billboard and not come out, I can reason
that there is a motorcycle behind a billboard. I assume that's what they mean.

------
fouc
And all of these issues will be fixed in Tesla’s over-the-air software updates
eventually I imagine.

~~~
threeseed
You can’t ship new hardware over the air.

Tesla is never going to have a solid self driving system without LiDAR.
Radar’s range is limited and cameras are simply not suitable for reliable
depth perception and bounding box detection.

~~~
belltaco
Humans don't have LIDAR, they have only 2 eyes and not even radar. Yet they're
able to drive pretty well, it's because they have great software to power that
limited hardware.

~~~
throwaway4220
Yes, but Tesla has a bunch of cameras not two human eyes.

Edit: I wonder if a human can learn to drive a Tesla remotely better than
another human with radar and multiple camera input by representing them in
some useful way.

~~~
nradov
Remote operation is not yet a viable option for safely operating vehicles on
most public roads. Our cellular data networks are simply too unreliable. What
happens when a construction crew accidentally cuts the backhaul fiber for the
closest base station? Or when the car is driving through a tunnel?

------
jarvar
Would keeping Autopilot 'dumb' be a good way to keep users engaged and
continue training the network? While Tesla keeps and improves a far more
capable version of autopilot internally? I think autopilot is a huge legal
burden for Tesla, and I would imagine being careful is a high priority. Once
Tesla starts marketing full self driving capability, it will open a gateway of
legal troubles that they'll need to deal with once the system is fully capable
and can no longer hide behind the 'beta' word. They've put out videos showing
autopilot doing complex things and I think that's a pre-release version of
autopilot they were using. IDK though, just wondering.

------
jarbus
Shameless self promotion, I wrote an article on this a while back:
[https://jarbus.net/Tesla-and-False-Advertising-In-
AI](https://jarbus.net/Tesla-and-False-Advertising-In-AI)

Elon Musk is just blatantly falsely advertising FSD by this point. He's trying
to sell a technology that a) doesn't exist, and b) has no guarantees of being
ready anytime soon. I wouldn't be surprised if people started demanding
refunds pretty soon.

~~~
judge2020
I don't think anyone takes Elon's word as final. Even if they did, when you
buy a Tesla it has the FSD computer disclaimer on the purchase page, and
enabling any of the features in the UI show as "beta" and have giant
warnings[0] before you can enable them. Given the beta wording, plus the fact
that you already are getting insane value with the promise of continuous over-
the-air updates for years (2012 cars still get updates, although the exciting
changes in recent updates usually deal with the new hardware old cars don't
have), I don't think anyone is going to start asking for refunds.

0:
[https://images.app.goo.gl/KphqD7BPbcCvDXpw8](https://images.app.goo.gl/KphqD7BPbcCvDXpw8)

------
shermanmccoy
Who would've thought that relying on a statistical model, whose many thousands
of parameters are not understand, would lead to such an erratic end product?

------
smabie
I don't understand, why would you ever want the car to stop at a green light?
Can it not reliably read the traffic light color?

~~~
lreeves
I am absolutely shocked by this and am amazed that this is the first time I've
heard of it.

------
nradov
What does HN think of the Ghost self driving product? Is it real or a scam?
They claim to be able to add limited level 3 autonomous operation to most late
model cars.

[https://gh.st/home](https://gh.st/home)

~~~
jeffreygoesto
Highway seems simpler, but it isn't. Just different. Being able to predict the
necessary time ahead with limited sensors at high speeds is as hard as
predicting a high number of participants going slower in the city.

Personally I think level 3 should be banned (can't find it, but I think Volvo
blogged about it years ago), because the system just cannot detect that it
will fail in some seconds and give the driver enough time to react.

~~~
jeffreygoesto
This well known Tesla fatality [0] illustrates my point. Unfortunately. The
model built up by the system was free (enough) of contradictions and the
reaction of speeding up right into that barrier was a totally correct one.
From the system's "point of view". Neither sensor- nor interpretation-wise we
are close enough to humans with those systems if the operating domain is not
extremely limited. All general highway systems are operating in a too open
environment to be safe (that is, we won't see real L3 or higher systems there
in the near future). Sorry, Ghost. Ask other people who have been there.

By the way, Google's tiny slow cars driving in the valley only were exactly
the strategy to limit the operational domain. Once they opened up that domain,
they started to struggle again with the same problems as all others
(admittedly on another level of mastering it though)...

[0] [https://www.theverge.com/2020/2/24/21150546/tesla-
autopilot-...](https://www.theverge.com/2020/2/24/21150546/tesla-autopilot-
fatal-crash-invesigation-huang-ntsb-hearing-live-stream)

------
256lie
Something about lidar being doomed to fail.

