
Tesla's self driving algorithm's overlay [video] - belltaco
https://www.tesla.com/sites/default/files/images/careers/autopilot/network.mp4
======
Animats
Compare Google's system from five years ago.[1] (at 7:42)

This new video is at n00b level compared to that.

[1]
[https://embed.ted.com/talks/chris_urmson_how_a_driverless_ca...](https://embed.ted.com/talks/chris_urmson_how_a_driverless_car_sees_the_road)

~~~
adrr
Waymo is at least 5 years ahead of Tesla. They are running driverless self
driving cars already on city streets. My Model 3 can’t even navigate a parking
a lot half the time I use the summon feature.

~~~
jjallen
Ancedote: this evening in Mountain View, the driver of one of Waymo's test
vans had to take over (I watched her grab the wheel).

The van had come to a complete stop with room to pass a USPS truck (it
appeared safe to me from 25 feet away), and didn't begin moving again for a
few seconds. She then took control.

~~~
toast0
The waymo cars have always been timid. That's why most of the collisions
involve the waymo car getting rear ended. They don't move when other drivers
expect them to.

Of course, this is annoying, and being so timid that you get rear ended is
unsafe, but the Tesla approach would be the delivery truck is detected as a
stationary object and ignored, so the vehicle would accelerate to the set
speed in 3 seconds and slam into it. (See the many reports of Teslas running
into stopped emergency vehicles). Of course, Tesla will tell you that their
car isn't really self-driving with one side of their mouth, while telling you
your car is equipped for self-driving out of the other side.

~~~
notyourwork
> Of course, Tesla will tell you that their car isn't really self-driving with
> one side of their mouth, while telling you your car is equipped for self-
> driving out of the other side.

This is my biggest worry with the way this is promoted. It’s false advertising
and misleading to consumers. I wish more companies (in all industries) would
be held accountable to accurately portray what they are selling.

~~~
adrr
Tesla promotes autopilot as safer than a human driver but blames the driver
when there is an accident. How can autopilot be safer than a human if the only
way to safely use is with a human monitoring it?

------
ipunchghosts
The amount of jitter in the estimates makes me nervous, especially when the
model thinks something is present in one frame and not there in the next.

~~~
TaylorAlexander
Agreed. It also said it was running at 13fps. Not stoked about a vehicle going
70mph updating at 13fps.

~~~
LeoPanthera
Tesla says their "Hardware 3", which is what you get if you buy it now, can
process all cameras at 60fps.

~~~
TaylorAlexander
I find it odd that they would publish a video showing performance numbers from
out of date hardware. I mean I believe you - I watched the presentation in
their custom processor and it’s quite impressive. Just weird that they’re
showing old performance numbers. Perhaps this video is old.

~~~
grecy
There is speculation going around the "big rewrite" Elon mentioned last week
is actually porting the code to run natively on the new hardware. Speculation
says it's just been running in an emulation layer, but now they're about to
unleash the full potential of the hardware.

If true, it makes sense the video would also have been captured using this
emulation layer, explaining why it's not latest-and-greatest-fast.

~~~
mike_d
If that is true you should call in to question the integrity of a company that
would run life-critical software on a non-RTOS.

~~~
eru
Realtime has nothing to do with absolute performance. It's about meeting your
deadlines.

And, of course, often real-time systems have much worse throughput performance
than a non-real-time system on the same hardware. After all, latency
guarantees are not free.

~~~
mike_d
Yup. I think we agree on that.

It doesn't matter if you have a highly performant pipeline for detecting other
cars if you have a random 200 ms VM pause as another car blows a stop sign in
front of you.

~~~
Dylan16807
...but if it starts braking hard in 250ms that's still a very superhuman
reaction time.

~~~
RugnirViking
As humans we know our reaction times - you naturally slow down as you get to a
junction or navigate crowded or tricky traffic conditions to give yourself as
much of a chance as possible to react.

This is the thinking behind many types of speed control street layouts. You
should also know where to look to anticipate where danger is most likely to
come from, and be ready with some kind of action. This is why we do hazard
identification tests as part of the driving test - looking in the right
directions at the right times is crucial for operating a vehicle safely.

250ms is a fairly average reaction time for something visual that you are
ready for - but you should really be giving yourself as much time as possible
- if somebody bombs past a traffic light at 70mph, even if it is green, most
people would agree that it was an unsafe move. This goes doubly for an
autonomous car, that is unable to play the positioning negotiation game that
humans are masters of as a result of being social creatures.

~~~
Dylan16807
It's very easy to configure a program to slow down before intersections. Also
if we want to be realistic here, you're going to be able to see that the other
car isn't slowing properly for _much_ longer than 200ms. You'd either already
be braking when you hit that pause, or you'd be so early on that the delay
doesn't make a real difference.

And if you're not hovering your foot over the brake pedal, you're not getting
250ms.

------
gimmeThaBeet
I kind of giggle because the jittery-ness of some of the graphics makes it
like I'm watching some robot version of Home Movies or Dr Katz.

I bring that up because my hypothetical personal questions working on a system
like this is 'smooth' decision making. Objects, lines, are jittery and falling
in and out of recognition, but the actual control inputs to the car are
smooth.

I know, good conditions and all that, but I always find it quite remarkable
watching any sort of automaton make relatively fuzzy decisions. I'm very
curious to know more about how this system 'thinks' about the things it
'sees'.

~~~
FatalLogic
The raw sensory inputs to a human brain must also be jittery. Eyes apparently
see highest definition detail only in a tiny area of the visual field, for
example. The brain stitches it together, interpolates, tags features, and
guesses. The preprocessing miraculously creates the impression of a big clear
image out of a dirty data feed from two jello cameras that are swiveling
around all the time. There's a lot a smoothing going on.

Imagine if you could see the raw input from an eye - it would be a big field
of view, but mostly blurry, mostly not in full color, with a blind spot hole
near the center, and the whole image jittering around violently.

One trait of practical real-world intelligence is ignoring 99.9% of
everything. It usually works.

~~~
01100011
There's smoothing, but maybe more importantly there is an expectation of
continuity and a bias towards likely interpretations. The brain is trying to
detect what it already thinks should be there, and not starting from scratch
every millisecond. I'm not sure if anyone is doing that with neural nets, but
I think we'll keep failing until we do that.

~~~
eru
Yes, mostly. Your brain also cheats and adjusts your memory retrospectively to
a certain degree.

------
ArtWomb
I believe this was released as part of Tesla's Autopilot hiring announcement.
More details here:

[https://www.tesla.com/autopilotAI](https://www.tesla.com/autopilotAI)

Definitely someone's dream job ;) I particularly like the applicant query:
"Tell us, what extraordinary work you have done?"

~~~
Traster
Two issues, first: I thought fully autonomous driving was meant to be done by
now? Second, don't Tesla & SpaceX have a reputation for being a terrible place
to work, with Musk expecting everyone to be working as hard (or harder) than
him, and firing people in weird and capricious ways.

~~~
axguscbklp
>I thought fully autonomous driving was meant to be done by now?

Musk was bullshitting when he made that prediction. Maybe he did it because he
had bought his own bullshit. Personally I think what's more likely is that he
was cynically conning people.

In reality, I think no-one is anywhere close to fully self driving cars. I
would be surprised if we saw fully self driving cars any time in the next 50
years.

The whole field of fully self driving cars has been astonishingly full of
empty hype, for some reason often believed by otherwise quite smart people.

~~~
spookthesunset
> for some reason often believed by otherwise quite smart people

I don't understand why so many smart "tech people" fall for the hype.

Maybe it is because "machine learning" is just abstract enough that even the
most jaded developer thinks they can treat it like a black box where if you
pour enough videos, photos and LIDAR readings for training into the top it
will somehow spit out a fully autonomous self driving car at the bottom.

I do find it funny though. On the one hand Alexa only manages to turn on the
lights successfully 50% of the time yet somehow we will magically have self
driving cars capable of safely navigating the roads any day now. I mean for
fsck sake, we don't even have a thing that can wash and fold clothes
automatically but somehow self driving cars will be on the market any day.

Like, if we cannot even get voice recognition to work right, how on earth will
you tell this magical car which street parking spot to take in a busy city?
How will you tell it to pull over to pick up a friend? Hell, how will you tell
it to go through a drive-through at McDonalds? A touchscreen?

~~~
tim333
As to "tech people" falling for the hype, it all depends which particular hype
but:

"surprised if we saw fully self driving cars any time in the next 50 years."

a couple of years after there have been self driving cars driving around
Arizona [https://youtu.be/aaOB-ErYq6Y?t=93](https://youtu.be/aaOB-ErYq6Y?t=93)
reminds me of quotes like

"The aeroplane will never fly." — Lord Haldane, 1907

a while after the Wright brothers had done many flights. You could argue both
the Wright brothers planes and the Arizona Waymos were a bit rubbish but these
things tend to improve rapidly.

~~~
toast0
Rather expensive self-driving cars, with safety drivers have been driving
around a very small part of Arizona, right?

That's not really that close to an affordable self-driving car that I can take
overnight door to door from the bay area to the Los Angeles area while I
sleep.

That may not be 50 years away, but it sure doesn't seem any nearer than 20
years. Progress towards something usable has been pretty slow, and the edge
cases are going to be tricky.

~~~
tigershark
A percentage of the trips have been without security driver for several
months. As for affordable they are comparable to Uber and lyft and you can
actually get a Waymo trip using the lyft app, so that seems pretty affordable
to me.

------
torpfactory
It's a fascinating video. Does anyone know of a similar video by Waymo,
Cruise, Uber, etc?

Love them or hate them its pretty cool Tesla put this video out at all.
Certainly gives us all a lot to talk about.

~~~
FreedomToCreate
This is from 6 months ago so pretty up to date. Drive with Nvidia
[https://www.youtube.com/watch?v=1W9q5SjaJTc](https://www.youtube.com/watch?v=1W9q5SjaJTc)

~~~
torpfactory
Nvidia is in an interesting spot - trying to create chips and toolkits for
self driving without anyone actually having created the full self driving
solution yet. How do you know that if you buy the new AGX chip from them
you'll actually end up with a product someone will buy? Some of the other tier
one suppliers (I'm looking at you, Continental) are kind of in the same boat.

~~~
stoops
If you're not doing your own chip design and fab, there's really no one else
in town to buy from. AMD, Intel have fallen well short in GPUs. TPUs from
Google/Amazon are a more likely threat long-term.

------
dylan604
I have often wondered about this, so I do find this interesting. Of all of the
info presented, one question I have is how does the AI decide when to go at a
4-way stop? In real life, I'm am constantly amazed at how confusing a 4-way
stop is to humans.

Not that I ever had any doubt into how complicated real-time video analysis
could be, this just makes my appreciation of the complexity of the problem
that much more qualified.

~~~
veeralpatel979
I've wondered why cars don't have a built-in communication mechanism.

I should be able to broadcast somehow (using a voice message?) to nearby cars
that "I'm going now, don't go".

~~~
outworlder
As long as it is there just to improve safety. But it should not be relied on.

In other words, if you 'hear' "don't go", you don't go. But if you don't hear
anything, it still doesn't mean it's safe to go, so you'll have to rely on
something else. And if you are doing that anyway, is the added complexity
worth it?

~~~
V-2
It certainly will be, once the ocean is boiled = as soon as self-driving cars
begin to comprise a significant percentage of cars on the road.

This, in the long run, is a selling point that seems to be missing from most
discussions in this comment section.

I can't communicate "telepathically" with fellow human drivers.

------
thedance
I wonder why they would let the public see this. To the critical eye it looks
like it barely works. It loses its mind right at the beginning, around
0:06-0:07, veers to the right for no reason and then rolls through a stop sign
without stopping. It spends the rest of the video hunting left and right like
a drunk.

~~~
Bedon292
Most of what is happening on that video appears to be a human controlling the
vehicle, the overlay is just what the computer sees. In current public
releases autopilot only warns you that it thinks you are going to run a red
light / stop sign. And it is up for the human driver to actually stop. So,
unless its on some beta release it was probably a human that rolled through
it.

~~~
interlocutor
> _warns you that it thinks you are going to run a red light / stop sign_

I need this feature, and I would prefer this to full autonomous driving.

A computer that warns you when you're about to make a mistake is achievable
today and will increase safety for everyone.

~~~
a_bonobo
I was in a Honda Stepwagn last week and that one recognised stop signs and
clearly displayed it.

~~~
thedance
These can make neat demos but what’s the false negative rate?

~~~
a_bonobo
I don't think that data exists (outside of Honda's internal servers), does
similar data exist for Teslas?

~~~
thedance
I doubt that even Honda knows.

------
exabrial
What's the FPS rate of the algorithm? It says on the OSD: `13.3 FPS`, which
seems very low:

50mph = 73 feet per second / 13 frames per second = 5 feet per frame. This
still way faster than a human I'm guessing, but seems very slow.

~~~
travisporter
Short of hardware, is there any way to speed up this classification by
"pruning" the neural network without losing accuracy?

As an aside, how do they get training data from their existing sold vehicles?
over cell connection probably is too expensive I assume, so maybe via wifi?

------
fcanesin
All this jitter couldn't get some physics ? If it recognized a object moving
couldn't it do some approximate dynamics calculation to predict it's location
on the next frame and use that to help the networks ? Especially in the end
where you see the stopped cars blinking.

~~~
silentwanderer
If you look closely, you'll notice that the stopped cars aren't actually
blinking - their outlines are changing color from yellow to black. Who knows
what that means, but at least it isn't losing track of cars completely.

~~~
blotter_paper
At 0:29 a parked vehicle blinks out of recognition briefly (no box of any
color), but it's partially obscured by another vehicle.

------
jdance
Imagine removing the camera layer and drive yourself using those lines and
boxes, without perceiving the "actual" world at all. Which is what the AI is
doing. It seems completely terrifying to me :)

------
akira2501
I like these videos, but I'd really like to see one in snowy conditions, or
other less-than-ideal road scenarios. I think there's a lot more to observe
and learn from in those conditions than in bright, sunny days, in well
maintained streets.

~~~
latchkey
I'd love to see a Tesla on auto pilot on a Vietnamese 'freeway'... buffalo
crossing street while 3 people are driving directly at you on your side of the
road, in the wrong direction, while 8 people are crossing the street laterally
from both sides (2 of them are drunk), while one person is merging without
looking and also dodging a wheel eating pothole and a truck driven by a meth
addict, is about to rear end you.

~~~
kn0where
Actually, sounds like it wouldn’t be tough for a Tesla to drive safer than the
human drivers on that road ;)

~~~
lolc
Trivially true since car driving must always include the option of refusing to
drive out of safety concerns.

------
iamleppert
Not sure I'd trust an autopilot based on crude visual odometry and
segmentation that can only achieve a paltry 17 fps.

~~~
aeternum
Not sure I'd trust humans since their crude visual systems lead to reaction
times of over one second:

[https://www.researchgate.net/publication/233039156_Brake_Rea...](https://www.researchgate.net/publication/233039156_Brake_Reaction_Times_and_Driver_Behavior_Analysis)

17x faster is an impressive improvement

~~~
iamleppert
Human beings visual systems are far faster than once per second. Your
reference is complete reaction time to both process visual information and
decide on an appropriate action and actuate a control surface given a state
model of the world. Your brain most certainly processes visual information
faster than 17 fps doing novel segmentation and odometry.

~~~
aeternum
Tesla's system is also doing much more than simple segmentation and visual
odometry. Tesla's latency around actuating the control surface is quite
insignificant (a few ms) so I'd argue it is a fair comparison.

~~~
rootusrootus
Does it have the object permanence abilities of a six month old baby?

~~~
trca
In the "FSD Preview" update that came out around Christmas time, the car now
shows a bunch of symbols on the road, and signs (aka stop signs) that it
recognizes. They don't flicker at all, even while you're driving and a large
truck obscures the car's vision of the sign / stoplight. So they likely have
started to enable some form of temporal memory into the system so that it
remembers what it has observed in the environment both frame-to-frame and
overall.

~~~
rootusrootus
That hasn't been my experience. The visualization only ever shows what I can
see with my own eyes, and the cameras are in a very similar position. I've
definitely watched traffic signals come and go, all while not moving an inch.

------
brian_
If Teslas navigate with cameras, does that mean a Wil E Coyote style tunnel
painting will actually work?

~~~
felipemnoa
If done right it may even work against humans.

~~~
starpilot
I think this is under-discussed. People bring up how self-driving cars can't
deal with unwinnable situations, like crashing into a school bus vs. an old
lady. Why is this scenario considered novel? Human drivers cannot avoid these
either.

~~~
Jamwinner
Because it hasent happened? The roadrunner gag comes up every few years only
to be debunked (by snopes and others). Computer vision is fooled by many
things that are still very obvious to the human observer.

------
Fabricio20
Any other HN users getting an Access Denied on their website? The usual "You
don't have permission to access
"[http://www.tesla.com/"](http://www.tesla.com/") on this server."

I'm not behind any proxies - I'm at home, connecting from Brazil here.

~~~
denisoliveirac
Same here. Tesla's website is blocked in Brazil by Tesla itself.

~~~
rapfaria
Same happens to officedepot, gamestop...

------
ghostbrainalpha
I wish they would show the video overlay WITHOUT the background video.

Is the car seeing enough on its own that we as humans would be comfortable
driving with just that information?

------
akerro
What are the blue squares? Features to estimate position and motion?

~~~
deckar01
Most of them are at the base of structures like signs and trees. In a few
places it looks like the Z projection is overlaid onto the video with the
wrong perspective, so it's hard to tell. My guess is that they are generic
stationary object recognition tags that serve as hints about what isn't part
of the road.

------
coolspot
My guess on tag meanings:

Orange labels:

O - Opposite traffic lane (used together with LA or S)

F - Same-direction traffic lane (used together with FA,LA,RA or S)

FA - Forward Allowed

LA - Left turn Allowed

RA - Right turn Allowed

S - Stop sign/line

C - pedestrian Crossing

T - Trash can =) (see 0:32)

Black labels:

P - Pedestrian (see 0:24)

M - Motorcycle (see 0:28)

C - regular Car ?

V - Vehicle or Van ?

K - truK ?

S - ?

L - ?

At the beginning, utility truck is labeled first as S, then as L, then finally
as K.

Almost all cars are labeled as C, some are labeled as V - in my observation
mostly Vans and SUVs.

------
rgovostes
I understand that it is common to use Hough transforms to detect lines, which
is critical in driving to understand lane markers and so on.

In my experiments with the implementation in OpenCV I haven't gotten good
results, especially with noisy detections of lines that aren't there. But here
they seem to get good line detection without many false positives, despite
difficult properties of the image such as worn down paint and low contrast
between the paint and the pavement. Anyone know what they are doing that works
so well?

~~~
repsilat
Hough transforms are "old-school" computer vision. They rely on a know
parameter-space, "This is a circle, that is a straight line" and that isn't
super robust to noise or to unfamiliar curves.

I don't know for sure, but I imagine these days the techniques are closer to
what you're more likely to call Machine Learning. Probably neural net
classifiers trained on manually tagged data, and maybe augmented with a lot of
map data. Maybe with some memory too -- I want to say you'd use a particle
filter, but there's probably some newfangled ML technique that does a better
job than those.

------
ivanech
I love how ugly and function-driven the overlay is. I think every other self-
driving overlay I've seen is much more polished and more clearly made for
public consumption.

------
PeterCorless
I know the exact location of the latter part of the video. It's in Mountain
View off Castro Street. I was like, "Woah. The Terminator lives in my burb!"

------
peteforde
I would love to get access to a 360 equirectangular video of this process. I
suspect that the narrow FoV makes this seem less reliable than it actually is.

------
arathore
The jitter artifacts could be removed by applying some kind of temporal
consistency constraints on the network predictions. But such an approach will
also definitely introduce a lot of edge cases where a sudden change is
actually ignored due to enforcement of the constraint. An adaptive consistency
parameter could probably work better, but it's non-trivial to have a meta-
algorithm to figure out the paramter.

------
felipemnoa
I love this! Anybody reminded of Person of interest?

Exciting to see that slowly but surely we are starting to approach the future
where a true AI will be born.

------
titzer
Is it just me, or other people out there terrified of trusting their life to
some closed-source neural net that sees in black and white at 18fps which
apparently believes that cars pop into and out of existence in a matter of
milliseconds? This is not the future I hoped for.

------
rootusrootus
What was with the "STOP" at the top of the screen blinking randomly even when
the car wasn't at a stop nor at a place where it should stop?

This doesn't really inspire too much confidence, IMO. It makes me think their
FSD efforts are on par with their autopilot implementation.

~~~
Klathmon
It looks like that just shows up any time a stop sign is anywhere in view of
that camera, not that it's actively stopping.

------
120bits
This is a dumb question. How does tesla self driving algo decides which car
was first at a stop sign. Does it store the time difference? How would it
decide if other human driver decides to go before someone else.

------
pier25
Do all these self driving systems work with regular IR cameras?

Somehow I always expected they also used some sort of depth camera to detect
distances instead of figuring it out based on pixels.

~~~
TaylorAlexander
Tesla uses only cameras (some color and some black and white I believe) along
with a radar system and perhaps sonar? But Tesla is controversial in their
approach - essentially all other self driving systems use depth sensors. Tesla
makes the argument that you can actually calculate depth from moving cameras,
which is true. The technique is called structure from motion and it is well
understood. However a lidar is a more direct and reliable way of measuring
depth accurately, so others use them.

The difference being that all Teslas manufactured in the last two years or so
have this camera sensor system installed on them, so they can all be upgraded
to self driving with a software or computer upgrade. No one else is shipping
cars that can be self driving. So Tesla has the advantage of collecting a huge
data set of real world data now, and if they get the system working they can
somewhat instantly “activate” a huge self driving fleet. Other companies have
more expensive sensor systems such that the price is too high to sell the
vehicles (only rent as taxis), and they will have to start manufacturing
vehicles after they get everything finalized.

If Tesla and Waymo perfect their systems at the same time, Tesla will be way
ahead.

~~~
pier25
> So Tesla has the advantage of collecting a huge data set of real world data
> now

Do Tesla owners know about this? Is this an opt-in feature?

~~~
trca
Can confirm: There is an overlay that appears which asks whether you consent
to collection of video (only from the 7 outside cameras -- there is one facing
the interior but it doesn't include that), and telemetry / GPS data for
autopilot training and sentry mode training. If you don't agree, I'm assuming
your car will not be used for the fleet learning since they can't gather any
telemetry. I do believe that critical logging w/o user-data(?) (aka drive
system failure, MCU crashes, etc) is still enabled independently &&
regardlessly of the data collection for AI toggle.

------
hkiely
Shouldn’t this jitter be seen purely as additional accuracy to the human eye?
It isn’t like there is a need for motion stabilization in execution of the
algorithm.

------
yarg
Interesting, but is it just me or does it seem overly jumpy?

~~~
Traster
One of the real challenges of these systems is that even a very good system
only needs to lose confidence in some object for a very short period of time
for the system to lose confidence in its navigation and return control to the
user. The easy way to get around that issue is to categorize objects and then
just ignore them out of the context you expect them to appear in -- which is
how Tesla managed to kill a pedestrian walking their bicycle across the road.

~~~
IceSentry
That event was done by Uber and if the human backup driver was paying
attention, as they were supposed to because it was a test run, it wouldn't
have happen. Yes the software was at fault of doing what you are saying, but
the incident was a test run and the driver should have caught it and report to
the engineer, not play on their phone and not pay attention.

~~~
Traster
Arguing that it happened because the human back up wasn't paying attention
isn't a great defense of an autonomous vehicle. The issue isn't that the bug
happened, the issue is that that particular bug only comes from doing
something fundamentally bad - hardcoding the situation.

------
YooLi
Does anyone know where to find more details about the sensor and hardware
stacks the different self-driving companies are using?

------
b34r
How does it work in the snow/heavy rain?

~~~
trca
Their head of AI talked about this during their AI Investor Day and he said
that even when the road is entirely covered, there is still enough subtle
clues for the neural net to pick up on that it will be possible to have a
self-driving car in the snow. Just like how humans are able to observe the
edge of the road (enough) to slowly drive to their destination, the car will
pick up and learn from those clues too eventually.

------
bozoUser
Is someone else worried that the vision is not 180 view in the front ?

At 22 seconds at the stop sign, after the first car drives past what if theres
a car that breaks the stop sign and ends up ramming into the tesla (as a human
sometimes you anticipate this by seeing that the car is not decelerating as it
is approaching the stop sign)

~~~
noveltyaccount
I assume that's just one of the forward cameras, the car has quite a few more
to build a 360° panorama.
[https://www.tesla.com/autopilot](https://www.tesla.com/autopilot)

------
compscistd
Probably minor but seeing the made up parking spot lines in the video is
really cool

------
EamonnMR
This is strikingly similar to to the imagery in Squarepusher's latest music
video: [https://www.youtube.com/watch?v=GlhV-
OKHecI](https://www.youtube.com/watch?v=GlhV-OKHecI)

~~~
trca
You mean just basic bounding boxes around an object?

~~~
EamonnMR
Well, yeah, bounding boxes overlaid on real footage.

------
dillonmckay
Is RESTRICTED the fourth prime directive?

------
codecamper
What happens when there is no stop sign?

------
spazx
[https://gfycat.com/shortobviousduck](https://gfycat.com/shortobviousduck)

------
OrgNet
too bad they dont show you the 360deg view like the car hopefully sees

------
mapleboi
would be really sick to see the code of this or something similar

------
hit8run
Will it find and terminate Sarah Connor?

------
daenz
>Vision fps: ~18

Surely that can't be right

~~~
jedberg
Probably is. The computer power necessary to go higher would use all the
battery and take up a ton of space.

If you've seen other self driving cars, they usually have no trunk because the
entire thing is server racks.

~~~
jamestimmins
Maybe this is ridiculous, but is it feasible that Tesla plans to use the
SpaceX Starlink satellites to offload data for processing remotely?

~~~
jedberg
Unlikely, too risky. Even the slightest glitch in the network could be
disastrous. The car has to be at least smart enough to pull over and stop
safely.

~~~
mAritz
When talking about computation being a power drain, one could imagine a
scenario where certain computations that can deal with the latency of starlink
internet could be offloaded by default to the cloud. When a network failure is
detected or results take too long to come in, the on-board computer could
still take over the same computations, resulting in more power drain for a
short moment.

------
krm01
It still strikes me why so many people are still against a future with self
driving cars. Sure, people throughout history are scared of change & new tech.
But if you look at this video and compare the amount of stuff the camera
catches AND processes, with the things you are seeing & processing, it's
really hard for me to see me being better at it than a computer.

The reality of driving (as a human) is that most of it happens on autopilot
anyway. It's rare to deal with an anomaly. So realistically, it makes total
sense to keep refining these algorithms and make driving safer and more
pleasant for everybody.

I just find it very hard to understand the perspective of those that are
opposed to self driving cars.

~~~
heavyset_go
There's no evidence that computers are any better at driving than humans, and
there is no evidence that this generation of machine learning will be able to
tackle this problem sufficiently. All signs point to driving requiring an
_understanding_ of what's going on around you, which this generation of ML
does not provide.

On the other hand, there is a lot of hype and promises being made without the
results to back them up.

~~~
daniel_iversen
Isn’t there already a mounting amount of statistics of accidents of self
driving cars vs manual ones and the difference is night and day that the self
driving ones are statistically safer?

~~~
cultus
Not at all. Self-driving cars cannot yet safely navigate complex or unusual
scenarios. The human backups in test cars still have to take over with good
regularity. That's not something you'd see if it were anywhere close to ready.
Even with all their data, there aren't enough training examples of every
situation. Deep learning models also have a tendency to be pathologically
wrong in rare but completely unpredictable circumstances.

Tesla's self driving has resulted in cars swerving into gore points, for
example.

~~~
rdm_blackhole
Point taken. However if you a look at statistics regarding the leading cause
of accidents in the US, for example by going to [https://www.after-car-
accidents.com/car-accident-causes.html](https://www.after-car-
accidents.com/car-accident-causes.html)

It seems that most accidents are caused by human error and humans being
distracted or under the influence of a substance.

Weather and complex situations would only rank at the bottom of the list and
therefore could be considered a YAGNI problem.

If you remove the human element from the driving, already the top 4 causes of
accidents on the road would be reduced drastically.

So at the moment a self driving car may not be able to handle some of the most
extreme situations that a human driver could handle however, I would challenge
that assumption and simply ask: does it have to?

~~~
blattimwind
> Weather and complex situations would only rank at the bottom of the list and
> therefore could be considered a YAGNI problem.

This is a logical fallacy. You are inferring from a comparatively low incident
rate that the condition must also be rare, which is not a reasonable
conclusion to make.

------
agumonkey
Brings back so many memories of 80s shows.

------
throwaway55554
Awesome! Now do it again and when it passes one set of parked cars on the
side, have someone roll a soccer ball in front of the Tesla. Will the car
assume a child will follow from in between the cars or not?

~~~
damon_c
A neural network needs to be taught this and then all cars with that training
will respond correctly forever.

Every individual human would need to learn this separately from experience,
which would require far more soccer balls (and children).

~~~
lexpar
By "taught this" do you mean kill a kid?

~~~
baq
Run this scenario in a thousand variants in sims.

------
willvarfar
Which sensor has primacy? radar, lidar or camera?

I recall it mentioned somewhere that, after a well publicized tragedy hitting
the side of a white truck on a bright day, Tesla was moving to radar?

And yet a lot of the cues this awesome video shows seems to be camera-based
line detection etc?

~~~
modeless
Tesla doesn't use lidar. They have always used radar, but radar can't always
tell the difference between a stationary metal sign near/over the road and a
stalled car in the middle of the lane, so for stationary objects they need the
cameras. This is why every adaptive cruise control system (not just Tesla's)
has a warning that it may not detect stationary objects.

[https://arstechnica.com/cars/2018/06/why-emergency-
braking-s...](https://arstechnica.com/cars/2018/06/why-emergency-braking-
systems-sometimes-hit-parked-cars-and-lane-dividers/)

~~~
tjoff
Though only(?) Tesla pretends that their adaptive cruise control is full blown
self-driving.

~~~
Erlich_Bachman
Source?

If you simply refer to it being called "autopilot", cmon that's not too
serious. The word "autopilot" does not automatically mean that it is full
self-driving. None of the Tesla drivers or potential Tesla buyers
realistically think that it has level 5 self-driving yet.

UPD: The buy page literally says:

"The currently enabled features require active driver supervision and do not
make the vehicle autonomous. The activation and use of these features are
dependent on achieving reliability far in excess of human drivers as
demonstrated by billions of miles of experience, as well as regulatory
approval, which may take longer in some jurisdictions. As these self-driving
features evolve, your car will be continuously upgraded through over-the-air
software updates."

~~~
aguyfromnb
> _Source? If you simply refer to it being called "autopilot", cmon that's not
> too serious_

The Model 3 Builder literally says: "Full Self-Driving Capability".

~~~
Erlich_Bachman
Did you just read the one sentence and not any other text on that page?:
"Coming later this year: Automatic driving on city streets.",

Oh and how about "The currently enabled features require active driver
supervision and do not make the vehicle autonomous. The activation and use of
these features are dependent on achieving reliability far in excess of human
drivers as demonstrated by billions of miles of experience, as well as
regulatory approval, which may take longer in some jurisdictions. As these
self-driving features evolve, your car will be continuously upgraded through
over-the-air software updates.".

\- Which I just copy-pasted from the exact page you are talking about. Who is
being disingenious here?

I honestly don't think that there is any significant number of potential
buyers who go onto a website to buy a car, and pull one sentence from that
page out of context, ignore the rest of the text and then make their decision
based on that.

~~~
FireBeyond
You mean the text in smaller face "disclaimer text" buried under the fold,
well under the price and the "Prices are likely to increase in the future"
("BUY NOW!"), and below things like:

> Summon: your parked car will come find you anywhere in a parking lot.
> Really.

(It may plow into a street to do so, but hey. And you should really have "full
attention" on the car. Unless you're reading our marketing copy which says to
feel free to "attend to a fussy child")

and so on. Ninety per cent of the length of that page is selling promises that
aren't there yet, and the fine print at the bottom is taking most of it away,
for some indeterminate period of time.

