
Autopilot Full Self-Driving Demonstration [video] - megakid
https://www.tesla.com/en_GB/videos/autopilot-self-driving-hardware-neighborhood-short
======
Animats
A longer Tesla demo video with no cuts: [1] This one is at a consistent speed,
about 3x normal, and it's worth playing at 1/3 speed to see what's happening.
No freeway driving; it's driving around the Page Mill Road / Los Altos Hills
area. This video shows an ahead view and three of the vehicle's cameras with
their annotations. The system puts rectangular 2D boxes around things it
recognizes.

Road centerline recognition seems good. Every road seems to have really clear
centerline markings, though. Road edge recognition isn't always successful.
Roadside obstacle recognition seems very dependent on recognizing cars and
people. The system doesn't recognize a large trash can in the roadway until
very close. Recognition of roadside traffic cones and barriers seems to be
about 50%. Guard rails aren't recognized at all.

This has the look of something using deep neural networks trained on common
obstacles. It doesn't look like something that profiles terrain.

[1] [https://vimeo.com/192261894](https://vimeo.com/192261894)

~~~
dag11
Is there an easy way to play a Vimeo video at 1/3 speed?

~~~
danthejam
Press F12 go to the console and type
`document.querySelector('video').playbackRate=0.33;`

------
the8472
There seem to be some issues. E.g. it recognizes the joggers (0:55) as
obstacles even though they're obviously (to a human) on the sidewalk. And when
taking a right turn (1:01) it mistakes a hydrant and a parked car in a parking
bay on the other side for an obstacle and stops for a moment before noticing
that they're not actually in its lane.

I think a polite driver may have slowed down gradually to not startle the
joggers, but not stopped if it were avoidable, especially since there was a
car behind the tesla.

In the latter case i guess the field of view of the cameras is not wide enough
to assess those things during a turn in advance?

~~~
frik
Does this new Telsa autopilot use stereo cameras? Would stereo camera help to
have a better 3D vision? Or is a video stream enough to get the 3D data.

From the video it looks like lane detection and 2D object recognition.

It might use a structure-from-motion (SfM) running on CUDA to get a 3D point
cloud out of the 8 cams. Does someone know more about how their new auto pilot
works?

~~~
hwillis
They do not. They have 3 forward cameras (1), but they all point directly
forward and are for separate use cases. I've always thought it would make more
sense to have two cameras at either side of the windshield rather than one in
the middle. The downsides are you need a new method to clear water and debris
since the wipers won't hit that area, and there might be limited FOV.

Stereo might require some level of calibration to account for camera
alignment, maybe they want to avoid that? I would think that implementing
stereo would be fairly cheap both computationally and in hardware- if they can
get by this well without it, they could pare the stereo computations down to a
few frames per second, on specific objects in the scene. Maybe a future update
will add binocular cameras in the dash, but they are pretty confident about
their current hardware.

1:
[https://www.tesla.com/en_EU/autopilot](https://www.tesla.com/en_EU/autopilot)

------
Reason077
Very cool demonstration (despite curious choice of music!).

I did notice that the lane markings are suspiciously bright and well-painted
throughout the video. Almost as if they'd been touched up by Tesla! Or maybe
this area just has particularly diligent road-painting crews.

In many parts of the real world you'd be lucky to get such clear lane markings
throughout the journey, certainly not here in the UK or in places that get
winter ice & snow.

~~~
martin_bech
In my autopilot gen 1 Tesla, when the software was first released, lane
markings had to be really clear, but as the software has matured, it now does
less than perfect lane marking in pretty heavy rain without issue.

~~~
cel1ne
What does it do on roads without any lane-markings? My country has lot's of
them outside the cities.

~~~
schiffern
It does two things.

 _Holistic path planning_ uses the entire scene to identify the lane, not just
the lane markings.[1] This is exactly how a human drives on a rural road.

Once enough fleet learning data is gathered, _high resolution maps_ will mark
out the road surface to high accuracy (cm).

Of course that's no good unless the car knows where it is within cm! For this
Tesla uses sensor fusion with GPS, an onboard IMU (similar to a smartphone),
four wheel wheel odometry (via the ABS sensors), steering angle, and torque
delivery. Owners report that the Autopilot 1.0 cars know their position in a
parking space to within about 10 cm. And it has demonstrated high accuracy and
low drift in no-GPS situations[2].

This type of sensor fusion (Kalman filtering) was originally developed for the
Apollo spacecraft, and was one of the responsibilities of the famed Apollo
Guidance Computer.

[1] [http://wccftech.com/tesla-autopilot-story-in-depth-
technolog...](http://wccftech.com/tesla-autopilot-story-in-depth-
technology/4/)

[2] [http://motherboard.vice.com/read/how-teslas-model-s-
tracks-y...](http://motherboard.vice.com/read/how-teslas-model-s-tracks-your-
location-in-a-tunnel-without-gps)

~~~
disordinary
I wonder how it works in situations like we have in NZ where the recent
earthquakes have moved parts of the country six metres.

------
tsaprailis
There's also the longer (3:28) video with the Paint It Black theme:
[https://www.tesla.com/videos/autopilot-self-driving-
hardware...](https://www.tesla.com/videos/autopilot-self-driving-hardware-
neighborhood-long)

~~~
mhomde
That's really weird choice, who are picking these songs? :) I can't imagine
they've licensed the music either... I guess next up is the "Highway to Hell"
video

~~~
GunboatDiplomat
My guess? Elon.

~~~
mhomde
Haha, I find the image of him doing that, and no one daring to try change his
mind, hilarious for some reason. I can see it escalating:

"So for our next video let's set up smoke machines on the road here, and then
we'll have a camera team in front capturing it coming out of the smoke in
slow-motion with "Smoke On the Water" as soundtrack. It's going to be
amazeballs!"

"Umm, ok Elon..."

"Put in a lion roar as well"

"I quit."

------
sliken
Impressive to watch. The Tesla drives quite politely, but not annoyingly so
(read that as impeding traffic). It actually stopped for the joggers who
wasn't actually in the road, but very close. It also waited for a bicyclist on
the right to turn left in front of the car.

Looks pretty promising, not bad, especially since the trend in software (and
sensors) is to get better over time.

~~~
inimino
On perfectly marked roads, in fair weather, with no bad behaviour seen from
other drivers in the video. Impressive, but the technology has a long way to
go yet before it can replace human drivers.

~~~
Gys
I do not understand why the focus on self driving is so much on the whole
trip.

90 - 95% of the time I am driving long stretches, I spend on highways. No
problem if only that part would be done selfdriving. Those few minutes in busy
town centers I do not mind to do myself. No problem of driving short stretches
(10 - 15 min) either.

~~~
kalleboo
Autodriving 90% of the time would be a fantastic feature.

Autodriving 100% of the time is where it gets revolutionary (mass-layoffs of
truck drivers, taxi drivers, summon features that park cheaply outside of
downtown).

The former is great, but there's a lot of hype going around right now so it's
not strange for people to be cynical.

~~~
rebuilder
Forget parking, the cars would be driving, ferrying people and goods around,
as much as possible when not charging. No need to own one, just hail one when
you need a ride.

~~~
bb611
That's practical, and for people like you and me who aren't bothered by the
idea of not owning our own vehicle, but it's very culturally desirable in the
US to own a personal vehicle. Not an unsolvable problem, but it will
definitely be a bump in the road.

~~~
zobzu
i don't think its an issue. live in the city? dont own a car. just like today.
uber is way cheaper than owing a car.

live in the gonnies? own a car. hailing one would take forever. having cars
near you in the gonnies,that you dont own wouldn't be economical.

basically, there would be less vehicule owners but not that many less.

~~~
rebuilder
I do wonder how many people living in cities would still choose to own a car
if they could just get a self-driving one on demand.

For example, friends of mine recently bought a summer cottage. It's a little
over an hour's drive. They don't have a car, so they currently rent one or try
to borrow one when they want to go there. Once there, the car will probably
sit still for a week.

Currently, there's no really good solution to this. Renting is expensive if
you just need to drive a couple of hours a week. Buying a car might make more
economic sense in the long term, but it's still pretty expensive for the
purpose. Taxi rides like that cost an arm and a leg too. But if the car could
drive itself out of the woods...

Of course it's a question of pricing, and I'm guessing we're still some way
off from having self-driving cars navigate dirt paths that pass through
people's yards as is the case here.

------
elisee
I believe Tesla's AP 2.0 is built on NVIDIA's platform (with probably a lot of
custom stuff, of course). If you want to learn more about how that kind of
system works, that CES 2016 demo is very instructive, especially from 42:50
onwards:
[https://www.youtube.com/watch?time_continue=2573&v=KkpxA5rXj...](https://www.youtube.com/watch?time_continue=2573&v=KkpxA5rXjmA)

------
Zombieball
Reading through Tesla blogs and watching this video, my understanding is the
self driving hardware is cameras and ultrasonic sensors.

Does the lack of lidar not scare anyone else?

Also, Out of curiosity, does anyone know how a Tesla behaves if you point
ultrasonic transducers in the same frequency at it? Do they have special
modulation to avoid tampering in this manner? I imagine you could confuse the
car into thinking there are barriers it can't see.

~~~
mastax
> Does the lack of lidar not scare anyone else?

I don't think lacking lidar should be inherently scary, humans don't have
lidar sensors either and they do well enough. I do hope that cheap solid state
lidars do come to market to improve low visibility driving nonetheless.

~~~
YZF
The human eye has an incredible dynamic range (>20 stops). It also have very
high resolution at the point of interest. I seriously doubt that Tesla's
cameras are anywhere close to the performance of the human eye. LIDAR can also
make up for deficiencies in the software. The human CPU is also probably a
little more advanced than the one driving this car so anything helps.

------
martin-adams
I would absolutely love to see how it handles roundabouts in the UK. Some of
the ones I use daily are pretty hairy even for a competent driver.

And then there's the magic roundabout in Hemel Hempstead!
[https://www.youtube.com/watch?v=ih223jffmek](https://www.youtube.com/watch?v=ih223jffmek)

------
Animats
Much better music video of real self-driving: [1] This is what traffic looks
like when you totally eliminate human drivers.

[1]
[https://www.youtube.com/watch?v=zm_rlLyelQo](https://www.youtube.com/watch?v=zm_rlLyelQo)

~~~
mariusandreiana
It's so amazing, I was wondering if it's a computer simulation or for real.
Great job! [http://www.ect.nl/en/content/euromax-terminal-
rotterdam](http://www.ect.nl/en/content/euromax-terminal-rotterdam)

------
comboy
To be completely fair, you can show me almost anything with Benny Hill music
in the background and I will enjoy it ;)

Anybody knows if these 3 cameras on the right are the only ones used by the
auto pilot?

~~~
disillusioned
No. The system uses 8 cameras. Three forward facing (narrow, wide, standard),
two rear facing sides (the two showed here), one rear, and two side cameras.

~~~
comboy
Thanks. makes sense, and I guess makes easier to implement some kind of depth
perception.

------
martin-adams
I think watching this in real-time would be more informative. At 1:32, it pull
out on what looks like could be a busy road, then stops before starting again.
It doesn't look so bad sped up in the video, but not great in reality.

~~~
skykooler
Here's one someone did, from the teslamotors subreddit.
[https://www.youtube.com/watch?v=hLaEV72elj0](https://www.youtube.com/watch?v=hLaEV72elj0)

------
annerajb
What's the most amazing is that Elon mentioned on twitter that it's only using
data gathered in the past month since they announced the hardware/first video.

~~~
kayoone
But why wouldn't they use old training data to make it even better ?

~~~
bbarn
Could be because there'd be assertions made under the old system, that were
later found to be unhelpful or unwise?

~~~
IanCal
A simple one, the old system didn't have 8 cameras so you don't have the full
training data you need.

------
pupdogg
Fastest speed clocked in at 35mph during the entire trip with an average of
15-25mph for remainder of the time. Does this suffice a realistic use case?
Specifically in US?

~~~
GunboatDiplomat
In town, with the level of traffic seen? Yeah, it's keeping to the posted
speed limits. Realistically, I'd be doing 5 to 10 over except in explicit
residential zones, when there are cops (limit to 5 over), and when traffic
doesn't allow it.

------
latch
Couple spots where I think an inattentive driver could have easily rear-ended
the car. ~0:55 mark is a prime example.

I understand that whether the driver was paying attention or not, he or she
would be at fault, I'm just pointing it out.

------
romanr
So it appears it's reading the road signs? I see it stops on the intersection
with a stop sign where's no traffic lights. But what if the sign is damaged or
removed? The car will go straight without stopping?

~~~
tomp
Maybe it's relying on maps & GPS data.

~~~
visarga
What if the GPS data is outdated?

~~~
TeMPOraL
What would _you_ do if you'd remember there _used to be_ a stop sign there,
and now it's gone?

~~~
linsomniac
I'm still having a hard time not stopping at the intersection by my old high
school and they removed the stop sign there 3 years ago!

------
kozak
I'm disappointed that radar images aren't included in the video. They ought to
be very interesting, a whole new perspective on the world around us.

~~~
skykooler
What the car sees with radar does not translate directly to an "image" (it's
not 2D radar, it's 1D with time of flight for depth).

~~~
kozak
Still it can be visualized.

------
snsr
I'm curious about the implications of this technology (which is amazing) on
courteous driving habits.

I regularly stop to let people cross the road, I cross the center line when
safe to give people and cyclists (if I plan to pass) extra space etc. I
suppose, in this Tesla at least, you can just take control, but some other
proposed implementations seem to have a less traditional UI.

------
Shivetya
Well it was certainly fun to watch. Aren't there more detection methods than
just cameras? This is certainly the best one I have seen from them but I still
want to see it at night, in rain, and such, times when people need the most
help with driving. This was still too nice of weather to get a good impression
of how far its come.

Also, if the driver wants to exceed the speed limit of a road will autopilot
allow them? If so, who becomes liable since the system will obviously know the
speed.

If the road had not been curved when it came upon the joggers would it have
passed without stopping? Is that something that needs work on? I understand
its erring on the side of safety.

So next drives, a night run, a rain run, and if they are really ballsy then
both at one time. Would be cool to have someone throw something in front of it
while its going through the parking lot, beach ball, etc.

~~~
handonam
It seems like the right-bound of the lane (the right red line in the midview
camera) was not calculated, so it thinks the joggers were inside a big lane.
that'd be my guess as to why they would think the joggers were not "outside"
of the lane for the system to breeze by. At 1:23-1:24, there's a person
walking their pet and it looks like they were really close, but the camera was
able to detect the right boundary.

------
gambiting
I'm surprised by the number of Stop signs everywhere. Is that normal in US?
Also, there are moments in this video where the car was turning left and a
driver coming from the opposite direction was turning right - yet Tesla turned
first? How?

I really wish they provided a non-sped up video.

~~~
pmyjavec
I agree, the roads are _perfectly_ marked. It's early days but incorrectly
hardly complex edge cases, like graffiti on the signs, could be a matter of
life or death.

I wonder if the AI just detects an intersection and if it doesn't know what to
do, disengage?

~~~
jfoster
Why would a missing sign cause life or death? It has radar and complete
around-the-car vision. Should do a lot better than a human.

~~~
gambiting
I'm almost certain that I can't see around corners though.

------
sabujp
I'd love to see this in $heavilyPopulatedIndianCity. ahha, found this :
[http://www.theunrealtimes.com/2016/09/17/top-secret-uber-
tes...](http://www.theunrealtimes.com/2016/09/17/top-secret-uber-tested-its-
self-driving-car-in-india-and-these-are-the-results/)

these guys are doing it right (teach your car to drive on the other side of
the road) : [http://www.deccanchronicle.com/technology/in-other-
news/0103...](http://www.deccanchronicle.com/technology/in-other-
news/010316/self-driving-car-indian-techie-builds-autonomous-tata-nano.html)

------
cs2818
Is a realtime version available?

I notice at 0:24 in the short version (0:41 in longer version [0]) it seems to
stop early for a stop sign and then resumes driving to the stop sign. It looks
like the vehicle in the oncoming lane may have caused the stop. I only noticed
because in the longer version the person in the driver's seat lifts his hands
when the stop occurs (not as noticeable in the short version).

[0] [https://www.tesla.com/videos/autopilot-self-driving-
hardware...](https://www.tesla.com/videos/autopilot-self-driving-hardware-
neighborhood-long)

------
sz4kerto
I think the most interesting part of this video is that there's a lot of fog.

------
fernandezpablo
Is it doing a weird maneuver on 1:32 or is it just me? Something like stopping
right after turning. I'm not full aware of the USA transit laws so this might
be required but it doesn't seem likely.

------
Asooka
That's really impressive, but I really want to see it driving through rush
hour in a densely populated old Italian city. Let's continue pushing the
envelope.

~~~
atemerev
I did it. Once. Friday evening in Milan.

Something you wouldn't wish on anyone who is even remotely your friend.

------
ema
I think the choice of music is quite clever, makes it seem jolly and
approachable, instead of the connotations of solemn futurism that self-driving
usually has.

------
carleverett
The car seems to have a bad habit of turning its steering wheel at stop signs
before making its turn. This isn't really a problem at stop signs, but would
definitely be dangerous if turning left on a green while waiting for oncoming
traffic.

The only left on green we get to see in the video is at 15s, but it didn't
have to wait for any oncoming traffic. Hopefully it knows to treat those
situations differently.

------
wslh
I would love to see how autopilot works in different cities (e.g Mexico City,
Rome, Buenos Aires). I Just found "I tested Tesla Autopilot in Manhattan
traffic — and lived to tell about it" [http://mashable.com/2015/11/06/tesla-
autopilot-new-york/](http://mashable.com/2015/11/06/tesla-autopilot-new-york/)

------
jbuzbee
I wonder how these self-driving systems are going to deal with wildlife or
even domestic pets running wild. It's fairly common on my commute to see deer
grazing along the roadside just a few feet from the pavement. I typically slow
way, way down and creep by as they'll walk right onto the road if the mood
strikes them. Getting enough test-cases for the software could be difficult.

------
oolongCat
I really hope Tesla would do a demo on the roads of India. If a car can self
drive there I would have no trouble purchasing one. Indian drivers are some of
the most resourceful drivers I have seen on my visits there, yes its complete
chaos, but how they navigate through all that, while utilizing every bit of
the road is, just amazing.

p.s I really enjoyed the choice of music on that video.

------
robbrown451
As I watch all the views from the cameras, and watch how it is making
decisions based on that information, I have to ask: who says a machine can't
be conscious? Who says it can't have free will? How do you define those terms
so that includes people and animals, but doesn't include this vehicle?

</deep philosophical mode>

------
PinguTS
Daimler done that 3 years ago even in high density city traffic. Daimler could
even pass vehicles that covered half the lane and didn't need to stop for
them. [https://youtu.be/SUOC8tE4bdM](https://youtu.be/SUOC8tE4bdM)

~~~
Klathmon
And a horse could do it centuries before that, but that's not a fair
comparison either.

The car in that video had so much extra custom stuff it's not even funny. And
while it's still a technical feat, I don't see every mercedes being sold with
that equipment in it already.

The reason this is amazing is because Tesla is planning on switching this on
for all of their new cars in the very near future. It might not be the most
technically advanced, but it's here.

~~~
Reason077
It's just like how Mercedes (and others) have shown some fantastic "Tesla
killer" electric vehicle concepts. That's nice, but it's not really something
that competes with Tesla until it's available to buy.

While the others make promises, Tesla's making cars.

------
sethbannon
Anyone understand how it's color coding the boxes on the right? Seems like
green, blue, and purple all likely stand for different categories of objects.

~~~
DuckyC
there's a legend at the bottom of the video

------
ape4
I wonder if it can detect when it doesn't know what do it. Like when it can't
find the edge of the lane.

------
51Cards
Camera based... so if you drive through mud or it's raining hard does this
still work?

~~~
troutaway123
Do your eyes still work when you drive through the mud or it's raining hard?

~~~
51Cards
Do the cameras have windshield wipers? At times if the window gets too dirty
or it's raining too hard I need to pull over to clear it, even with my high
tech eyes. Will the car go into a fail safe mode? Just stop in the middle of
the lane because it thinks the bug-strike on the lens is a person?

------
rbobby
Ok... I'm officially old now. I've been keen on the idea of self driving up
until now. Watching this just makes me want to grab the wheel... too much
traffic, too many people on the side of the road... nope I don't think would I
be comfortable.

Oh well... I guess I'll have to wait for the mini-van style with blacked out
windows :)

------
lazyjones
That's impressive, but still an easy course compared to what I see daily in my
(large) city. I'll get mine in February and am curious to see what it'll do in
those narrow roads where 2 vehicles going in opposite directions can't pass
each other sometimes.

~~~
jfoster
Wouldn't the answer be "same as an average human?"

~~~
lazyjones
I hope not; the average human can't pass an obstacle at 1-2cm distance safely,
a machine could.

------
amq
Can someone explain the left turn on 1:41? Seems a bit aggressive.

~~~
dexterdog
Watch out at regular speed. Ashley consider the acceleration power a Tesla
has. It can get into a tight spot quickly without altering traffic.

------
codecamper
what happens when a few autopilot cars come to different roads on a 4 way
stop?

What about driving through a parking lot (lots of cars around).

Seems dang near impossible to deal with all the edge cases.

------
thomasthomas
the design possibilities and field day designers will have when steering wheel
and forward-facing seats aren't retrictions will be amazing

------
em3rgent0rdr
best part at end: you can save time by exiting before the car parks itself.

~~~
jakubp
I thought the best part was the car doesn't need to park anywhere near. Large
parts of cities will be freed for us to use.

~~~
jzwinck
Large parts of cities will be turned into roads for driverless cars to ferry
themselves to and from the office 2x as many times to save the owner $10 on
parking. Traffic will go slowly because driverless cars don't care about speed
when no one is inside. And then all the plebs still driving their own cars
will be stuffed.

~~~
pmyjavec
This is kind of true, as much as this tech is fascinating, I can't help but
feel trains and better planned cities are the real solution to the car
problem.

Even in parts of Europe I noticed they put big trucks on trains to automate a
large part of the journey and a driver finishes the last leg of the trip.

Trains are also much easier to automate, I also imagine a train track costs
less to maintain than a road.

~~~
kayoone
Trains are complementary but there will always be need for personal
transportation.

~~~
pmyjavec
I never said there wasn't, however; What is obvious is that if taxis and
trucks (privately owned or not) are now so cheap to run because there are no
labor cost involved in having them on the road, then the next problems we will
face are increased congestion, increased pollution (electric cars still
consume a lot of power), increased road degradation. The only way to control
this might be through increased taxes or car registration fees.

This still means the car problem needs solving.

------
LeonM
I don't know about other countries, but at least in the Netherlands the Benny
Hill song is used mainly in videos to ridicule the subject. I remember when
funny 'Home Videos' were a thing on TV (before cats took over on the interweb)
this song was used in about 90% of them. So to me, using this song in
something as serious as a self driving car demonstration is pretty weird.

~~~
Aardwolf
This song is also often used in a recurring absurd segment in cartoons when
some people are running after each other and they go into / come out of random
doors in a hallway.

~~~
dexterdog
Which is an homage to Benny Hill.

------
noir_lord
The Benny Hill theme tune was a pretty unfortunate choice of background music,
not sure if that's the impression they wanted to make (maybe it was, not
sure).

~~~
JimDabell
When the last video came out, there were quite a few people saying "wait, that
video's sped up, right? It doesn't seem to be taking turns at a safe speed."

I expect Tesla got feedback indicating that some people didn't notice that the
last video was sped up, so they decided to make this video a lot more obvious
by speeding it up even more and adding the music. I agree, it's not really a
good choice, but I can see how they might have come to that decision.

~~~
TeMPOraL
Indeed; for me this music has two primary connotations - "funny" and _" sped
up"_.

------
smnscu
Wow so many comments about the Benny Hill music. I for one love that Elon
Musk's quirky sense of humour. So many people take themselves far too
seriously.

~~~
justinv
Do you think that Elon Musk was the one to put this out (or even design the
video)?

I'd have a hard time believing that, to be honest.

~~~
jdiez17
Elon is known for being closely with involved with his companies' PR. Here he
acknowledges the music choice explicitly:
[https://twitter.com/elonmusk/status/799908000689336320](https://twitter.com/elonmusk/status/799908000689336320)

------
Pica_soO
His fingers, so twitchy.

------
Dowwie
What I've learned from this is how boring demonstration videos of self-driving
are, regardless of whether Yakety Sax plays in the background

