
MIT 6.S094: Deep Learning for Self-Driving Cars - kennethko
https://selfdrivingcars.mit.edu/
======
Animats
There seems to be this assumption in the educational community that deep
learning is the answer to automatic driving. Yet that's not how Waymo does it.
They're geometry-first. Most of the control is based on "flat road here, big
obstacle there", plus map data. Machine learning is only for "what's that".
See the TED talk by Chris Urmson.[1]

Driving on deep learning alone will work great, most of the time. Some of the
time, it will do something completely bogus. Like Tesla's self-crashing cars,
which use Mobileye's system to not-recognize obstacles which block part of a
lane.

Sometimes, the system will sense an obstacle and won't be able to identify it.
That has to be handled safely.

[1]
[https://www.ted.com/talks/chris_urmson_how_a_driverless_car_...](https://www.ted.com/talks/chris_urmson_how_a_driverless_car_sees_the_road)

~~~
bitL
AFAIK most SDC companies are using all kinds of control inputs, whether they
are based on classical Computer Vision (RANSAC, HOG, SVM), on Deep Learning
with 2D convolutions, on Deep Learning with 3D convolutions and RNN (2D + time
on a series of images), Deep Reinforcement Learning (lane keeping, realtime
path planning/obstacle avoidance), LiDAR+Radar+Ultrasonic combo via Kalman or
particle filters, GPS, IMU etc. and then mix all those signals together to
figure out which ones are the most probable. And this all has to happen with
super-low latency. With the latest NVidia 500W GPUs for cars we might get far
better precision than what was possible with Jetson TX2. You are right Deep
Learning alone won't make it reliably, as sometimes output goes completely
wrong and simple filtering/averaging past few frames might not be sufficient
to prevent your self-driving car becoming a self-flying car. But combining all
these inputs together, maybe via another DNN, seems like the best way forward.

~~~
skgoa
Yes, we are using a fusion of different types of sensor input. I have to be
careful to not divulge trade secrets, but I would also add that sensory input
(deep learning based or otherwise) is only the very first step in a long and
complex chain of tools, algorithms etc. The media and academia are focusing
almost exclusively on that step, but it's really only a small piece of the
puzzle we are working on.

You are also correct on the performance issues. The DNNs you would require for
the very complex decisions are currently too large to be used on automotive
ECUs. But we also still have big open questions regarding provability of
safety when using neural networks that need to be answered before we can use
them for core decision making algorithms.

------
bmc7505
University of Montreal also has an autonomous robotics class called Duckietown
(formerly at MIT). Having taken it last Fall, I can say it's pretty
comprehensive. We covered a bunch of related topics from computer vision, deep
learning, filtering, SLAM, with guest lectures from industry researchers and
roboticists: [http://duckietown.org](http://duckietown.org)

If you're in Montreal, we're having a public demo day next week.
[http://diro.umontreal.ca/departement/evenements/une-
nouvelle...](http://diro.umontreal.ca/departement/evenements/une-
nouvelle/news/duckietown-43516/)

------
j_s
This is the course home page for the YouTube video discussed briefly here 2
days ago:
[https://news.ycombinator.com/item?id=16157843](https://news.ycombinator.com/item?id=16157843)

Thread winner goes to bitL summarizing the differences between several similar
courses as he also has in this discussion:

>bitL: _TL;DR: Once you finished Udacity, MIT gives you more [...] you 'd
understand instantly (except for Deep Reinforcement Learning_

Apparently the chance to order this year's class t-shirt has been extended to
Monday!

------
jstewartmobile
So nowadays, even MIT is a vocational school? This just seems _too damn
specific_ to not be a direct industrial request.

edit: corrected per bitL

~~~
omegaworks
Expect more of this as society continues to devalue degrees that don't tie
directly into a lucrative professional career. College is expensive, with
tuitions not anchored by a limited credit market. People will rationally turn
away from fields of study that won't let them escape from their unforgivable
debt.

~~~
jstewartmobile
The joke will be on society after half of this stuff gets refined down to
ultimate simplicity (like bicycles), and the other half is refined to a point
of "good enough" (like terminals) where no one cares anymore.

------
guptat59
...It is open to beginners and is designed for those who are new to machine
learning, but it can also benefit advanced researchers in the field looking
for a practical overview of deep learning methods and their application....

Someone who took this course before, is it really good for beginners? Also,
the course title suggest it would deal with models for self driving cars.
Description says "practical overview of DL methods and their application". How
specific is the course ?

~~~
bitL
I'd say it's for intermediate level. Let's say you have already finished
1-year long Udacity self-driving car nanodegree and reached intermediate
level, then these lectures will be understandable to you. If you are total
beginner you are going to be lost, but if you are an MIT newbie, you might be
fine with some work in your spare time. To understand some subjects like Deep
Reinforcement Learning you might need some related graduate-level coursework
under your belt. It's not bleeding edge though, so researchers might think the
lectures are trivial.

------
pattycake
Dang looks like the class already started.

Would it be worth it to formally enroll in the class if you're trying to get
into the industry?

~~~
senatorobama
why would someone pick a guy who did this over an MS grad from Stanford

~~~
dang
You've posted a ton of unsubstantive comments to HN. We eventually ban
accounts that do this, so would you please stop?

~~~
skgoa
It's an important question, though. I see a lot of interest in MOOCs, the
Udacity nanodegree etc. on the internet, but I can't see my employer/me hiring
someone whose only qualifications are online courses. I know absolutely no one
in my entire department who does not at least have an MS. And we are doing
exactly the jobs that people who take these courses want to get hired for.

I like to do MOOC courses to get an overview of / intro to a certain topic I
am interested. But once people charge a substantial amount of money for a
degree, it is absolutely pertinent to question whether the degree is worth it.
E.g. IMO udacity is very close to being fraudulent with the promisses they
make in their advertizements of their nanodegree.

~~~
dang
Ok, but an important question deserves a substantive comment.

------
squared9
Shameless plugin: Here is a recent presentation I did while finishing
Udacity's Self-driving Car Nanodegree. It might help with some of the
concepts, show you what was Udacity teaching and complement the material from
MIT:

[http://bit.ly/2EOKIXy](http://bit.ly/2EOKIXy)

Have fun!

------
ronilan
Kind of disappointing to see it is about cars not _for cars_. Kitt and
Lighting Mcqueen must feel kind of left out.

~~~
taneq
What about Herbie? Just because the Love Bug doesn't have blinky lights or
smooth Pixar animation doesn't mean he's obsolete!

...We might want to think twice about admitting Christine though.

