
A self-driving toy car using end-to-end learning - sytelus
https://github.com/experiencor/self-driving-toy-car
======
2dollars27cents
Check out [https://diyrobocars.com](https://diyrobocars.com) and
[http://donkeycar.com](http://donkeycar.com) for similar projects. There are
great communities around both, especially if you want to build your own.

~~~
cconcepts
I feel like a whole new world has just opened up before me. I have a rasp pi,
what is the route with least technical barriers for me to get into this and
start tinkering?

~~~
kajecounterhack
For DonkeyCar: follow the guide, join the slack group, ask questions!

You'll need to purchase a number of things more than the raspberry pi itself
(camera, battery, car, servo controller board, wires, 3D print or purchase or
build a mount for it all).

------
mike_mg
Funnily enough I did very very similar project half a year ago or so:

Code: [https://github.com/ghostFaceKillah/mini-self-driving-
car](https://github.com/ghostFaceKillah/mini-self-driving-car)

Writeup:
[https://ghostfacekillah.github.io/car.html](https://ghostfacekillah.github.io/car.html)

Good job for getting PWM steering right, I had only 3 state steering (full
left, center, right) and it turned out to be quite a limitation in the end.

~~~
kookoopuffs
best github username. Mighty healthy!!!

------
Karlozkiller
I had to look up what 'end-to-end' means in regard to neural nets and machine
learning. And the answer seems to differ slightly or very much with every
webpage I hit.

Stack exchange says it means 'all classifiers are trained jointly', nvidia and
quora seem more in line with what I assume it means in this project. 'No human
input to training sets or results as part of learning' is that what this talks
about?

Is it then, unsupervised learning?

Really cool project!

~~~
skgoa
In the context of automated driving, 'end-to-end' tends to mean that the net
learns a transformation function that maps input images to steering wheel and
pedal positions. In that sense, there is 'no human input' at any stage.

In contrast to this, every significant AD effort uses hand crafted parts for
most of their pipeline. Machine learning is mostly used in the initial object
detection and tracking steps.

------
ghgr
Nice project! In a similar vein there is this project [1] who claims _online_
end to end learning in a Raspberry Pi (included training).

[1] [https://youtu.be/0ibVhtuQkZA](https://youtu.be/0ibVhtuQkZA)

~~~
taneq
Wow, they seem to be using Hierarchical Temporal Memory rather than more
traditional deep learning approaches. We've seen some rather grandiose claims
about HTMs' capabilities from Numenta (who I think invented the approach?) but
no really convincing applications yet, so it's really interesting to see a
concrete demonstration like this.

------
tostitos1979
Here is another one in Python/Lego mindstorms:
[http://conference.scipy.org/proceedings/scipy2012/pdfs/iqbal...](http://conference.scipy.org/proceedings/scipy2012/pdfs/iqbal_mohomed.pdf)

------
edshiro
NVidia published a very interesting paper on this technique called "End-To-End
Learning For Self-Driving Cars":
[https://arxiv.org/abs/1604.07316](https://arxiv.org/abs/1604.07316) . It's an
enjoyable read.

The car learns to steer itself on an empty road. It's a good experiment to
witness the power of deep learning and neural nets. For autonomous vehicles
though, you need much more than that (e.g. sensor fusion, obstacle detection,
localization, behaviour prediction, trajectory prediction, path planning,
motion control, etc.).

~~~
Bromskloss
From that article:

> Compared to explicit decomposition of the problem, such as lane marking
> detection, path planning, and control, our end-to-end system optimizes all
> processing steps simultaneously. […] Better performance will result because
> the internal components self-optimize to maximize overall system
> performance, instead of optimizing human-selected intermediate criteria, e.
> g., lane detection.

One can imagine that it might be more difficult to get a network to solve this
large problem all at once, and that there might be easier to decompose the
problem and solve each part. Would it be a good idea to guide the end-to-end
system by first decomposing the problem and solving each part, then using that
solution as a starting guess for the whole problem? I mean, the decomposition
might perhaps be a reasonable approximation of how the whole problem should be
solved. (Then again, it might not.)

------
skypanther
Perhaps I'm misunderstanding this. But why is a CNN necessary for predicting
the turning angle? Seems like OpenCV would be sufficient (assuming sufficient
contrast between the line and the floor). Grab image, threshold, find the
largest contour around the line, grab its minAreaRect() and from that
calculate the angle.

Editing to add -- this is still a cool project. I don't mean to detract from
it by pointing out that I think it could be done without the AI piece.

------
amelius
Are there any DIY projects for creating a LIDAR?

~~~
joshvm
You need to be more specific about what you want to build.

A true LIDAR system infers the round trip time of photons, either directly
(time-of-flight) or based on some proxy like phase or frequency shift
(AMCW/FMCW) LIDAR.

Lots of people include triangulation systems, which I disagree with. For
example the RPLidar A2 is just a nicely packaged version of Kurt Konolige (et
al)'s RevoLDS [1]. It projects a laser spot and takes a picture of it. Add in
some known geometry, and you can use triangulation to measure distances. It's
easy to build, you can do it with any old camera and a laser pointer. It's
incredible how overpriced the A2 is, given that the Revo was published as $30
system.

A true LIDAR is much more difficult to build. If you go time-of-flight then
you need a good laser and excellent timing electronics. If you go phase-shift
then you need beam mixing/separation optics, beam modulation and a phase
detector. Neither of these are easy for the average hobbyist. If you're
interested, buy a laser tape measure from Leica and have a look inside to see
how it works.

I belive the LIDAR-Lite is a proper LIDAR system - it ranges up to 40m which
is a good hint that it's probably a true time of flight system. The Scanse
Sweep is a LIDAR-Lite turned into a 2D scanner, like the sibling post. A good
indicator is price. Real LIDAR systems are expensive - starting at £1k
typically for something like a Hokuyo. They also tend to be much better built
than the hobbyist stuff.

In principle all you need is a pulsed laser, a big collecting lense, a fast
rise-time photodiode and a good (picosecond accurate) timing circuit.

[1] [https://www.robotshop.com/media/files/PDF/revolds-
whitepaper...](https://www.robotshop.com/media/files/PDF/revolds-
whitepaper.pdf)

------
the_rock_says
UPENN had something called F1 autonomous racing competition where they design
and build 1/10th version of a F1 car race. They also have a tutorial on how to
build one. Here's the link: [http://f1tenth.org/](http://f1tenth.org/)

------
KempFood
Toy cars seem perfect for perfecting AI driving skills.

With a model train like setup, you could have construction, peds, etc.

Wow!

------
bjornsing
Does anybody understand where the supervision (target steering angles) comes
from? I checked Learn to Drive.ipynb but that seems to just read steering
outputs from a file. Shouldn't there be manual "labeling" involved?

~~~
edshiro
The steering angles are probably automatically obtained from the script you
run when manually training the toy vehicle. I presume at every frame an image
is captured along with the steering angle for this image.

You then create a neural net architecture that is being fed images and
steering angle positions and outputs steering angle predictions. This is a
regression problem that can be expressed in plain English as:

"First train the neural network with a collection of images and associated
steering angles. After training, if I were to give the neural network a new
image it has never seen before, what steering angle would the network
predict?"

NVIDIA has a paper on it[1], and I blog about a similar coursework I complete
as part of Udacity's self-driving car nanodegree[2]

[1] [https://arxiv.org/abs/1604.07316](https://arxiv.org/abs/1604.07316)

[2] [https://towardsdatascience.com/teaching-cars-to-drive-
using-...](https://towardsdatascience.com/teaching-cars-to-drive-using-deep-
learning-steering-angle-prediction-5773154608f2)

------
pishpash
This is what I don't understand. Self driving cars could be doing lots more
toy simulations in toy worlds before "having to" collect data on public roads.

~~~
sp332
Self-driving cars are already reasonably good most of the time. They need to
get better at specific cases, which would be a lot harder to model in a toy.
Stopping distance is going to be a lot different, just for one example.

------
amelius
Nice, but the downside is that you can only train a rather limited set of
traffic situations this way.

------
mythrowaway1124
This is the best kind of project to read about on HN. Well done!!

