
ChosunTruck – Euro Truck Simulator 2 autonomous driving solution - mastazi
https://github.com/bethesirius/ChosunTruck
======
hasperdi
If you want to start learning programming autonomous driving, you might start
with Desert Bus*

*) [http://www.newyorker.com/tech/elements/desert-bus-the-very-w...](http://www.newyorker.com/tech/elements/desert-bus-the-very-worst-video-game-ever-created)

------
zappybiby
Thank you for your interest guys! We went from about 20 stars to 200+ stars,
landing us on the 'trending' section of github! We could vastly improve the
program by hooking into the game's memory (Such as the totally sick mapping
done in
[https://www.youtube.com/watch?v=3YYyGGgoWfs](https://www.youtube.com/watch?v=3YYyGGgoWfs)).
But this project really just explores computer vision and ML (Kinda,
tensorflow is still being worked on).

------
pmontra
A mostly off topic side note.

It's amazing to see a game with the roads of half continent. I've been playing
driving games for so many years that I become fed up with all the grinding
(licenses, career path, races blocked by other races, etc) so I'm not buying
those games any more. I want all the cars, all the roads, no traffic, no
police, only driving. I could pay an extra for that. I wish there is something
like Euro Truck with only roads, some fast car and curated lists of
interesting road to drive on.

And kudos to the authors of ChosunTruck, obviously.

~~~
kogepathic
> I want all the cars, all the roads, no traffic, no police, only driving.

Then you don't want to play ETS2.

> It's amazing to see a game with the roads of half continent

As someone else noted, the road network is only an approximation of the
highways. Each city is essentially the same layout with slightly different
buildings to make it look like the namesake city (outside the drivable world,
basically a 3D skybox).

The AI vehicles are quite stupid. I've had them suddenly brake in front of me,
or merge into the trailer from behind. Oh, and you get penalised for this
(ticket & damage repair costs).

Apart from the PS of trucks, there's virtually no difference between them.
Want a basic Volvo? It'll haul trailers. Want the top of the line Volvo
Globetrotter with sleeping cabin? It'll haul trailers. Oh, and you still need
to stop at the designated rest areas to sleep, even though you have a bloody
sleeping compartment in the Globetrotter!

You have to follow all the laws, all the time. Speeding? That's a ticket.
Running a red light? Ticket. Bumping another car? Ticket!

I wish they would include an arcade or "god mode" so you could actually have
fun.

/rant

Can anyone comment on how this is actually obtaining the video output from
ETS2?

Is there some way you can feed the output of an OpenGL application into this?
Or is it just capturing the desktop area where ETS2 is running?

~~~
yread
What I'm missing is road construction. You can't go 100 km pretty much
anywhere without encountering something out of the ordinary - narrower lanes,
blocked lanes, lanes running on the opposite side. Also the ramps look more or
less the same no exit within exit or those weird X-shaped exits - where an
off-ramp is an on-ramp at the same time.

~~~
NTripleOne
That would make a pretty cool mod, randomised roadworks.

------
kijin
Interesting choice of a name. "Chosun" is not widely used in contemporary
South Korea unless you're in a self-deprecating mood ("Hell Chosun") or
referring to the Kim dynasty up north. You guys planning to drive to Pyongyang
one day in an autonomous truck? :)

Anyway, cool project! It's really nice to see what used to be very expensive
proprietary technologies become available for anyone with an Ubuntu box to
play and experiment with.

~~~
lifthrasiir
> "Chosun" is not widely used in contemporary South Korea unless you're in a
> self-deprecating mood ("Hell Chosun") or referring to the Kim dynasty up
> north.

I rather see it as a tongue-in-cheek name, it doesn't necessarily convey a
self-deprecating mood (though it somehow implies the comparison itself).

------
nandhp
Video:
[https://www.youtube.com/watch?v=w6H2eGEvzvw](https://www.youtube.com/watch?v=w6H2eGEvzvw)

~~~
pragmatick
As a german I can only think "Gah, why isn't it driving on the right lane?"

~~~
zappybiby
Its to avoid exits :)

~~~
astrodust
Yeah, for whatever reason the right lane very, very frequently turns into an
exit with almost no warning.

This is just how the maps are designed. It's only on the rare three lane
stretches that you're able to bump over to the middle lane confidently.

------
djsumdog
There was a GTA5 one earlier this month wasn't there?

These devs should really start/join a DARPA challenge team; try out their
ideas on real courses.

~~~
joshu
I am running
[http://selfracingcars.com/events](http://selfracingcars.com/events) in a few
months... Udacity has arranged for some folks to run code on their vehicles as
well.

~~~
sundvor
[http://selfracingcars.com/event/](http://selfracingcars.com/event/)

That looks awesome, nice!

------
notjustanymike
Also, if you haven't played ETS2, it's actually quite fantastic.

------
doozler
I appreciate this is a big ask but I find this incredibly interesting and
would love to know how it works, would someone mind walking me through some of
the main functions please? Which files on GitHub are actually doing the work
to keep the truck within the correct lines? Does this learn as it goes or have
they somehow trained it before hand?

~~~
omegaham
I'm doing my part![1]

It's not trained; it's entirely programmed.

Looking at /windows/src/main.cpp:

1\. Grab the game window with `FindWindow("prism3d", NULL)`.

2\. Get an image of said game window with `hwnd2mat` (found in
windows/src/hwnd2mat.cpp). This part is complicated, as it deals with the
Windows API.

2\. Apply Median Blur to the image, as shown with this transformation[2].

3\. After creating the 4-point correspondence between the input image
(`image`) and the output overlaid image, (`outputImg`), create an IPM object
that will perform the Inverse Perspective Mapping. This is a function that
turns a flat-angle shot into an overhead shot [3].

4\. Resize the `outputImg` to 320x240 for the smaller window.

5\. Convert `outputImg` to grayscale, storing the result in `gray`[4].

6\. Perform the `cv::blur`[5] operation on `gray`, storing the result in
`blur`. This reduces noise in the image; there are artifacts gotten from the
Inverse Perspective Mapping.

7\. Perform the `cv::Sobel`[6] operation on `gray`, storing the result in
`sobel`. This gets all of the outlines, including (surprise!) the lines on the
road.

8\. Perform the `cv::threshold`[7] operation on `sobel`, storing the result in
`contours`.

Result at this point: A flat perspective of the road has been turned into an
overhead perspective, converted to grayscale, blurred, turned into an outline,
and then thresholded to solely have the lines on the road. Note that there's
an option to perform Canny[8] as well, which is another alternative.

9\. Create a `LineFinder` object `ld` (found in /windows/src/linefinder.cpp).

10\. Perform `ld.findLines` on `contours`. This performs the `cv::HoughLinesP`
operation[9], returning a vector of lines (`vec4i` is a vector of 4 integers
that stores x_1, y_1, x_2, y_2). It'll use these to draw the detected lines
for informational purposes.

\---

Okay, now we have a black-and-white image of road-line contours, stored in
`contours`. Unfortunately, keyboard and mouse input doesn't work in Windows,
so we have to go over to linux/src/main2.cc. Note that the line-finding part
is the same thing but requires some Linux-specific stuff to get the original
image of the game.

Starting from Line 198:

11\. Define the center of the image to be `bottom_center` at column 160.

Our x-value for the pixel values is going to be `j`. Our y-value is going to
be `i`.

12\. Create a for loop that iterates through all values from y = 240 to y =
30. Inside this is effectively two loops - one that acts on the x-values from
160 to 10, and one that acts on the x-values from 160 to 310. This loop looks
at the pixel in the (j, i) spot in `contours` and sees if it detects a line
there. If it does, and it hasn't found a line yet, it sets the
`center_to_right` and `center_to_left` values, respectively. This finds the
x-values of the left and right lines. I think you know where we're going from
here...

13\. Create a `centerline`, which is going to be the average of the two.

14\. For informational purposes, gather the average `centerline` values and
the average `center_to_left` and `center_to_right` values.

15\. Move the mouse according to how far away `centerline` is from the center
of the image. I don't know how to move in ETS, so I'm going to take their word
for it that their gnarly-looking trig function is on-point.

[1] [https://i.imgur.com/cMfkFGP.gif](https://i.imgur.com/cMfkFGP.gif)

[2]
[http://docs.opencv.org/3.1.0/median.jpg](http://docs.opencv.org/3.1.0/median.jpg)

[3]
[https://marcosnietoblog.files.wordpress.com/2014/02/sample.p...](https://marcosnietoblog.files.wordpress.com/2014/02/sample.png)

[4]
[http://docs.opencv.org/2.4/_images/Load_Save_Image_Result_1....](http://docs.opencv.org/2.4/_images/Load_Save_Image_Result_1.jpg)

[5]
[http://docs.opencv.org/3.1.0/blur.jpg](http://docs.opencv.org/3.1.0/blur.jpg)

[6] [http://www.mon-club-
elec.fr/mes_images/javacvpro/javacvpro_e...](http://www.mon-club-
elec.fr/mes_images/javacvpro/javacvpro_exemple_sobel.png)

[7]
[http://docs.opencv.org/2.4/_images/Threshold_Tutorial_Theory...](http://docs.opencv.org/2.4/_images/Threshold_Tutorial_Theory_Example.jpg)

[8]
[https://3.bp.blogspot.com/-ybDW9YBG05s/UsvD3HRj7JI/AAAAAAAAB...](https://3.bp.blogspot.com/-ybDW9YBG05s/UsvD3HRj7JI/AAAAAAAABWI/5bUbVWpXx-U/s1600/canny.JPG)

[9]
[http://answers.opencv.org/upfiles/13661345957308836.jpg](http://answers.opencv.org/upfiles/13661345957308836.jpg)

~~~
cr0sh
This sounds real similar to Project 4 of Term 1 in the Udacity Nanodegree
(Advanced Lane Finding) - basically using OpenCV on a video to find the lane
markings and highlight the road.

The only real difference is some of the order (in the project, you first have
to fix the camera distortion, then do the image thresholding, then warping the
image from "perspective view" to "overhead view") and the lane finding (where
you do a "windowing" method to find the lines after doing a histogram search
to find the start of the actual lane-lines; after that, you essentially do a
polynomial fit of the found pixels, which you then use to render the road
shading, then re-warp that from flat back to perspective and overlay it on top
of the road image).

Can you tell I just recently completed all these steps...

~~~
skoocda
End of term is fast approaching!

------
boksiora
wow man this is really amazing!!!!

