
Europilot: A toolkit for Euro Truck Simulator to develop self-driving algorithms - daftshady
https://github.com/marshq/europilot
======
mastazi
I'm glad I saw this link, I was looking into building a solution based on
parts of the ChosunTruck codebase[1], but Europilot seems more flexible since
it's built as a "bridge" rather than a fully fledged self-driving solution.

[1]
[https://github.com/bethesirius/ChosunTruck](https://github.com/bethesirius/ChosunTruck)

~~~
rakete
Exactly the same for me. I had ChosunTruck bookmarked but Europilot looks much
nicer at first glance.

Another related thing that I wanted to check out is this:
[https://github.com/Microsoft/AirSim](https://github.com/Microsoft/AirSim)
although that only gives you the sim and you have to do the machine learning
part completely yourself.

------
sosa2k
Here's a YouTube playlist where a guy made a self-driving vehicle in GTA V
using Python.

[https://youtu.be/ks4MPfMq8aQ](https://youtu.be/ks4MPfMq8aQ)

~~~
mastazi
He also streams the self driving vehicle 24/7 on Twitch, it's quite amusing to
watch, every now and then the autonomous vehicle causes an accident and ends
up being chased by the Police LOL
[https://www.twitch.tv/sentdex](https://www.twitch.tv/sentdex)

It was also very interesting in the early days because you could see the agent
getting much better day after day.

~~~
notahacker
haha, I switched just in time to see it meekly surrender to the police.

It looks like it still has a lot of improving to do to manage basic stuff like
not accelerating into oncoming vehicles, but it's quite neat to see that its
mastered the art of stealing another vehicle after writing its own off, which
isn't something your average self driving neural network gets trained on.

------
anindha
There is quite a lot of interest in a self driving car simulator. I wonder if
we could use Kickstarter to raise some money to get the game maker to add
simulated radar, lidar, and cameras. You should be able to customize the
position of these sensors.

~~~
KGIII
A self-driving car simulator would be interesting and maybe there's some
chance at making it open source?

A self-driving sim, to be realistic, is also going to need realistic traffic
simulation. I helped bring traffic modeling to the computer age, that is what
my business did.

When asked for a quick description, I like to say that I built the greatest
traffic simulator on the planet. Of course, I'm heavily biased. Eventually, we
could even render some of that data into graphics.

Yeah, so it probably wasn't a thrilling simulator with broad appeal and it
required a whole lot of compute power and huge data sets. Either way, it
probably would have resulted in a good, albeit very large and computationally
expensive, engine for a traffic simulating game.

I signed a non-compete that lasts for life. It may not actually be legally
binding, but I feel ethically bound to it. But, a game would not be competing
in that space - less so if it were FOSS.

What I'm trying to say is that I'd be interested in such a project, though I'd
want to seek legal council. I can program a bit, but I eventually hired
professionals to do that on my behalf. Let's just say that you probably don't
want me submitting actual code, but I can almost certainly help with the math
involved.

I'm just spitballing, really. If someone wants to do something like this, I'd
be willing to take a peek and see if there were a way to contribute. I sold
and retired, but I still hold great fondness for the art. Modeling traffic
isn't easy, it's a chaotic system and you never really know the initial
starting state.

So, if you decide to go FOSS, let me know. If it is in C, or C++, I might even
remember enough to lend a hand coding, though that's probably a horrible idea.
My entire code base was rewritten by actual competent people and programmers
were different back then. They were quite comfortable expressing their
frustrations with working on my existing code. They were, shall we say, very
eager to share their knowledge - sometimes with various levels of vulgarity.

Anyhow, to model traffic at a high level of accuracy, even things like the
reflectivity of lane markings will have a quantifiable impact on traffic. You
can observe the effect by monitoring traffic before and after lane markings
are painted. Traffic is even influenced by the fonts used in signage. Lots of
seemingly small things can be accounted for and the models made more accurate.

I'd love to lend a hand, assuming it isn't going to get me sued.

~~~
JoeDaDude
So, which was the traffic simulator you worked on? I've always been fascinated
by traffic and traffic modelling, read books [1], seen articles where they use
cellular automata, psychology, discrete events, etc etc.

[1] [https://www.amazon.com/Traffic-Drive-What-Says-
About/dp/0307...](https://www.amazon.com/Traffic-Drive-What-Says-
About/dp/0307277194/ref=sr_1_1?s=books&ie=UTF8&qid=1505601119&sr=1-1&keywords=traffic)

~~~
KGIII
I'd prefer to not disclose the specific names for reasons of a modicum of
privacy. It's long since sold, anyhow.

But, yes! It's absolutely amazing what goes into modeling traffic as
accurately as possible. When we expanded to model pedestrian traffic as well,
we even went so far as to build a mock store.

It's fascinating to see how small changes can influence traffic and alter the
throughput. Simple things, like frequency of signage has a large impact. There
are also regional differences where whole regions will, as an average, drive
very differently.

You can run all the models, collect all the data, and someone will still get
drunk and reverse the wrong way on a one way street. You'll never get perfect
predictions and so there are constant changes being made. That kept it
interesting.

What I worked on was all proprietary. It'd be interesting to see an open
source project in this arena.

------
amelius
An obvious disadvantage is that you don't have LIDAR inputs. I guess it would
be rather difficult to hack ETS such that it generates them.

~~~
wyager
I'm not sure. LIDAR is just about the simplest thing to render in a
simulation: a relatively small number of ray-scene intersections.

~~~
CardenB
Simulating lidar returns in a way that has noise similar to real devices is
difficult

~~~
sytelus
Indeed. There are tons of things that happen in real world lidar. For ex, if
object is too close it may not detect. When you are going at high speed,
returned light ray may arrive early or late giving bit of error. Also, think
about multiple reflections, noise, cone size and so on.

------
m3kw9
How does training the NN using a limited frame per second affect the training
accuracy? Why not go higher or lower than the one you are using?

~~~
imgyuri
In terms of capturing the screen input for training, the fps is capped at 10
fps. This is following the number suggested in NVIDIA's paper[1], which
cautions against overfitting. The program captures faster when the steering
angle gets larger, because it suggests that curvy road is ahead, which are the
more challenging scenarios for the program.

In terms of testing, the max fps is determined by the performance of the GPU
and the complexity of the model. I couldn't find any definitive studies done
on the matter, but anywhere from 10-30 fps seems to work OK.

[1] [https://arxiv.org/abs/1604.07316](https://arxiv.org/abs/1604.07316)

~~~
bitL
Add a RNN with a 10-frames history on top of NVidia's model and you get the
winning Udacity challenge entry (mentioned on selfdrivingcars.mit.edu). I
assume you also programmed this model during Udacity's Self-driving car
nanodegree.

------
crudbug
Does this publish CAN bus events ?

I was looking into similar solutions to capture CAN bus events for telematics
usecase.

~~~
gregschlom
It's a video game so there aren't any CAN bus events.

~~~
jeroenhd
That sounds like an interesting idea for a mod though...

------
tomdre
Do you guys know if it's possible to programmatically alter ETS environment?
(such as weather, brightness, traffic, etc). Thinking of using the simulator
to generate augmented data.

~~~
scott_karana
Most likely. All those effects are already built-in, and it looks like there's
a pretty comprehensive modding community.

There might not be an API callable from outside the app, though...

------
pj_mukh
This is brilliant! I built a similar system on top of Flight Simulator for an
internship. So useful!

------
amelius
> With europilot, you can capture the game screen input, and programmatically
> control the truck inside the simulator.

Can't this be easily generalized to _any_ game?

~~~
zpr
This is really cool. Consider introducing this concept to an FPS game, at what
point do you consider it to be an aimbot?

~~~
octalmage
That is exactly what aimbots are. Sometimes you can make bots easier by using
an addon system to export information(WoW), or themes to color your targets
(CS), but at the root you read the screen and control the mouse.

~~~
tokenizerrr
No. At root you read and write the game's memory. Most aimbots just read the
memory to get locations and call the game's own methods to adjust the aim from
their injected dll. Barely any actually bother reading the screen, it's
horribly inefficient and limited.

~~~
octalmage
I've used and written a ton of bots that read the screen, sometimes it's the
best option.

~~~
tokenizerrr
Ditto, but with manipulating the game directly either through memory
reading/writing, dll injection or packet sniffing.

Can you give some examples where screen reading was more effective?

