Hacker News new | past | comments | ask | show | jobs | submit login
Europilot: A toolkit for Euro Truck Simulator to develop self-driving algorithms (github.com)
254 points by daftshady on Sept 16, 2017 | hide | past | web | favorite | 39 comments

I'm glad I saw this link, I was looking into building a solution based on parts of the ChosunTruck codebase[1], but Europilot seems more flexible since it's built as a "bridge" rather than a fully fledged self-driving solution.

[1] https://github.com/bethesirius/ChosunTruck

Exactly the same for me. I had ChosunTruck bookmarked but Europilot looks much nicer at first glance.

Another related thing that I wanted to check out is this: https://github.com/Microsoft/AirSim although that only gives you the sim and you have to do the machine learning part completely yourself.

Here's a YouTube playlist where a guy made a self-driving vehicle in GTA V using Python.


He also streams the self driving vehicle 24/7 on Twitch, it's quite amusing to watch, every now and then the autonomous vehicle causes an accident and ends up being chased by the Police LOL https://www.twitch.tv/sentdex

It was also very interesting in the early days because you could see the agent getting much better day after day.

haha, I switched just in time to see it meekly surrender to the police.

It looks like it still has a lot of improving to do to manage basic stuff like not accelerating into oncoming vehicles, but it's quite neat to see that its mastered the art of stealing another vehicle after writing its own off, which isn't something your average self driving neural network gets trained on.

I just started watching this series independently, it's great to see how far it has come! I had no idea there was a 24/7 twitch stream, this is hilarious.

There is quite a lot of interest in a self driving car simulator. I wonder if we could use Kickstarter to raise some money to get the game maker to add simulated radar, lidar, and cameras. You should be able to customize the position of these sensors.

A self-driving car simulator would be interesting and maybe there's some chance at making it open source?

A self-driving sim, to be realistic, is also going to need realistic traffic simulation. I helped bring traffic modeling to the computer age, that is what my business did.

When asked for a quick description, I like to say that I built the greatest traffic simulator on the planet. Of course, I'm heavily biased. Eventually, we could even render some of that data into graphics.

Yeah, so it probably wasn't a thrilling simulator with broad appeal and it required a whole lot of compute power and huge data sets. Either way, it probably would have resulted in a good, albeit very large and computationally expensive, engine for a traffic simulating game.

I signed a non-compete that lasts for life. It may not actually be legally binding, but I feel ethically bound to it. But, a game would not be competing in that space - less so if it were FOSS.

What I'm trying to say is that I'd be interested in such a project, though I'd want to seek legal council. I can program a bit, but I eventually hired professionals to do that on my behalf. Let's just say that you probably don't want me submitting actual code, but I can almost certainly help with the math involved.

I'm just spitballing, really. If someone wants to do something like this, I'd be willing to take a peek and see if there were a way to contribute. I sold and retired, but I still hold great fondness for the art. Modeling traffic isn't easy, it's a chaotic system and you never really know the initial starting state.

So, if you decide to go FOSS, let me know. If it is in C, or C++, I might even remember enough to lend a hand coding, though that's probably a horrible idea. My entire code base was rewritten by actual competent people and programmers were different back then. They were quite comfortable expressing their frustrations with working on my existing code. They were, shall we say, very eager to share their knowledge - sometimes with various levels of vulgarity.

Anyhow, to model traffic at a high level of accuracy, even things like the reflectivity of lane markings will have a quantifiable impact on traffic. You can observe the effect by monitoring traffic before and after lane markings are painted. Traffic is even influenced by the fonts used in signage. Lots of seemingly small things can be accounted for and the models made more accurate.

I'd love to lend a hand, assuming it isn't going to get me sued.

So, which was the traffic simulator you worked on? I've always been fascinated by traffic and traffic modelling, read books [1], seen articles where they use cellular automata, psychology, discrete events, etc etc.

[1] https://www.amazon.com/Traffic-Drive-What-Says-About/dp/0307...

I'd prefer to not disclose the specific names for reasons of a modicum of privacy. It's long since sold, anyhow.

But, yes! It's absolutely amazing what goes into modeling traffic as accurately as possible. When we expanded to model pedestrian traffic as well, we even went so far as to build a mock store.

It's fascinating to see how small changes can influence traffic and alter the throughput. Simple things, like frequency of signage has a large impact. There are also regional differences where whole regions will, as an average, drive very differently.

You can run all the models, collect all the data, and someone will still get drunk and reverse the wrong way on a one way street. You'll never get perfect predictions and so there are constant changes being made. That kept it interesting.

What I worked on was all proprietary. It'd be interesting to see an open source project in this arena.

How realistic are traffic simulations in games compared to what you had worked on?

I don't actually know what current games are like. I played some sim transport games back 15 to 20 years ago, and they were pretty bad. I don't play any games really, but I'd make time for a good traffic sim. I have played a few games, here and there, that included traffic but they weren't very good. Like Need For Speed and GTA? Those aren't very realistic.

I should probably try something current that runs on Linux. Got any suggestions?

Play/try out Grand Theft Auto IV (or V).

(The GTA4 is more realistic simulation.)

Thanks, I'll give it a shot. My friend's kid has a nice setup. I'm not much of a gamer. I'll buy the game for him and make him let me play it.

I'll try to add a comment the next time this subject comes up. He's got things like a seat and steering wheel/pedals set up.

On Feb 17th I posted this request to their forum: https://forum.scssoft.com/viewtopic.php?f=137&t=229146

You can show your support by giving that post a +1.

Thanks for pointing me to this. You request is well written. I imagine they would want to be paid to build that. Did you ever ask the team to give you a quote?

Say for example they open source a plugin that outputs everything you requested but didn't disclose any IP related to the game.

I didn't ask for a quote but if they just built it and sold it as a product I think they could make a whole bunch of money.

An obvious disadvantage is that you don't have LIDAR inputs. I guess it would be rather difficult to hack ETS such that it generates them.

The contents of the z-buffer could be a close approximate.

I'm not sure. LIDAR is just about the simplest thing to render in a simulation: a relatively small number of ray-scene intersections.

Simulating lidar returns in a way that has noise similar to real devices is difficult

Indeed. There are tons of things that happen in real world lidar. For ex, if object is too close it may not detect. When you are going at high speed, returned light ray may arrive early or late giving bit of error. Also, think about multiple reflections, noise, cone size and so on.

Tell that to tesla !

How does training the NN using a limited frame per second affect the training accuracy? Why not go higher or lower than the one you are using?

In terms of capturing the screen input for training, the fps is capped at 10 fps. This is following the number suggested in NVIDIA's paper[1], which cautions against overfitting. The program captures faster when the steering angle gets larger, because it suggests that curvy road is ahead, which are the more challenging scenarios for the program.

In terms of testing, the max fps is determined by the performance of the GPU and the complexity of the model. I couldn't find any definitive studies done on the matter, but anywhere from 10-30 fps seems to work OK.

[1] https://arxiv.org/abs/1604.07316

Add a RNN with a 10-frames history on top of NVidia's model and you get the winning Udacity challenge entry (mentioned on selfdrivingcars.mit.edu). I assume you also programmed this model during Udacity's Self-driving car nanodegree.

Does this publish CAN bus events ?

I was looking into similar solutions to capture CAN bus events for telematics usecase.

It's a video game so there aren't any CAN bus events.

That sounds like an interesting idea for a mod though...

Do you guys know if it's possible to programmatically alter ETS environment? (such as weather, brightness, traffic, etc). Thinking of using the simulator to generate augmented data.

Most likely. All those effects are already built-in, and it looks like there's a pretty comprehensive modding community.

There might not be an API callable from outside the app, though...

This is brilliant! I built a similar system on top of Flight Simulator for an internship. So useful!

> With europilot, you can capture the game screen input, and programmatically control the truck inside the simulator.

Can't this be easily generalized to any game?

Yes, and this is by design. However the controlling logic will probably have to be customized for each additional game.

Euro Truck Simulator was chosen because it runs on Windows/OS X/Linux, and because the game consists mainly of highway roads, which is an easier target than urban environments.

Also I just really like the game itself.

This is really cool. Consider introducing this concept to an FPS game, at what point do you consider it to be an aimbot?

That is exactly what aimbots are. Sometimes you can make bots easier by using an addon system to export information(WoW), or themes to color your targets (CS), but at the root you read the screen and control the mouse.

No. At root you read and write the game's memory. Most aimbots just read the memory to get locations and call the game's own methods to adjust the aim from their injected dll. Barely any actually bother reading the screen, it's horribly inefficient and limited.

I've used and written a ton of bots that read the screen, sometimes it's the best option.

Ditto, but with manipulating the game directly either through memory reading/writing, dll injection or packet sniffing.

Can you give some examples where screen reading was more effective?

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact