This is super cool, especially how it is done as a side project. It's resourceful using a game as the physics simulator. I wonder how it compares to Microsoft's [1] and Udacity's [2] implementation.
I think a fair comparison would be to AirSim [1] and Carla [3]. These are much more mature projects and are similar to Autodrome in a lot of ways. As far as I know Udacity's simulator is done their class and it's not being actively developed.
+ Both [1] and [3] have much fewer assets (like 3D models of houses, factories, bridges, cars, trucks, and so on) you'd have to buy them on the Unreal/Unity model marketplace and it still wouldn't be enough. Autodrome can take advantage of almost entire Europe and a third of USA at 1:20 scale.
+ Autodrome has a sparse map representation that is really easy to randomly fuzz. I.e. it's easy to shift a segment of the road a little bit and see how the algorithm would react to the fuzzed scenario. I believe this is only way how to achieve robust agents and effectively prevent testing on the training set.
- Biggest disadvantage of Autodrome is a lack of access to in-game dynamic NPCs (like other trucks, cars or pedestrians). As far as I know there's no API for this. Without help (or a lot of very fragile memory hacking) from the developers of the game this feature is very hard to achieve and both [1] and [3] already have it.
Thank you for the detailed response! Is there any thoughts on ground truth segmantic segmentation camera view? Simulated lidar data would be super awesome too.
Both are also not easy like the dynamic NPCs. I think Carla [3] supports both raytraced lidar and segmented rendering now so this is another minus “-“ point.
Not as fancy as the other ones but Nasa has a toolbox called trick[1]. This is completely numerical based and allows you to do all the necessary numerical calculations easily.
I love how this is based on a game. One might argue that software development was really accelerated by games just as the hardware development was accelerated by the military (by hardware I mean physical stuff, like materials etc.).
Not sure if I entirely agree with this. A lot of the luxuries we have now were enhanced for the military. For instance, airplanes (aileron and jet turbine development), internet, semiconductor, and even PCBs (or mores specifically, the predecessor Hybrids ICs). A good amount of grants actually come from the Navy and Army for research grants in the academia. Though I did go to grad school for Mech Eng so we probably saw more grants from the military because of that.
That is a logical fallacy. Sure a lot of things happened because of the military, but what didn't happen because we were busy on the military things. There is no way to know.
If we had spent as much on genetic research as we did on the moon landing we could have had fire breathing dragons. Think of all the spin offs from that that we are just starting to see now, we could have had them sooner.
It’s true that military necessity has spurred a lot of things, but it sure would be nice if we had the same cultural agreement around basic research without that.
[1]https://github.com/Microsoft/AirSim [2] https://github.com/udacity/self-driving-car-sim