I'd be lying if I said OpenAI Universe / GTAV wasn't one of the primary drivers in me trying to learn ML.
Anyone know what kind of hardware I will need to play with this? The AMI page on github mentions an AWS g2.2xlarge (iirc a GTX 1060 is slightly better than this?). And it seems like GTA is actually running on a different system you use VNC to share the screen/input? Any estimates about whether a 1060 or 1080 on a beefy gaming PC could handle both at once?
Most reviews I've read actually complain that GTAV is a step back in realism from GTA4 (for example, body-part specific injuries were dropped), but let's not beat about the bush - no GTA deserves the title of "realistic", because as you say "it's GTA and you have to shoot everyone, duh". They're made to feel realistic, but in fact they powerfully reward violence and punish being law-abiding, in precisely the opposite way that the real world does.
It didn't try to be realistic. The whole point is that the police response is always massively disproportionate to the crime.
It's a pastiche of society, everything is exaggerated: they have a legalize medicinal cocaine campaign, you can buy supercars online, rocket launchers and machine guns at your local gun shop, you can launch your car at 100 miles an hour of the edge of the highway and survive.
They even reference the outrageousness of it in a scene where Michael goes to the therapist and comments that he may have even killed someone on the way there.
Like obviously no real person would be able to survive the injuries my character gets every 10 seconds. But if I was dying all the time it wouldn't be fun to play.
The one thing that really bothers me, as far as realism goes, is lack of persistence. Leave a car somewhere, walk 10 feet away, turn around, and it vanishes. Cars and NPCs vanish when you aren't looking. Damage you do gets quickly reset as if it never happens. And a number of small details like this. It's not just that it's unrealistic, it really limits what you can do with an otherwise awesome sandbox.
Still, I think the game that deserves most credit for realism is dwarf fortress. It will be a long long time before AIs master that.
If the game randomly JIT generates all local phenomena while keeping just enough coherent higher level data to suspend your disbelief, that's only the storage size of the high level information.
However, when a player begins to specifically perturb that generated phenomena at the detail level, how do you prevent leaving an ever-storage-requirement-increasing trail back through the player's space-timeline? Reconvergence to normal over a reasonable timespan?
I assume this is a thought about problem, so links greatly appreciated!
Look into confirmation bias
I think a lot of time outside observers are better at parodying something than someone who is inside of it. They just nail the ridiculous culture in the US. I remember GTAIV had a television you could watch, and "Republic Space Rangers" was one of the funniest shows I've seen anywhere.
Every animal in GTA V will call the cops on you if they witness you doing a crime -_-
You're right, realistic maybe wasn't the best word. I think "believable" would have gotten my point across better. It's true that GTA V mimics the real world pretty terribly, but it's game world and AI is so incredibly internally consistent that it feels less like a game and more like a another world.
> I accidentally bumped a police boat with my jetski. Apparently this warranted being shot in the head.
Were you playing as Franklin? The GTA V devs have denied this, but from personal experience and plenty of online reports, the police in the game are racist. More realistically, it's probably because they're unable to pathfind to a state where they can arrest you (because you're both on boats on water), so they just go guns blazing.
Except if you're into politics.
depends on what you prioritize to mean "realistic." I think most people would agree that the world of GTA V (and IV) is incredibly realistic by video game standards, to the degree that it's the obvious choice for this AI project.
In fact it's quite the oppose. What makes games fun is taking what we call reality and make it "closer" and "unreal" at the same time. Like ... "hey, look, i'm shooting a RPG ...". But it's does nothing to do with shooting a RPG, you are just pressing a key and seeing something shining in a block of glass.
Problem with games are that they trick you mind to think something is real. But it's far, very far away from the real. It's just you, sitting at there, imagining things.
These days I play an undead thief in Dark Souls 3. (But maybe it is realistic, I have no idea how professional thieves live.)
(I had right of way and they didn't have lights/sirens on)
I don't know--these day's I'd believe it.
Sounds like a 1060 should be a good start for me to play with Universe then.
The transitive property of teaching an AI to drive given a world simulation, makes me wonder if you can train a network to recognize speech by feeding War and Peace through a text to speech program and then sending the audio data generated into an RNN. Where is the ground truth in such a system which you can calibrate your results by?
Answering standard reading comprehension test questions? - "Why did Pierre meet Natasha at the church?"
But I do think GTA's unrealistic physics engine will hurt it for simulating the cases we care most about (accidents, lost of traction, etc)
 GTA is a regular submission on /r/GamePhysics which collects funny game physics glitches: https://www.reddit.com/r/GamePhysics/search?q=GTA&sort=top&r... https://www.reddit.com/r/GamePhysics/search?q=GTAV&sort=top&... https://www.youtube.com/channel/UCy332GE6lcMVP612IUs7ODw/vid...
my argument then was 'imagine prototyping on a machine that needs to be kept at 17millikelvin'
here the argument is 'imagine prototyping on a two ton missile traveling at lethal speeds through highly populated areas'
Next step is obviously teaching it how to kill people in the game and get away from police.
I'd like to see the AI go on a rampage and win :).
That should be enough to wipe off a lot of smiles off our idealistic faces :)
anyways, electronic actuation of a firearm is already illegal in the US. if you're firing a gun in any way besides actually pulling a trigger with a finger, chances are good it's illegal (with some small leeway such as turning a handcrank)
But if you're going for some sort of obscure criminal achievement by only breaking one law at a time, then just make your lethal device something that does not legally qualify as a firearm. A black-powder muzzle-loader or a Bulgarian ricin-umbrella would do the trick, depending on whether you care about noise or not.
Or, like the parent mentions, make a grenade, and deliver it via remote-piloted aircraft.
Realistically, though, very few people on this planet would bother with any of that when they can often just kick down the door of your home, kill you with a tool purchased from a hardware or sporting goods store, and just leave before police can respond. The clearance rate for violent crimes in the US is already low enough, even for people who give no thought whatsoever to covering their tracks.
I'll continue to assume that unmanned aircraft are carrying cameras, and assume that if someone wants to murder me, they need a compelling reason to have a more complicated plan than to just bash my skull in with a big rock when I'm not looking.
Huh, that explains why Mythbusters always used a string rather than any sort of servo system. Neat.
https://www.youtube.com/watch?v=nTs7VRFV36c (and others)
Substituting other devices to be actuated seems... trivial, and well within the reach of even hobbyists.
ABSTRACT: We explore the adage "pimpin' ain't easy" in the context of deep reinforcement learning in the OpenAI environment GTAV. We train a Deep Q-Network (DQN) to optimize player cash flows. Over the course of training, our network learns to keep these hoes in line, and discipline misbehaving customers.
Will be super interesting to see how all of the randomness in GTA V driving influences autonomous driving AIs.
I could see them being a bit more "cautious" in order to deal with all of the crazy drivers (and events) sharing the roads of Los Santos.
Also it'd be interesting to see how it reacts to getting shot at
It might just as well don't care about being shot at.
distance_reward - off_road_penalty - speeding_penalty - slow_penalty - discomfort_penalty;
Imagine a game like Warcraft 3 where the units actually seem to have a personality and follow "their guts" in battles rather than following simple patterns like attacking the nearest enemy.
Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?
> Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?
It's a really tough trade-off between fun and realism. It's cool that your units have personality for a bit "Oh, look, those 2 dwarves are fleeing from the troll fight early!", but not fun once they disobey your intentions "That fucking troll has 3 hit points left. If my 2 dwarves didn't run away, it would be dead! Now it's destroying my base!"
One game I remember striking a good middle ground with this was the original Battle For Middle Earth. It changed animations, sounds and other aesthetic things based on AI, while the game remained the same mechanically. If your units ran up against a mountain troll, you would see the ones in front of it cowering and holding up shields, while units behind attack furiously, but every unit was really doing the same damage. Or if you left troops of Orcs just standing around too long, they'd break out into fights and kill other orcs, but the animation is done as a trick where as one orc model "dies", a new one spawns in the group. So you didn't really ever lose Orcs, even though there would be orc corpses strewn wherever you had been camping.
I think the main reason though is just that AI doesn't really sell. Graphics, physics and voice actors sell. You can blame a failure on AI, but very rarely is success attributed to good AI in a game.
This is hardly arguable, but personally, after some time playing I begin to recognize animation/sound/timing (i.e. behavioral) patterns of non-plot entities and that ruins the entire experience. That's what retains me on the storyline and prevents free exploration -- there is nothing really interesting besides nice graphics. I think there is a room to evolve from [hundreds of] programmed random encounters to purely independent world.
Actually, I'm fine with GTA3/GTASA-like graphics, like I was with roguelikes and physical book-and-dice quests, so idk how that applies to wide public.
ps. Another conclusion, it may appear too frustrating. We probably play games to win and achieve. If there is no end, no goal and no perfect winning strategy, that understanding alone may cause depression, like IRL.
The main reason that I avoided buying Civilization 6 is that the AI was notoriously bad. For any kind of turn-based strategy game, I want the AI to be superb.
*I didn't intentionally rhyme lackluster with gangbuster. Rather these are the words that sprang initially to mind, and it's a kind of satisfyingly weird combo, so I'm leaving it in.
For instance, bad pathfinding in an RTS game can make the game almost unplayable. Halo is just about legendary at this point for teammate AI that is worse than worthless . Any one of these things can destroy immersion at a critical point.
AI is to video games as IT is to a modern enterprise. If it's doing its job, nobody notices, but if it's bad, it sticks out like a sore thumb.
My former company had a backgammon game with a really good AI. We constantly got accused of cheating, even though the dice-roll algorithm was identical (it didn't have a parameter to specify who the roll was for). What people missed is that a good backgammon player will position themselves to benefit from a wider set of possible dice outcomes.
This is what I said though. A game's failures can be blamed on bad AI. But once you reach the point of okay AI, you stop hearing about it. There's very little reason to put in whatever effort is required to make your AI great, because the only people who will take notice are hardcore players who put in tons of hours. And that's nice and all, but the game costs $60 whether you play it for 10 hours or 100. So why spend millions on a feature that's hard to make a trailer for, and is only noticeable to your most hardcore fans?
As discussed in other comment, it was a pain to manage them, because they didn't follow all the (stupid/suicidal) orders.
I'm not sure that's the case. Last of Us and MGS:V both had intelligent AI where they would change their behaviour based on what you've done in the past and work together as a team to flush you out of a hiding spot. This was widely covered and praised in reviews and pre-launch interviews.
Most of them look pretty much like that, with very little difference until you start getting to really low-end or old chips. Games are mostly GPU limited these days and those that are CPU limited are mostly leaning on one or two cores and thus haven't really gained much from recent generations.
If anything, this shows that games aren't strongly multi-threaded (which comes with severe development and debugging impact) or aren't made to scale beyond the highest end chips consumers are likely to have.
Lastly, consider that while the graphics effects and physics effects of a game can be scaled down, scaling down the AI in many instances would cause balancing issues.
The Valve Developer Community has some okay documents about the AI , moddb has some nice articles too (no links offhand). And most of the source for combine guards is available in the Source SDK , you can see tons of if's and hard coded sequences of actions.
I've read that the AI was changed after the Orange Box release, so maybe there was some learning features to them originally. Or maybe those features do exist within HL2's private source files, though it doesn't "feel" like it.
If it thought players had had it easy for a while it might throw a larger waves of monsters and specifically deny spawning health for a few minutes.
There's a good presentation somewhere about how it would try to balance the rising action with periods of "stress" and "relief" over the course of a level.
Have to start somewhere, though.
As a side note, the writeup of F.E.A.R's AI is a fascinating read. By combining very simple goals and actions, they created AI that is still remembered as incredibly good. http://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear....
I wonder, what makes it hard to rely on solely the 2 cameras to calculate/guess the depth, similarly to the way us non-robotic drivers do it, when it comes to machine sensing.
EDIT: found this: https://web.stanford.edu/class/ee368/Project_Autumn_1516/Rep... on the topic. Is there anything else I should read up on?
EDIT: found lots of articles: https://www.google.com/webhp?sourceid=chrome-instant&ion=1&e...
There's a ton of them on YouTube, but I can't figure out how they make them.
Some use the GTA V. director:
The creepy thing is that this video is targeted at 0-3 year olds who randomly tap on mom's iPad. If you think like that now it's clear why the title is just keyword stuffing, the lame music and the spiderman outfit.
80 million views in a month is a lot of money:
Just checked their channel: 4,316,098,805 views
What would 80 million views translate to in dollar terms?
Where do you see that count?
be careful, those videos melt your brain in an instant
Here's few links I found so far:
Also I'm reading that GTA folks went out of their way to prevent modding. So may be everything you do with GTA is a hack and there is no supported APIs for developers?
That said, they don't publish anything about modding and don't have an official API available. Everything has been done third-party.
The modding has caused real problems in the online play - player in godmode, handing out in-game money, etc. No matter the efforts of R*, they seem to be unable to eradicate it for good. The group of players I previously played the game with, including myself, have all become extremely disenchanted with the game by the online modders.
The anticheat process itself will hide and run in kernelmode. I don't know about you, but I don't feel terribly comfortable having a closed source binary running in ring0.
But, I run on a machine that was purpose-built for running games and if it all goes to hell, I just wipe and reload. All the games are from Steam, so they're easy to get back on there (and I only play a handful of those, so ...).
We have algorithms, data, and computing power which are unevenly distributed. Providing easy-to-access training data for a self-driving car feels like a huge step in democratizing the data aspect of car AI.
Did you see the "self driving mario kart"? 
That's a lot of complexity just to try out some algorithms for driving.
Is GTA V the same as setting up a rig which controls I/O for a real car? No. Of course not. You're not being fed data from LIDAR, which Waymo [Google X driving] seems quite invested in. 
However, much of Waymo's self-driving training is simulated [of course it is, how else would you get enough data?]. This is filling out a gap that is currently otherwise only filled by proprietary solutions. That's a big deal.
I've been coming around to the conclusion that LIDAR isn't going to be the way forward, unless prices come down greatly and mechanical scanning is done away with. Perhaps something like Flash LIDAR, if it could be made 360 degree FOV and cost under $500.00 in quantity - that might be worthwhile for manufacturing a vehicle with, but currently even the cheapest 3D LIDAR is still way too expensive.
I honestly think we'll see camera-based solutions becoming the norm; high-speed, high-resolution machine-vision cameras are already an established technology (though mainly black-and-white; pair those with lower-res color cameras and fuse the output as needed). Set up multiple stereo-camera pairs for overlapping 360 coverage - it would likely be a fairly cheap solution.
At that point - the frame-buffer view data from GTA (or some other simulator) could be considered close to "real world" for data gathering and model training.
[which I also think will happen]
I think it will become cheaper - probably to ~$1-2k in the next two years.
And even if it doesn't, we aren't expecting self-driving cars to replace the fleet of cars on the road, we're expecting to increase utilization % of cars. So we won't need as many vehicles, and the cost per mile will still be much cheaper.
But GTA V is scriptable. We'll be able to simulate LIDAR in-game within a few months, if desired. So... all-told, I'm pretty excited about the possibilities here.
This is really awesome, great work to those involved.
re snow: It is possible to lay fresh snow anywhere in addition to drive through this area with snow burms, etc..
Traction is definitely affected in snowy areas.
re tires bursting: it seems possible
i'm sure this is to offer relief to the nn but does this further simulate 'reality'?
are our autonomous vehicles seeing at 8 frames per second?
there is more going on than just input in that reaction time figure
8fps fails to assume a reaction time of 0.125s for the same reason that the human eye's ~220fps.. now this is a mental stretch of the true underlying mechanisms but i'll make it for sake of consistency of argument(o).. fails to imply a reaction time of .00454545s
i guess i assumed autonomous vehicles sensor data would be measured in hertz, but now am intrigued by what framerate the computer vision system works
Anything that has wheels is dangerous to humans!
For instance, they could have gone with TORCS:
...or Stunt Rally:
...or Trigger Rally:
(i'm not sure - but Stunt Rally and Trigger Rally may be from the same Trigger codebase)
Trigger Rally also has a webgl version:
All of this is just to say that if it were wanted, a completely open-source driving AI simulator could be easily made.
E.g. Maybe it learns that pedestrians made braking instruments in a tight turn.