While I was watching some of these replays, I was thinking to myself of all the things a bot needs to keep track of and all the things it could potentially try, and it just seemed so overwhelming. There are just too many things you can try, and your resources are limited.
StarCraft is a very, very deep game. It seems like it would be very easy to waste time on paths that lead to no real gains. And that rather than using interesting AI techniques, the most important thing might in fact be the StarCraft knowledge of the author. And how good they are at distilling the game down to the parts that is going to yield the biggest returns.
Also, StarCraft has very distinct early, mid, and late stages to the game. Focusing only on the early game, and going with a single strategy eliminates a ton of complexity. If you spend all your time on getting an all-or-nothing rush perfected, you've essentially "skipped" all the sophisticated techniques some of the other authors spent their time creating for the mid and late stages. In fact, it seems like some authors did just this and were quite successful.
But even an all-or-nothing rush has a surprising amount of depth, and perfecting that would be a quite a feat. The biggest thing being micromanagement/unit control and knowing what to target first (this is actually very hard). If you don't perfect that, beating a pro doesn't seem like it would be even remotely possible.
I was a huge StarCraft BW fan, and it's the only game where I got to the point where I could sometimes hold my own against pro players. I'm tempted to try BWAPI. Creating a full bot seems like a huge amount of work that I probably don't have time for, but creating a unit control AI sounds like it would be fun. I remember there used to be micromanagement custom game maps that people used to create (basically a series of situational challenges where you have a bunch of units and you could face other human players 1v1). Doing a competition for that as well would be pretty cool I think.
I see you even hosted a micro tournament recently. I didn't know that was a thing. How did it go? So there's at least some interest for that it seems? If so, I think I'd like to give it a shot too in the future (it'll probably be a while before I can get to this though).
My micro bot is basically generating heuristics in the form of if/then statements and testing them in against itself playing the other side. It already has found hit&run on 1 level on its own, but it still isn't good enough vs human players.
> My bot has a variety of different strategies that it can execute. The strategy to execute is chosen randomly using weights, and the weights are adjusted based on previous match results. It's pretty simple but I guess it could be called learning.
The top 3 bots are independent entries, and beat the bots using learning algorithms such as UCB1 from research institutions. The author of the bot ranked second does think that other bots that minimized hand-crafted solutions would be more interesting: https://dl.dropboxusercontent.com/u/23817376/Starcraft/AIIDE...
The winner is very interesting, while the runner up uses the simplest possible strategy every single game. The runner up had a favourable record against the winner too, but lower win percentage overall. Thank goodness the more interesting one could win.
Basically, 4 pool beats all but one of the bots in the game over 50% of the time (often above 80-90%).
Developer set parametres are pretty common in industry "machine learning", the kind of thing Google NEtflix and Amazon use. Academic models aim to reduce the amount of human input, but most real world applications have no such motivation. This kind of AI can be seen as a developer parenting a machine.
I'd noticed too that the winner claimed to use just a variety of hand-rolled strategies, and wonder why that is. Has anyone had the time to dig into the code? Is this a case of well-writen "naive" code trumping theoretically-better but poorly-implemented algorithms, or are there properties of the game state-space which make e.g. Monte Carlo tree search perform poorly?
Speculation: the winning author is simply good at StarCraft and knew what to implement. His implementation was probably simple but did one thing very well, which beat more complicated/smart AI implementations from worse players.
Monte Carlo over the StarCraft state-space would be very hard to implement but I don't see any particular reason why it wouldn't work decently well.
This is great, thanks! The fact that it's possible to get started in Clojure, I no longer have any excuses to not try this.
Having an interactive REPL seems like it would be helpful for iterating on micro routines (this is what I want to focus on). I see you already have some situational micro examples too.
I feel like a kid again. Can't wait to try this! Now I'm wishing I still had my StarCraft discs with me.
Yeah, I saw that. :) This is impressive. I wish I knew about this project sooner. I didn't see this project on the Clojure subreddit, so I shared it there in case others haven't seen it before either.
I see the issue you created about separating the Clojure API from the AI code. I think I'd have to do that because I'll be creating an AI from scratch.
I noticed in the thread you linked to that you mentioned Java packaging and possibilities for contributions. I might be able to help there. Let me know, or if you want to create some GitHub issues, I can take a look later and see which ones I can take.
Would be interesting to get your hands on a large set of replays from a lot of gosu players and run some machine learning offline prior to each map for the current match map.
Would be fun to experiment with the data and see if it would be possible to infer game tactics from professional players and "simulate" those against another bot based on the current game state.
The author of the winning bot (tscmoo) is currently experimenting with training NNs on data acquired by parsing replays of pro-level games. He's been mostly focusing on automatic learning of build orders (BOs) in specific match ups.
> Would be interesting to get your hands on a large set of replays from a lot of gosu players and run some machine learning offline prior to each map for the current match map.
There are probably too many variables in Starcraft for this to be practical. With a game like chess there's a fairly obvious set of state variables to keep track of (max of 32 pieces that each have a position in essentially an 8x8 2D array), with Starcraft it's pretty nuts though, each side can have dozens of buildings, dozens of units of ~15 types, each of which has a position on a much larger grid, health, sometimes energy, status effects, etc. Obviously there are RTS games that are more complicated yet, but the jump from a turn-based board game to an RTS is just huge.
That's why it would be interesting to see how much can be inferred from offline learning first.
For example, looking at unit positions over time for the map can be quite rewarding, especially between the different phases of the game (early, mid, late).
If you still haven't had a look at my other post, here is the link to the source code repository:
https://github.com/bwapi/bwapi
We also gladly accept PRs regarding bug-fixes or other improvements.
Here is the doxygen generated documentation of the project structure:
https://bwapi.github.io/annotated.html
In the IRC channel, you could ask about more details BWAPI's author and main contributor - heinermann. He is usually more than happy to shed some light on such questions
If anyone remembers Asimov's 'I, Robot' series, it ended with Machines (was that the exact name?) taking on the centralised planning of the whole globe, quietly removing recalcitrant humans from the supply chain, and apparently benefiting all humanity.
That problem is the AI problem these game designers are trying to tackle.
The obvious solution here is pattern matching. Whether traditional machine learning or NN's, certain situations need dedicated pattern matching looking for them constantly with some probability and algorithm to decide how to act on that. Other areas for improvement, too. Here's some suggestions:
1. Human players use scouts to counter fog of war followed by looking at build patterns to determine likely strategy. Spotting strategy should be easy for pattern matching. They also often micro the scout to (a) disrupt mining or (b) keep it alive for as long as possible. This should be similarly easy.
Note: Units damaged from a prior battle can also be used as scouts and simply not factored into further combat.
2. Simulations should be done using most likely combos of units for each strategy against most likely combos of opponents' units. This should be done for macro and micro. Patterns here will be used in the micro-engine. Bonus against machines: use unlikely combinations that prove effective in simulations.
3. Combat analysis should be a global thing that factors into overall goals. That the human player moved units from one base to another should never surprise an AI. The AI should've shifted some or all defence to that base immediately with use of fast-moving scouts (eg zerglings) to track opponents if they run.
4. Building laying and build order heuristics should be datamined from top players while attached to specific strategies. No one size fits all. Have two for each: one ideal, optimized build order; a heuristic, on-the-fly set if an attack or change throws off the plan.
5. Identify the common strategy changes that happen as a result of what enemy does. Simulate likely situations with units or build orders for those. Identify heuristics for safe transition. Will help for adaptation.
6. Put as many human strategies in there as possible from the A-listers, both specifically and generically.
Those are where I'd start on improving the situation, esp against human players. However, I think an even more interesting line of research is augmenting human players with an AI to efficiently execute their strategy. One that constantly assesses, micro's, follows build orders, etc. Basically never misses a step. More practical for real-world applications.
Really interesting, I started playing on SC2 so I'm not sure about the dynamics of BW but I wonder how the lack of micro skews the results. For example a fast reaper push hopping on/off the enemy base can't probably be done by a bot. Terrans are also much easier to automate, while Zergs require microing with injections, etc are the demos uploaded anywhere?
This couldn't be more wrong. The amount of glitches in BW that allow got micro potential is huge. It's actually one of the biggest complaints from BW players about SC2, the lack of available micro. Check out th video "Depth of Micro" by Lalush for more info on things like muta stacking, patrol micro, carrier leash range, and tons of other hidden micro.
You are also wrong about your last statement, the limiting factor for zerg that's you're talking about it APM which is pretty much unlimited for a bot. Injects can be automated without taking any attention from anything else.
This would be a really neat application for making one. With the order of complexity problem from an earlier comment, it'd be neat to see how the AI's behavior changes as the number of states exponentially increases.
Surprisingly high number of "Random Race" bots. I'd have thought that gives a fairly limited advantage against an AI while being a significant disadvantage in the programming side (i.e. AI needs to be good at 3 things).
Knowing what race you are facing allows you to pick a build which is defensible against the greatest number of early strategies common to that race (you know if you might get hit with six pool, 4 gate, or bunker rush.)
It's relatively easy to program a version of all these basic builds for each race into the AI. It's practically impossible to configure an AI so that it can scout all possible common early builds for 3 different races and respond properly to each one. Even a human has no single build that works against everything, and will go into really really strange improvised responses when they get hit by something unexpected.
I get the basic logic of using random. I just doubt it translates well to AI.
As you say "practically impossible to configure an AI so that it can scout all possible common early builds" so an AI is essentially flying blind anyway. Hence no advantage in picking random just more things to worry about.
StarCraft is a very, very deep game. It seems like it would be very easy to waste time on paths that lead to no real gains. And that rather than using interesting AI techniques, the most important thing might in fact be the StarCraft knowledge of the author. And how good they are at distilling the game down to the parts that is going to yield the biggest returns.
Also, StarCraft has very distinct early, mid, and late stages to the game. Focusing only on the early game, and going with a single strategy eliminates a ton of complexity. If you spend all your time on getting an all-or-nothing rush perfected, you've essentially "skipped" all the sophisticated techniques some of the other authors spent their time creating for the mid and late stages. In fact, it seems like some authors did just this and were quite successful.
But even an all-or-nothing rush has a surprising amount of depth, and perfecting that would be a quite a feat. The biggest thing being micromanagement/unit control and knowing what to target first (this is actually very hard). If you don't perfect that, beating a pro doesn't seem like it would be even remotely possible.
I was a huge StarCraft BW fan, and it's the only game where I got to the point where I could sometimes hold my own against pro players. I'm tempted to try BWAPI. Creating a full bot seems like a huge amount of work that I probably don't have time for, but creating a unit control AI sounds like it would be fun. I remember there used to be micromanagement custom game maps that people used to create (basically a series of situational challenges where you have a bunch of units and you could face other human players 1v1). Doing a competition for that as well would be pretty cool I think.