
How I built a poker-bot - run4yourlives
http://www.codingthewheel.com/archives/how-i-built-a-working-poker-bot
======
ssharp
His rules engine doesn't appear to trend other players at all. There are good
tools out there that gather data on players and with enough data, you can
pinpoint a player fairly well. It also shows his bot playing no-limit tables
which seems like a horrible idea for a bot. There is a lot more to take into
consideration in no-limit poker and I'm not sure a bot that only takes 3 or 4
things into consideration for every move could ever win at worthwhile stakes.

If your bot can play 14 limit tables at a time with even the slightest % edge,
you're going to make money. However, if you added more advanced player
profiling algorithms into your bot, you're edge would likely increase and
allow you to set it up at higher stakes.

~~~
mattmaroon
I don't think I agree with you on no-limit vs. limit. There's a pretty simple
strategy you can use to be a marginal winner at low stakes no-limit (or
especially no-limit sit and goes). It' called short stacking, and essentially
involves buying in for the minimum and playing mainly all-in or fold (with
some preflop calling).

You have to leave the table as soon as you get past 1.5x the minimum buy-in
though. It was mainly done on Party Poker, which was the only site dumb enough
to let you cash out and then return to the table for the min again. Any other
site either made you bring back what you left with or wait an hour.

On a site with enough tables in action, though, even that would not be a
limitation. A bot could do fairly well at it.

Also I can tell you from first hand experience that breaking even at low
stakes NL sit and goes can be done with an AI consisting of nothing but
if/then statments. I did this with WinHoldem back in the days before it was
banned from everywhere. I wasn't able to replicate that success with limit.
The main problem with WinHoldem was that it didn't join or leave tables on its
own (also, it would often bug out when the tournament got heads up), which
meant that the time spent managing it made it less profitable than just
playing myself.

That's why we tried to build a bot ourselves. Maybe I'll blog about that some
time.

~~~
sanswork
The reason limit is easier is just that there is less variables to account for
much like it's easier to write a bot to play poker with fewer cards.

While I will agree sit and gos are all strategy that strategy is dependent on
putting your opponent on reasonable ranges which a bot like this won't do.

Bots and winning bots are possible(stick most casual people up against the
UofA bots for example), but I find it hard to believe this one is one of them.

~~~
mattmaroon
That's wrong too. If you think there are fewer variables in limit hold'em to
figure when making your decision, you don't understand the game.

If you went to a no-limit tournament and just pushed all of your chips into
the middle at every single opportunity, it's been shown that you're only a
slight loser. At the WSOP main event, you'd be a better player than at least
1/3 of the field.

This is not true if you were to play in a limit tournament and raise every
time. You'd probably be the worst player in the field.

The fact that it's less complicated is why no-limit is the tournament game of
choice, but high stakes live poker is generally limit.

~~~
sanswork
There are fewer variables when factoring in your opponent modeling. I'm sure
being who you are you are very familiar with the UofA research and I'm pretty
sure it backs up this statement.

Pushing every hand isn't right, it's pushing every hand with certain cards
that Sklansky taught which admittedly is only a slight loser but a loser none
the less.

I'm not saying limit holdem isn't a complex deep game, all I'm saying is that
when it comes to bots no-limit bots are a lot harder to get right(in opponent
modelling primarily which is a big part of bots these days).

I think no-limit is the tournament game of choice because it's seen by the
general public as being the more exciting of the two(not that I agree but
thats another discussion).

~~~
mattmaroon
I still disagree and think that limit is, in general, a much more complex
game. But even if it's not, I think you're thinking of building a bot that's a
world class player. That may be harder at no-limit, I'm not really sure.

But if you're building from the standpoint of breaking even at low stakes,
it's far easier to do for no-limit, especially sit and goes. It's literally a
couple hundred lines of if/then statements.

------
ctkrohn
It's interesting, but to be honest, if you're smart enough to make a
successful poker bot, you're probably smart enough to write a program to trade
stocks or other liquid instruments like government bonds or commodity futures.
The potential winnings are much higher, it's perfectly legal, and due to the
depth of the markets such programs are capable of making much more money than
poker bots. It's also doable, as funds like Renaissance Technologies
(<http://en.wikipedia.org/wiki/Renaissance_Technologies>) and D. E. Shaw have
demonstrated.

~~~
hugh
_if you're smart enough to make a successful poker bot, you're probably smart
enough to write a program to trade stocks or other liquid instruments like
government bonds or commodity futures_

I'm not so sure about that. I think the main reason is: there's a lot more
"stupid money" at the poker table than there is in the market.

The bot as mentioned doesn't seem to do anything particularly fancy -- it just
looks at the cards on the table and figures out your odds of winning based on
that. This is information which is easily extracted mathematically, but which
most players don't have direct access to. Good players will have a pretty good
idea of these probabilities, whereas lousy players won't. If you can find
yourself on a table full of lousy players, you're at a huge advantage.

In the market, on the other hand, there's only one poker table, and all the
best players in the world are already sitting at it. Any time statistical
arbitrage opportunities arise in prices, you can bet that D.E. Shaw and
Rennaisance Technologies have already noticed it and moved a few billion
dollars that way. If you want to win, you don't just have to be smarter than
six guys in their basements doing calculations in their heads, you have to be
smarter than thousands of overpaid quants with supercomputers.

~~~
ctkrohn
Fair enough. You're probably right. I don't play online poker (and am a poor
casual player) so I don't know what the level of skill is on those websites. I
was just making an assumption based on how difficult I imagined it would be
for a computer to play very good poker.

------
gruseom
A few years ago, I wrote a program that would screen-scrape to read all cards
showing and then do a Monte Carlo simulation to randomly complete the hand and
compute what % of times my cards would win (along with pot odds). It took a
second or so to do the simulation, so I could use it to make decisions in real
time. It was a fun project, but I soon stopped using it. Your odds against
random cards are not at all your actual odds, because most players fold weaker
cards. What matters most is who you're playing against.

Anyway, the article, although it describes well the techniques for
programmatically interacting with a poker client, is majorly misleading
because _this is not the hard part_ of making a bot. The hard part is figuring
out whether, and how much, to bet. He alludes to this as "the million dollar
question", but then omits to even mention it under "What skills will I need to
write a bot?" Perhaps he'll address this in later articles?

If you look at the last screen shot, it's clear that he built a UI for an end-
user to specify the actual logic for poker play. ("Call if it has not been
raised, I'm later than 7th", etc.) His "bot" doesn't actually make decisions.
It collects rules from an end-user, calls those to decide what to do, and
passes the result on to the poker client. In other words, the hard part of the
job - writing a program to play poker - has been passed to somebody else.
Maybe that somebody else is the author, creating the algorithm through his
UI... but then why not just code it?

There are serious researchers who have made credible bots (he links to them),
bots which would (and, for all we know, do) make money on poker sites. But
when I looked at it, the literature was not nearly detailed enough to code up
their algorithms. (Also, they were working with simplified versions of poker,
since full no-limit holdem is a hard nut to crack). What they _didn 't_ use
were the "extensible rules-based systems" which this author claims "are a lot
more powerful than you think". Count me skeptical. Good players eat such
systems for snacks. Maybe they make money in low-stakes games against bad
players, but I'd like to see evidence.

In short: never mind the dll injection, show us your bankroll over 100,000
hands!

~~~
mattmaroon
"Count me skeptical. Good players eat such systems for snacks."

You're thinking about it wrong. As he mentions, your bot doesn't play against
good players. It plays .5/1 or 1/2 nl and makes a shit ton of money if it just
breaks even. There aren't good players at those levels, because anyone who was
good would win and quickly move up. You only need all of your if/then
statements to add up to being better than the worst 5% of poker players, and
that's not that hard to do. I know only because I've done it.

Thats why extensible rules based systems are better than you think. They can
get you to that with a good amount of work. Especially in short stack nl
games. They're clearly not going to win at a $5/$10 nl table or a $215 SNG.
But they don't need to to rake in millions per year.

~~~
mattmaroon
Also I should mention, that with the exception of heads up, good players (at
least by my definition of the word good) eat even the UofA's AI for snacks.

------
Flemlord
I was involved in a fairly elaborate pokerbot effort. Believe me, the
mechanics of playing the game are the easy part. The hard part is writing the
poker logic. Check this out if you want an intro to the "basic" concepts:

Lokibot: <http://www.cs.ualberta.ca/~jonathan/Grad/papp/thesis.html>

The bot described in that paper lost about 8c a hand at the $1 tables. After
extensive modifications, we got it to the point where it was losing about 2c a
hand. This when Bush made poker illegal in the US, and we killed the project.

------
xirium
> none of the online poker sites have really stepped up to the plate and
> either a) made bots legal (similar to the way that they're legal on Internet
> Chess Club) or b) put effective prevention measures in place.

That's a tough choice for a casino. Allow automation and you create a ghetto
which discourages casual users. Disallow automation and you encourage an
escalating "arms race".

Regardless, the techniques discussed could be detected using anti-virus
techniques. I'm surprised that native clients don't perform integrity checks
or is this the state of play in the "arms race"?

It would be fairly simple to disable integrity checks using methods to
circumvent software DRM. Then it would be a matter of implementing distributed
state to determine and ignore circumvented clients.

~~~
ericb
Interestingly, the casino's financial interest, since it makes its money only
on the rake, is to have as many players or bots as possible. In fact, from
their perspective, an infinity of bots battling each other should be just fine
--bots scale better than humans. From Ultimate bet's perspective, bots are
great:

It doesn't feel pity, or remorse, or fear. And it absolutely will not stop,
ever, until _UltimateBet is rich_.

~~~
xirium
I considered this possibility but a casino which is too open to bots could
become marginalised. Real players would be scalped too easily and bot writers
would be playing a zero-sum game. Both factors would reduce casino turnover
and profit.

So, it is in a casino's interest to leave the whole issue unresolved and with
a level of tolerance which is in-step with competitors.

~~~
ericb
I agree to an extent, however, I would be most happy owning a bot
battleground, personally. It would dispense with all that time wasted on
"thinking" and reduce the speed to a processor/internet lag dependent game.
Bots are no more zero sum than people, they are just _faster_. I love the idea
of a money making machine that scales with processor cores/mhz.

------
nanexcool
Can't wait to read the rest of the series. Promises to have a bit of
everything: OCR, screen-scraping, rules engine, WinAPI hooks, DLL injection,
and that's just part one!

------
hollywoodcole
Really Awesome, now I will apply to WoW.

~~~
henning
Convincing others that your goldfarming bot is actually human seems like a
highly challenging variant of the Turing test.

~~~
Hexstream
Not sure it's so hard, first you introduce some randomness and inefficiencies
in the gathering process so that it's more credible that it's a fallible human
that's doing it.

And as a second crucial step, you hook groups of bots into a "monitoring
computer", at which an actual human sits and responds "No, I'm not a bot lol."
when the supposedly fatal question arrives from an admin. You can even
outsource this to the third world.

~~~
henning
You've never played WoW, have you? "Randomness" is a dead giveaway of a bot.
Other players will grief you, report you, and you will be banned quickly.

Just the computer vision aspect of it alone would be amazingly hard.

Efficient goldfarming (e.g., with a frost mage that can take on 15-30 monsters
at once) involves carefully coordinating a complicated series of actions where
timing is critical and reacting quickly and intelligently when things go wrong
is the difference between living and dying. It involves making complicated
judgments about relative positioning, reasoning about surroundings, cause and
effect, and other things that require human-level intelligence.

If you can make a bot that can seek out and kill monsters in a non-trivial
manner without dying all the time there's no reason you couldn't program a
robot to replace a wide variety of menial real-world jobs. The consequences
would be tremendous and using it to make $20/day off the Internet would be
retarded because you could license the technology for billions.

~~~
murrayh
Bots don't require computer vision or any kind of sensory input; they can
extract the native data they need straight out of the application's memory
space. Effectively gaming bots have "perfect information". So the bots
actually have more timely and accurate information than humans do, they can
react quicker than humans do, and they can react as intelligently as your
programming skill allows.

Check out <http://www.easyuo.com>. That application allows you to easily
automate Ultima Online. Whilst deciphering WoW's application memory would
probably be harder, an equivalent program is certainly possible (it probably
already partially exists in various forms), and intelligent bots are a simple
step after that.

Unfortunately, once the memory accessing has been abstracted away, programming
a competent WoW bot is not a difficult task. The WoW bot doesn't have to be as
awesome as the best human players, nor need it be ultra-efficient. What it
lacks in ability it makes up in working without fatigue indefinitely.

~~~
andreyf
_Bots don't require computer vision or any kind of sensory input; they can
extract the native data they need straight out of the application's memory
space._

I'm pretty sure WoW keeps track of all applications running on your computer.
I think the way people do it is by using a hub and packet sniffers on a
separate computer, although I imagine WoW traffic could easily be encrypted...
so _shrug_.

~~~
randallsquared
All this hypothetical talk is very nice, but there actually _is_ a wowbot that
Blizzard has been unable to effectively use countermeasures against, called
MMOGlider. They've decided to simply sue after repeated failure to solve it
technically.

------
goofygrin
awesome article.

I wonder mattmaroon's thoughts on this.

