

 AI beats human poker champions - nickb
http://www.eetimes.com/showArticle.jhtml;jsessionid=BDYCRFWYJLEDOQSNDLRSKH0CJUNN2JVN?articleID=208802992

======
mattmaroon
They at least used a couple very good heads up limit players this time, rather
than Phil Laak and some other guy nobody has heard of.

~~~
ivankirigin
Would you agree with me that they are playing a different game?

If you remove the human element, the players can't use one of their skills.

~~~
jm4
Exactly. When playing against a machine, a human can't use one of his skills
but it doesn't apply anymore. There's no need to read an opponent or employ
deception because a machine doesn't understand this.

Any serious poker player knows that the best strategy for winning is to make
your opponent make mistakes while minimizing your own. One of the primary
tools for accomplishing this is the use of deception.

Once the game is reduced to calculating the odds at any given moment and
betting when they're in your favor, even a simple machine should be able to
beat a human as long as it makes reliable calculations.

It should really come as no surprise that a machine can beat a human at a task
that machines are really good at.

~~~
gizmo
Oh c'mon. Good poker players know all the odds of all hands with an error of
at most 2%. Especially in heads-up.

If it's just about number crunching then the first computer AI would be in
Omaha Hi-Lo or 7 card stud - because these poker variants have much more
complex odds.

~~~
jm4
Hmm... Maybe poker AI is about more than number crunching. Anyway...
Interesting tidbit: craps is structured so that the pass line bettor will lose
50.7% of the time. That 0.7% provides casinos with billions in profits every
year. I would think that any decent poker player would have to be able to
calculate odds with better than a 2% error rate.

~~~
mattmaroon
Poker is not craps, and it's certainly not a game where people sit around a
table and whoever calculates the odds best gets the money. There's absolutely
no parallel there whatsoever.

Have you ever even played?

------
aggieben
I would expect this to be true of any probability-based game, particularly
where:

1) the distinct human advantage of unquantifiable sensory perception is taken
away (i.e., a person can't "read" a computer's body language to make his
probabilistic estimations more accurate), thus reducing the game to a number-
crunching contest. We haven't been winning those contests since the 40s.

2) enough time is injected into the experiment.

~~~
gizmo
You have clearly never played poker.

Suppose the PC has 2 aces. It knows it has the best hand (worst cast: tie
against the last two aces), so it wants to put all the money in. However, the
human will just fold because the computer is so predictable. Result: the PC
essentially wastes his big hands because of the predictable behavior.

So the computer has to start bluffing, and semibluffing, just like humans.
However, with every bluff you make you're wasting money (as bluffs are by
definition sub-optimal play). So you have to estimate how much you gain in the
future by making that bluff. But if the PC uses simple heuristics then the
human can once again get the upper hand. E.g. the human can make outrageous
bluffs, knowing that the PC knows the bluffs are unprofitable and therefore
unlikely. That way the PC will lose once again.

Poker is not trivial - and people without any number crunching skills can win
from the best number crunchers if they're predictable.

~~~
aggieben
_You have clearly never played poker._

What is it with people that think this kind of quip buys some sort of
credibility? This is the second time today that someone has started a response
with "Clearly, you have never...". I have played poker. I have also baked
(reference to other thread).

Anyway, I don't buy what you're selling. It doesn't seem that it would be hard
to make the computer be unpredictable, unless suddenly humans become good at
figuring out the machinations of a pseudo-random number generator
instinctively, and in real-time. Make the computer do irrational things at
random intervals. The whole point is that a human player, not knowing what
cards the computer has, will have little introspection into the computer's
intended course of action. I suppose you could tell some things, depending on
the rules of the game (i.e., if the computer traded 3 cards in a 5-card hand,
you might assume he had a bad hand).

Essentially, it seems that you just said "it's easy to beat the computer if it
plays stupid". That doesn't have anything to do with this. The computer can be
both a perfect number cruncher and entirely unpredictable, and given enough
dealt hands, I don't see how you could beat it.

~~~
gizmo
The remark wasn't meant to establish my credibility, I was just saying you
make elementary mistakes. My hunch is you have no experience playing poker.
This doesn't necessarily make you wrong, but it doesn't exactly work in your
advantage.

Are you really saying that the computer doesn't need to use any psychology -
and can become "world champion" just by playing a mathematically perfect game
with some random bluffs thrown in?

You've got to be kidding me.

(Just for the record - I'm talking NL hold'em here. If you're talking about 5
card draw you have a point. As that game doesn't have the same opportunity for
psychology)

~~~
aggieben
Well, if it's any comfort, your hunch is correct. I'm not very experienced.

My observation, perhaps incorrect, is that poker outcomes are almost entirely
based on psychological manipulation, not on probabilities - because people can
be psychologically manipulated. In this regard, poker is like Risk. There's an
element of probability, but it only rarely tends to dominate outcomes (I _am_
an experienced Risk player). It's possible to write Risk-playing software that
is very difficult to beat except in the rarest of circumstances where luck
dominates, because you can't convince the computer to invade Asia or fortify
in Australia when it would really be against his interests. I see poker in the
same way.

Perhaps I'm over-trivializing the difficulty of writing such a system for
poker, but I suppose in essence, I am indeed arguing that you could create
software that could just ignore the psychological aspects of the game, and
instead play a total numbers game, and that over time it should be a winner. I
guess I can concede that if you don't ignore the psychological aspects and try
to use them in formulating strategy, that would be very difficult indeed; I
just don't see the point.

~~~
gizmo
Thanks for explaining your argument.

Let's suppose you're right - that playing by the numbers is sufficient. In
this case you would not have to keep track of player histories, as you
consider every hand without context. But this can only result in one thing:
your play style will be constant. You will always take the best action with
probability P and (semi)bluff with probability 1 - P. The human notices this
and starts playing very aggressively. As a result the PC will fold winning
hands and lose many blinds. What the PC should do is change tactics. Get
aggressive or start laying traps (e.g. slowplay a great hand so it looks weak,
to lure the aggressive human player into bluffing a lot of money).

If the computer always plays optimally from hand to hand it cannot deduce
changes in playing style and will therefore always lose. From this I can only
conclude that psychological aspects can never be ignored in poker.

~~~
aggieben
I never really meant that the calculations should be without context. One
could keep a "running distribution" to update P (probably a bunch of Ps,
actually) as you play.

Maybe we're now just looking at the same thing from different angles. You say
"use psychology" and I say "use numbers", but in the end, maybe it's the same
thing and it's harder than I'm giving it credit for.

Thanks for the poker lesson, by the way. Next time I see a game I'll be more
interested.

~~~
gizmo
Ah-hah. But that's the problem. So you need some kind of distribution for each
of the different tactics the player plays. And then you need some heuristics
to predict which tactic is going to follow which tactic. It's a slippery
slope, I guess.

I don't know where to draw the line myself. It can be argued that as soon as
something is modeled by a computer it ceases to be psychology and becomes
mundane math. But I do still believe that most tactics people use to trick
others with need to show up in the code in some way.

I won't use the "Clearly you've never done x" line again. It was cheap and
uncalled for. Thanks for calling me out on that.

------
llimllib
more at <http://www.overcomingbias.com/2008/07/poker-vs-chess.html> and
<http://www.cs.ualberta.ca/~games/poker/man-machine/>.

~~~
mattmaroon
"far more people play poker than chess"

Is that true? Certainly in the US, but worldwide?

~~~
coglethorpe
Well, I've heard that many online players are from Sweden, and I think tour
players as well. Other countries have poker rooms and allow online gambling
openly.

------
schtog
Most people here need to look into what OPTIMAL in a gametheoretic sense
means.

It doesn't mean optimal in the way people normally use it.

A player playing gametheoretically optimal will beat ALL other players. That
is however not the same thing as playing the most exploitative poker against
every single player.

So the perfect bot would beat all the other players in the world, including
the best human but if the bot plays the worst pokerplayer in the world and so
does the best human player then the best human player might beat the worst
human player for more money than the bot would beat the worst human player.

~~~
mattmaroon
Won't a player playing gametheoretically optimal tie with all other players?
That's what my math-nerd poker friends all tell me.

~~~
schtog
In for example roshambo that is the case but not in poker.

Poker has dominated strategical choices so someone playing gametheorhetically
optimal will beat someone that doesn't.

Of course if his opponent is also playing GT-optimally they will tie.

Then of course there is rake to consider so if the opponent plays close enough
to GT-optimal you might both still lose if the rake is high.

------
steveplace
I'm really surprised no one has yet to mention this:

codingthewheel.com is currently going through an explanation on how to build a
winning poker bot online. Here's the latest post.

[http://www.codingthewheel.com/archives/how-i-built-a-
working...](http://www.codingthewheel.com/archives/how-i-built-a-working-
online-poker-bot-4)

------
wallflower
> BioTools Inc. (Edmonton, Alberta) has built previous versions of Polaris
> into a downloadable poker coach called the Poker Academy.

------
lpgauth
Wonder if this was no-limit hold em? If it is it's impressive. If not well the
probabilities are not that challenging.

~~~
MaysonL
Duplicate limit hold-em (same hands played at other table, with human and
machine reversed)

