
Upping the Ante: Top Poker Pros Face Off vs. Artificial Intelligence - aaronyy
http://www.cmu.edu/news/stories/archives/2017/january/poker-pros-vs-AI.html
======
arikrak
> In spite of the diversity of strategies one can design, it is important to
> remark that the heads-up limit Texas hold'em variation has been claimed to
> be "essentially weakly solved" in January 2015 by the Cepheus poker-playing
> bot.[1] This means that on average the program is so good that a human would
> have no chance of ever edging ahead of it, even if the two played 60 million
> hands.[2] The bot can be played online at poker.srv.ualberta.ca, and users
> can even query strategies from the software

[https://en.wikipedia.org/wiki/Heads_up_poker](https://en.wikipedia.org/wiki/Heads_up_poker)

~~~
Tenoke
This is for _No Limit_ which is a whole different class of problem.

------
Iuz
Doug Polk, a pro that was on the first challenge just talked about this on his
channel:
[https://www.youtube.com/watch?v=gz9FJfe2YGE](https://www.youtube.com/watch?v=gz9FJfe2YGE)

~~~
conistonwater
I watched that, and I think his understanding of what a "statistical tie" is
is extremely dodgy
([https://youtu.be/gz9FJfe2YGE?t=10m43s](https://youtu.be/gz9FJfe2YGE?t=10m43s)).
What's up with that?

 _Edit:_ he does make a good point later on that playing thousands of games is
exhausting, which would affect the reliability of the comparison between the
AI and the pros. I don't know anything about poker, but it sounds plausible.
Presumably this means they should make the AI so good that it can demonstrate
its greatness quicker than a human gets tired.

~~~
DavidSJ
Chalk it up to poker being something you can be skilled at without really
having much mathematical understanding, and throw in a little "common sense"
(i.e. anti-intellectualism), and you get comments like that.

~~~
eutectic
The charitable interpretation of his argument is that the outcome is only a
statistical 'tie' if you accept the arbitrary significance threshold of p=5%.
Even relatively weak evidence is still evidence.

Unfortunately, some of the examples which he claims don't pass this
significance threshold actually do. This does not help his credibility.

~~~
conistonwater
FWIW, the win rate was -91mbb/g, and so they reported the probability of being
equal in skill somewhere between 5% and 10%, hence the "tie". The introduction
to the other poker paper on HN now
([https://arxiv.org/pdf/1701.01724v1.pdf](https://arxiv.org/pdf/1701.01724v1.pdf)),
by a different team, from Alberta, is rather less gracious and says it was a
"huge margin of victory". I think that whatever the interpretation, his video
makes it clear he didn't understand any of this.

~~~
hnkain
Significance works the other way round than you described it -- if two players
were equal in skill, then a winrate of -91mbb/g or larger would happen 5-10%
of the time.

I don't know if Doug Polk understands that or not, but I agree with his
criticism and I think his analogy with sports reporting is sound. While
"statistical tie" is true in a technical sense, it's not usually how matches
are reported, and it's somewhat disingenuous and self-serving to use that
language. It would be more honest to say that it's very unlikely that bot is
better than humans -- given the advantages the bot had (a gruelling 2-week
schedule, and pressure on the humans to get through N hands per day) it still
lost by a significant margin.

~~~
conistonwater
You're right, I would have liked them to report posterior probabilities, but
those were p-values instead.

> _It would be more honest to say that it 's very unlikely that bot is better
> than humans_

I don't think the analogy is sound. Consider the alternative: over a long
match, the humans _failed_ to beat the bot _convincingly_. But in any case,
the examples he gives are so wide of the mark that it's clear this isn't the
sort of argument he's making—he talks about "spin" and stuff.

------
petters
Personally, I think it would be much more interesting to see an AI compete in
a multi-player game with, say, ten human pros at the table. Not just heads-up.

~~~
jim-greer
There's so much variance in the game that it would be hard to know how good
the AI really is. If they're doing 120K hands heads-up, they'd need to play
far more in a full table. Getting that many pros to play that many hands would
be difficult.

And what's wrong with heads-up, anyway? It's not as popular as multiway, but
it's a fascinating game. In a full table game, you usually only end up playing
20-25% of your hands. Heads up you play 90%+. You end up in leveling wars with
your opponent more often, too ("he knows that I know that he knows I usually
wouldn't play a premium pair this way").

~~~
petters
Maybe you are right, but in Poker championships you see many of the same names
in the top from one year to the next, right? (I could be wrong about this)

That could suggest the variance is manageable.

I'd like to see more players because we have seen so many two-player AI
contests before (chess, go). It would for example be interesting to see MCTS
in this setting.

~~~
jim-greer
The variance is manageable if you play enough hands. This is one reason why a
lot of pros prefer to play online - by playing many tables at once, you get
more hands in and can better overcome the variance.

As for tournaments, the variance is even greater, since it is closer to all or
nothing. To mitigate this, pros "buy a piece" of each other. Or swap equity.
If either wins they pay the other a percentage. If you do this with enough
other pros, you even out your swings...

------
andrewclunn
Oh, so playing online with access to statistical tools and analysis, so really
the computer advantage for knowing the odds is completely nil. It's really
only then competing in the strategy element... very interesting indeed.

~~~
danenania
Calculating odds against hand ranges is too complex to be done by a human in
real time while playing, even using calculators made for the purpose.
Computers definitely have an advantage there. Assigning hand ranges in the
first place is the trickier part for an AI, as it is more rooted in
psychology.

~~~
jat850
This is a great way of putting it, your last part especially - assigning hand
ranges is one of the things many players would probably attribute to gut or
feel. Some implicit decision making ability that causes them to prune away
vast parts of the possible tree and then make quick decisions on the remaining
ones. Often based on knowing opponent history, observing patterns, etc.

~~~
danenania
Yep, being good at this is what separates great players from mediocre. If AI
can figure it out, humans are toast.

~~~
jat850
Agreed. I would say one difference in poker is that strategy can be (and is,
constantly) adapted over the life of a competition (and over the span of
several competitions). Tempo, adjustments, stack and blind
management/inflection points, even opponent avoidance in certain circumstances
- I expect some of those to be things that humans may hold an edge to some
degree. Those factor in a lot differently in other types of games (tournament
play especially). I don't understand AI well enough to know what would be the
hardest possible game for AI to conquer.

~~~
philosopheer
imagine a human player playing solo against a table full of optimally designed
AI's that compute perfect statistical and economic odds based on a huge
library of prior games including human games. What you are calling
"advantages" for the human are no advantage at all. So now add a second human
at that table, now the advantages you cite, they work only against that other
player.

in terms of intial hand evaluation based on "gut", this is similar to chess
position evaluation, a key part of master and computer play, a part that's
been hard to get right, a part that brute force depth helps to get right, a
part that is now completely solved for computers in play against humans. There
is nothing particularly challenging about poker that stands in the way here.

------
iopq
Any online poker pro will tell you that 120,000 hands is nothing. You can win
over the first 120,000 hands and lose over the next 120,000 hands or break
even. This is because the margins are low and the variance is very high.
Someone winning 2 big blinds per 100 hands (with 80 bb variance) will win
between 7901bb and -3141bb 95% of the time.

~~~
asher_
This x 1000.

Also, it appears they are playing a tournament, which increases the variance
significantly. I actually think tournament play is less suited to measuring AI
performance than a cash game, because you're basically playing a series of
different games as the tournament progresses that may not be generalizable
because the decision math changes based on blinds and stack size unlike a cash
game.

> To ensure that the outcome of the competition is not due to luck, the four
> pros will be paired to play duplicate matches — Player A in each pair will
> receive the same cards as the computer receives against Player B, and vice
> versa. One of the players in each of these pairs will play on the floor of
> the casino, while his counterpart will be isolated in a separate room.

This technique won't work for the same reason mentioned above. Hands in
tournament poker are necessarily not independent events.

------
scotchio
Does the AI ever bluff?

Edit: Yes.

> During last year’s contest, the pros noticed Claudico was making some all-
> too-obvious bluffs that they were able to exploit.

~~~
danenania
Certainly. Bluffing at an optimal/unexploitable frequency is something that is
quite difficult for humans to do, but is easier with access to a RNG.

Some pros use systems where they look at the second hand of a clock to decide
whether to bluff or not, as a proxy for randomness.

~~~
philosopheer
gotta remember to look at the clock when you are not bluffing too, otherwise
you are teaching your opponents how to "tell" time.

~~~
milcron
They look at the clock to decide _whether or not_ to bluff, so this is already
taken care of.

~~~
bronson
Except, if you don't look, the table knows you aren't considering bluffing.

It's still a tell.

~~~
morkalt
I use the second hand on my watch to help with mixed strategies; sometimes
using it to inform size of raise, or the direction of raise/fold, raise/call,
and (rarely) call/fold decisions.

As such, if you happen to notice me look at my watch, you can deduce I might
be raising, or calling, or folding, with a strong, weak, or medium hand that
is already made, or on the come; or that I am interested in the time for a
reason such as wondering how long i've taken, how long i've been at the table,
if i need to leave, if a new dealer might sit and collect time soon, or
something else.

I get that you're a pedant who wants to be right; but it's not a remotely
useful tell unless the person does it only in one type of situation. but
nobody who is concerned about how well they mix a mixed strategy is dumb
enough to mix only _one_ strategy.

edit: and i'm not a pro. i play in mid-stake games with $1-5k buy-ins.

~~~
bronson
Apparently you're also a pedant who wants to be right.

------
necessity
I'm guessing they're training the AI independently for each player, with that
in mind I wonder how good it would perform when matched against a different
player. Knowing your opponent's playing style is one of the main things you
have to master in poker. This is specially difficult as playing style will
vary as the game goes, and sometimes solely by the player's will (to trick
you) and not by some information derived from the game. I can see a bot
performing reasonably well on this task for a specific player despite all the
difficulties, but for the general case it seems like a huge challenge (as it
is for humans).

~~~
conistonwater
The article says they're computing (approximating) the mixed Nash equilibrium,
which means it doesn't depend on the opponent's style.

~~~
xapata
If it's just looking for equilibrium, it won't make any money. It also does
live analysis of end-game situations, so I'm guessing it'll make use of U of
Alberta's research into deviating from equilibrium to exploit leaks in the
opponent's play.

~~~
spectrum1234
It will if the human is not playing perfectly. Which of course they aren't
because they are human. The question is if the computer is playing close
enough to equilibrium/optimal compared to the humans.

~~~
xapata
> equilibrium/optimal

That assumes that the Nash equilibrium is optimal. It isn't. Optimal play is
to deviate from Nash equilibrium to take the most exploitative strategy
against specific opponent weaknesses. This necessarily creates weakness in
your own play. Optimal play sends misleading signals to the opponent.
Basically, you signal "rock" so that the opponent plays "paper" but you
actually play "scissors".

(In this metaphor, neither "rock" nor "paper" nor "scissors" are equilibrium
strategies.)

For example, I might order a couple beers, straddle a few times and start
egging on the rest of the players for a round of straddling. To get it going I
might throw in a few blind bets. Ask folks to go all-in blind once or twice,
for fun. Do that a few times and no one expects you to have a good hand. Well,
except for the folks that have seen that story play out before. Except for the
beers and bending the rules, the computer can do the same thing.

------
rampage101
It's really unlikely the computer will win this. If it does it will be a big
shock to the poker community.

The pure Nash Equilibrum strategies do not work that well against thinking
opponents.

~~~
splonk
Nobody who's been paying attention in the poker community would be shocked by
this. HULHE has already been a AI win for a few years. HUNLHE is basically
just a matter of time.

Nash equilibrium strategies specifically do work better against thinking
opponents - the usual complaint about them is that they don't maximally
exploit "bad" opponents.

~~~
tekromancr
For anyone else who was lost: HULHE - Heads-up Limit Hold'em HUNLHE - Heads-Up
No-Limit Hold'em

------
spectrum1234
Does anyone else find it fascinating that AI can beat Go and Chess but not
poker (no limit)?

I played poker professionally for 7 years (until banned in US) and am still
amazed by this.

~~~
wapz
I feel a big difference is Google was behind the Go AI.

------
mmanfrin
I feel that Poker is one of those games that an AI could develop a winning
strategy for, but not the most winningest. If you deployed this AI on
pokerstars, it would win you money. If you deployed it at the WSOP, it would
not win a bracelet.

~~~
jknoepfler
nit: as a former professional poker player, the skill level required to be a
long-term winning player at a high-stakes cash game is considerably higher
than the skill requirement to win a single high-stakes tournament. (I would
not voluntarily sit at a high-stakes cash table without knowing there were a
mark present, I would voluntarily play most WSOP tournaments)

~~~
teej
Well of course. If you don't know who the mark is then it's likely the mark is
you.

~~~
xapata
I think "fish" is more typical. Though I suppose in Rounders it was "sucker".
Still the best poker movie, despite depicting the actual poker play badly.
[https://en.wikiquote.org/wiki/Rounders_(film)](https://en.wikiquote.org/wiki/Rounders_\(film\))

------
return0
It's unfair and rather cruel to pit humans vs computers. How about we use the
AI to design a poker variant that is unbeatable by another AI ?

~~~
jat850
Unfair and cruel to whom?

~~~
new_hackers
Why the computers of course

------
IamHWengineer
i wonder why Durr (tom dwan) and Isildurr (viktor blom) are not in the mix.

~~~
curun1r
Probably because playing 30,000 hands each for a chance to win a share of
$200k is spectacularly not worth it for those guys.

~~~
osti
And Durrr is known for not finishing his challenges...

------
lintiness
"Jason Les, Dong Kim, Daniel McAulay and Jimmy Chou ..." are the pros? not
exactly ivey.

~~~
jat850
They're heads-up pros. Ivey and many other well-known names are not heads up
specialists like these competitors. They'd likely have an edge over almost any
"recognizable" name player.

~~~
lintiness
i'm informed and know next to nothing about two of the names. there are plenty
of guys out there the poker world recognizes as better players.

~~~
sekasi
you're missing part of the point.

The $200k pool of money for an actual 'celebrated' poker player is absolutely
not worth the time and effort. Hence why you get a few tiers down in terms of
player skill. Still valid tests though, you can't only compare it to the top
1%

~~~
splonk
It's not a few tiers down in skill, it's a few tiers down in fame. Each of the
four players named is likely a favorite over Ivey in this format. Being a
specialist counts for a lot here.

~~~
iopq
Ivey played HU vs. Tom Dwan, Galfond, Blom, etc. and did fine. There's no way
they're favorites against Ivey.

~~~
splonk
Galfond thinks otherwise. From the article:

“Your favorite poker player almost surely wouldn't agree to play any of these
guys for high stakes, and would lose a lot of money if they did,” Galfond
added. “Each of the four would beat me decisively.”

~~~
iopq
Interesting, because for the durrrr challenge Galfond was named as the only
person Tom Dwan won't play. Maybe I've been out of the loop too long.

~~~
ramblerman
Yeah these would not be considered the world's best HU NL players anymore.

Names like Ben Sulsky, Doug Polk, Isaac Haxton would be big favourites over
your lineup.

Galfond while still playing is playing mostly Omaha

------
CoryG89
Seems unfair. Hard to read the face of an AI to tell if it's bluffing or not.

~~~
KODeKarnage
The 90's called. They want their poker strategy back.

Poker is not about reading your opponents "tells". That is a myth perpetuated
by Hollywood and ... professional poker players.

While there definitely could be some value in using psychological manipulation
to best your "live" opponent, gaining some advantage via a physical tell is a
crap shoot at best except against the absolute worst players or the biggest
mark.

~~~
xapata
I did play with a guy for several sessions who would consistently shake his
leg whenever he had a good hand. I felt so bad for him (after busting him a
few times) that I told him about it so he'd fix the shaking.

