
AI Beats Four Top Poker Players - jeremyleach
http://www.bbc.co.uk/news/technology-38812530
======
bcassedy
Anyone that's interested in reading a more detailed account of the experiment
can do so here:

[http://www.pokerlistings.com/libratus-poker-ai-smokes-
humans...](http://www.pokerlistings.com/libratus-poker-ai-smokes-humans-
for-1-76m-is-this-the-end-42839)

The above article spells out some of the details of the competition. The
winrate (14.72bb/100) that the AI achieved over the 120k hand sample is almost
certainly not due to luck. It is a huge winrate that most pros have to employ
strong game selection techniques to achieve (only play against bad players).

Here's a layman's explanation of how a poker AI can be trained:
[http://www.pokersnowie.com/about/technology-
training.html](http://www.pokersnowie.com/about/technology-training.html)

And some details about the weaknesses resulting from how they've abstracted
the game:
[http://www.pokersnowie.com/about/weaknesses.html](http://www.pokersnowie.com/about/weaknesses.html)

~~~
fierycatnet
That's pretty impressive, the win rate and amount of hands played. It's
certainly not a fluke. At the same time, these 4 pros, I've never heard of
them. This AI and organizers should focus on HU specialists. Perhaps even
invite players like Negreanu, Ivey, and more recent online pros who've made a
fortune. Just don't let Hellmuth to play this machine, it would be a disaster.

~~~
coltnz
These pros are considered to be top 20 HU online specialists, they would
destroy Negreanu and Ivey over the long run at this particular variant (which
they have played millions of hands at).

~~~
marklgr
Back when I played poker (up to ~2010), Negreanu was mostly a celebrity, as
well as a pretty good blog writer. He was no match for the best pro players,
but perhaps he has dramatically improved.

Ivey was pretty good at managing his image, too.

~~~
mod
Say that all you want, those guys play(ed? I'm out of the loop) in the big
game, and...whoever you're thinking of, basically--didn't.

~~~
adamlett
Conversely, not many of the celebrity players have had much success playing
online. Ivey did have some good years but even he has been struggling for the
past few years.

Almost all pros who have had success both online and live agree that live
games a ridiculously soft compared to online games. The reason more online
pros don't play live is because you have to live in Las Vegas or Macau to play
a the highest stakes, or you have to be invited. Also, the big live games are
usually mixed games that don't afford a huge edge to online pros who
specialise in a few games.

~~~
mod
I don't dispute any of that, I simply dispute whether or not Negreanu is a
"top pro," or whether he would be a match for "the top pros."

He is a top pro. Not in HU cash. He couldn't hang with these guys at HU cash,
at least not with his current skill in the discipline. But hey, poker is not
that narrow of a term. It includes all kinds of disciplines, live and online,
horse and stud and hold 'em, the list goes on.

Those same "top pros" who Negreanu wouldn't play HU Cash wouldn't sit in the
big game with him.

Arguably bankroll management is the most important skill of a top poker
professional, and that's maybe what big-game players are best at.

Also, a small caveat to all of your comment is that live games are soft
compared to online games _of the same limit._ There is no online equivalent of
the big game, or at least there wasn't when black friday hit and knocked me
out of the professional poker scene.

~~~
adamlett
There is no question that Daniel Negreanu is a very good tournament player,
possibly one of the best. I don't think there is any evidence that he is
anywhere near the top when it comes to cash games of any popular variety.

And while bankroll management is certainly an important and required skill to
endure as a professional poker player at any level, it is by no means what
differentiates high-stakes pros from pros at the lower levels. There are lots
of small stakes and mid stakes players who practice sensible bankroll
management, but who will never acquire the skill necessary to make it at the
highest levels. If you want to get a sense of how much skill goes into playing
poker at the highest level, you should watch some of Phil Galfond's strategy
videos on YouTube (see for instance [1]). Poker strategy has come a _long_ way
since Super System[2] and The Theory of Poker[3]. Even players who were
considered very good just five years ago, can no longer compete at the highest
levels.

[1]
[https://www.youtube.com/watch?v=khdoSFCQ9iA](https://www.youtube.com/watch?v=khdoSFCQ9iA)
[2]
[https://en.wikipedia.org/wiki/Super/System](https://en.wikipedia.org/wiki/Super/System)
[3] [http://www.twoplustwo.com/books/poker/theory-of-
poker/](http://www.twoplustwo.com/books/poker/theory-of-poker/)

------
doesnotexist
Andrew Ng posted a comment about this on facebook: "I'm thrilled about
Libratus' Poker triumph--this is a huge step for AI. Othello/Checkers/Chess/Go
were theoretically solvable with minimax tree search and sheer computation;
but poker, which requires bluffing, needs sophisticated modeling of your
opponents and new algorithmic principles. CMU's Tuomas Sandholm has also (in a
private email) promised to publish their algorithms, which I look forward to.
Congrats CMU!!!"
[https://www.facebook.com/andrew.ng.96/posts/1260889373966967](https://www.facebook.com/andrew.ng.96/posts/1260889373966967)

~~~
andrewprock
I'm surprised that Andrew Ng made this claim. The strategy that was built for
Libratus' predecessor did not do sophisticated modeling of the opponents, or
use new algorithmic principles.

Poker is solved using a very large game tree, just as with the other games.
The structure of the tree is modified to support the notion of hidden state,
but beyond that it is essentially the same as the other games. The structure
for representing hidden nodes was developed in the 1950s by Von Neumann. Most
of the algorithmic innovations related to how to update the game tree.

My guess is that the primary innovation for the Libratus strategy was that of
scale.

~~~
leeleelee
This was my thought too. Poker is quite "solvable" meaning, whenever you're
confronted with a decisions -- there is always a "correct" answer which does
not have to depend on the other players' behavior or style. And you can find
that answer by simulation, or game trees, and other methods.

It's also important to keep in mind that the best AI can still lose, and the
worst AI can still win (and everything in between). Poker involves randomness,
obviously whereas chess/go/etc does not.

~~~
mormegil
That depends on what you mean by "correct". Sure, you could theoretically find
Nash equilibrium of poker and by playing the equilibrium strategy, you can
ensure you won't lose. But that does not mean this is the best strategy to use
at a given table against the given opponents, who (being imperfect humans)
almost certainly do not play the equilibrium strategy themselves. And, by
playing a proper nonequilibrium strategy, suited to the specific players, you
can win more.

~~~
andrewprock
The usual way these games are solved is to create an "abstract" game which is
tractable, find the Nash equilibrium, and map state in the real game back to
the "abstract" game. In the limit, the solutions for a well designed
"abstract" game will converge to that of the real game.

~~~
mormegil
You are explaining how current algorithms try to find the (approximate) Nash
equilibrium (and those algorithms are far from perfect; as noted in the recent
DeepStack paper, current abstraction-based programs are beatable by over 3000
mbb/g, which is four times as large as simply folding each game). But my point
is that even the (exact) equilibrium strategy would not necessarily be the
best strategy against given non-equilibrium-playing players.

~~~
andrewprock
Yes, you are correct on every point. Opponent modeling and exploitation is
significantly more difficult than coming up with a Nash equilibrium to an
abstract game.

------
brilliantcode
it won't be long before we hearing more headlines like:

"AI beats top 10 hedge fund managers"

to

"AI run hedge fund blows up due to black swan event"

regardless it's an incredible feat. It really casts questions into what our
edge as humans are which is slowly disappearing and we didn't even need to put
a brain in a jar and hook it up to a computer....it's deep learning reinforced
algorithms that is appearing to outlearn, outthink the best of humans.

I just can't emphasize what a monumental period in history we are at. Humans
are producing specialized algorithms that learn and hold information about the
deep web of relationships between myriads of parameters to produce superior
performance than humans.

It's almost like we've uncovered ways to automate our intelligence very much
like we've been automating human and animal labor in the past couple
centuries.

So the question is, how does an average joe hacker like me exploit and
leverage this wonderful thing called deep learning? I'm not interested in
reading PHD papers with advanced calculus.

I want to have a map of what AI, ML, DL, NN methodologies to use and when and
who to hire based on that. This is no time to be a luddite and don't count on
basic income from appeasing the masses anytime soon. Much like people took the
most hit in the early rise of industrial revolution, our generation and
immediate generation will be hit the hardest.

~~~
feral
> "AI run hedge fund blows up due to black swan event"

Knight Capital, 2012:

[https://www.bloomberg.com/news/articles/2012-08-02/knight-
sh...](https://www.bloomberg.com/news/articles/2012-08-02/knight-shows-how-to-
lose-440-million-in-30-minutes)

~~~
reso
I believe the poster meant a machine-learning system blowing up (catastrophic
trading loss) due to a black swan event that the system wasn't trained to deal
with.

Knight's blow-up was a process/sysops failure, not due to their strategies
going wonky.

[As a side note I believe their core strategies were human-analyst designed,
not ML based, but I could be wrong.]

~~~
beambot
Wouldn't process & sysops failures count as blackswan events if the AI or ML
algorithms weren't trained to deal with those types of hiccups?

------
xapata
I suspect Libratus' overbet frequency is overfit to this particular reduced-
variance game format. In a normal game, the opponent doesn't take chips off
the table after winning a hand and might stand up at any moment.

It's hard to know how much that affected the strategy, but in the Reddit
thread, the human players said the overbet frequency was what they were most
surprised by.

~~~
6nf
Maybe overbetting is better EV and humans just don't know it yet. It will be
interesting to see if this changes the game at the higher stakes Heads Up
tables

~~~
xapata
Yes, but expected value isn't everything. A good investor considers the ratio
of expected return to variance of return. In a normal game, the variance on an
overbet is enormous and therefore the denominator of its Sharpe ratio [0] is
large and drives the value (or should I say "score" so as not to get confused
with "expected value"?) of that action down.

You'd happily go all-in pre-flop AA vs KK. On the other hand if you got 4-bet
pre-flop by a 22 and you're holding AK, you might ask, "Check it down?" This
is assuming the opponent really likes small pocket pairs and will call an all-
in, etc.

[0]
[https://en.wikipedia.org/wiki/Sharpe_ratio](https://en.wikipedia.org/wiki/Sharpe_ratio)

~~~
6nf
> you might ask, "Check it down?"

In a heads up game? Really? "Check it down?" ???

~~~
xapata
The example applies more to a ring game, where you might be going after a
particular fish and don't want to get involved with a better player. Still, I
think it illustrates the principle of trying to keep the variance down.

Also, if you're first to act on the next round, it might be worth asking, even
if you don't think they'll agree.

~~~
6nf
That's collusion and it's not cool. No top pro would ever seriously ask to
'check it down'

Kelly's criterion and the sharpe ratio is not relevant for the format played
in these human-AI heads up games. They play with an unlimited bankroll.

~~~
xapata
I'm not talking about a tournament. And yes, it could indicate collusion. But
most of the time, it's just a friendly low- or mid-stakes game and the players
are tired of thinking.

My point exactly about the unlimited bankroll. The experimenters may not have
realized that an unlimited bankroll would significantly affect the strategy.

------
spurgu
Reddit AMA from the live match:

[https://www.reddit.com/r/IAmA/comments/5qi3i9/we_are_profess...](https://www.reddit.com/r/IAmA/comments/5qi3i9/we_are_professional_poker_players_currently/)

------
sethbannon
"Each night after the play ended, the Pittsburgh Supercomputing Centre added
computations to sharpen the AI's strategy."

This sounds more like an "advanced chess" setup, where a human teams up with
an AI to play. The title of the article should really be "amateur poker
players + AI defeat professional poker players". The real test would be if the
AI self-corrected over the length of the tournament, without human
intervention.

~~~
bduerst
"He added that the professionals had been sharing notes and tips in an effort
to find weaknesses in the AI's game-play."

This doesn't sound like they had the goal of conducting a fully controlled
experiment here, but it's still interesting none the less.

~~~
thefalcon
Especially given that the professionals were live streaming and kibitzing with
chatters on Twitch during the event.

------
nojvek
Once this is out, it's going to be have a big impact on online poker games.

They way things are going with AI. You have a good algorithm, you can get rich
very quickly by being a one man business with hardware rented in AWS.

Libratus AI player is modelling it's human counterparts and predicting how
they think to outsmart them.

When Google started, they got the page rank algorithm and distributed
algorithms good enough to run on shitty unreliable cheap computers. They are
well on the way to become the world's largest company overtaking Apple
someday.

Their ad algorithms already know that I am applying for a house loan and are
blasting me with ads every fucking page I visit on the Internet.

I can totally see Google and Facebook personalizing ads per person and taking
advantage of the person's vulnerablaties. Like psychologically modelling them
to make them click ads and buy random shit. I can see the start of ultimate
God algorithms for marketing.

The ability for AI to create drug like experiences for us that we can't stop
craving.

~~~
bamboozled
Not if you have uBlock ([http://ublock.org/](http://ublock.org/)) installed.

------
gexla
Do a search for "long term" in online poker and you'll find that the
suggestion for players to determine their level of play based on that "long
term" is something like one million hands. That's running 4 tables for full
days of poker over a long period of time. For sure it's playing more than 4
players.

The probability space of poker is such that 4 competitors isn't going to tell
you much.

And defining "top" is difficult because "top" may be more celebrity than
anything. Everyone has their different objectives. If you are "top" then you
sure as ____didn 't get there by building a case history against AI poker
bots. Give these guys a chance to adjust, and give them a chance to figure out
why the effort to adjust might be worth bothering with.

~~~
halflings
From the article:

> A poker-playing AI has beaten four human players in a marathon match lasting
> 20 days.

20 days seems like plenty of time. No ?

~~~
mod
No, the number of hands is relevant here, not the number of days played.

~~~
6nf
They won 14.7 big blinds per 100 hands, playing 120k hands. That's a pretty
good win rate and a decent sample size but I don't know how to calculate the p
value

------
nippples
Can someone clarify why exactly is this being presented as more impressive
than beating Chess masters or even Go masters?

People talk a lot about number of states in poker, but the hand and visible
cards can be easily (to a statitician) reduced to a scalar "probability of
having the best hand".

At least in terms of crunching the possibility tree this should be a far less
computationally intensive challenge.

~~~
roenxi
What struck me looking at the AlphaGo papers was how human its algorithms were
- it was basically doing what a highly orthadox human player would do (pick
some moves that look interesting, read as deeply as possible) and then scaling
up with a completely inhuman thoroughness and accuracy.

At this stage, AI wins in competitive games simply will not impress me. From
here it may simply be a tour of force showing the breadth of fields AI can
dominate in.

After that enthusiastic agreement; poker wins are evidence for the laypeople
of something I suspect most people in AI research or game theory already know
- a human bluffing isn't a magic advantage in a fair game. The AI can
outperform on fundamentals.

------
RyanRies
Did I read the article incorrectly, or did it say that the bots creators were
feeding it additional data each night before the next day's tournament? If so
that defeats the entire purpose, doesn't it?

~~~
niketear
in what way would that defeat the purpose? the players can also review their
days play during the night

~~~
RyanRies
Yeah... I guess I was just expecting the AI to be more self-learning and self-
tuning...

~~~
09bjb
It built most of its decision model by playing itself over "trillions of
hands", according to Prof. Sandholm. You could almost think of the nightly
reviews as a similar process to a human sleeping: we benefit from memory
consolidation, pruning, time to reflect, etc. I don't see giving the computer
12 hours a night to do something close to reflection and self-analysis as
"cheating," or even very different from what its opponents did over Mexican
food each night.

~~~
kkoomi
The four humans were even "colluding" too.

------
hooloovoo_zoo
This article [http://spectrum.ieee.org/automaton/robotics/artificial-
intel...](http://spectrum.ieee.org/automaton/robotics/artificial-
intelligence/ai-learns-from-mistakes-to-defeat-human-poker-players) says
120000 hands were played over the course of more than two weeks. How much of a
factor was sheer boredom?

~~~
thefalcon
It was a factor for sure, especially as the AI would tank (stop and think)
quite a bit at predictable spots and so the "Pros" would occasionally use
strategies to try to knock out the AI before it would get to that point (they
were shooting for a targeted number of hands, so maximizing the hands per hour
is to the Pro's benefit). That said, they still had both money and pride on
the line, and at least one Pro insisted that he was genuinely doing everything
he could to try to win, despite the boredom and desire to be done with it.

------
limbicsystem
Is 6-handed a significantly harder game or is it just (from the computer's
point of view) just like playing 5 single games of heads-up?

~~~
snarf21
I would argue that it is significantly harder. In heads up, you can call a bet
and see the flop and go from there. In a full table, calling early means you
may have to fold to a raise. There is also a much more complicated analysis of
the possibility that you are beat. Lots of times I may be in a hand and think
I have the original raiser beat but when two other people call before me, the
odds are much less that I have all of them beat.

I still think the hardest part of poker is the grind. The computer doesn't get
bored or tired and play hands it doesn't have a reason too just because they
are stuck in a dead streak. Taking a mostly conservative style, you'd expect
the computer to out perform over time.

It would be curious to know what percentage of the time the players bluffed
the AI successfully and vice versa.

------
moosinho
Hmm, why is it surprising that AI is good at poker? The way I see the game is
that a bad poker player will just hold a model of his hand in mind. A slightly
better one will also hold a model of his opponent's hand. Even better one will
also model his opponent's model of himself... and so on recursively. And who's
really good at recursion? Computers.

~~~
niketear
People have been trying and failing to do so for decades, the state space of
the underlying incomplete information game is absolutely massive.

------
rapfaria
Put it against Negreanu.

"Oh you have AJ don't you?"

~~~
Fuzzwah
Negreanu wouldn't sit HU against the 4 players who played the bot.

~~~
xapata
Depending on the stakes, of course.

------
doug1001
The University of Alberta bot has been beating top pros at limit hold'em for
over 10 years.

no-limit and tournament play were deliberately placed outside the scope of
their poker-bot projects--at least during the period of time i was following
it which was approx. 2004 - 2010.

anyone know if the CMU team trained their rig on these variants?

------
jedberg
EDIT: Read below, I am wrong. I clearly didn't know what I was talking about.

This is a great achievement in AI, don't get me wrong, but the headline should
read, "AI beats the best four poker players we could find who were willing to
play for a mere $200K".

All the actual best players play for millions and have a reputation to uphold.
They would never agree to do this.

They four guys they got are pretty good, and could certainly destroy me, but
they aren't the best of the best.

I'd love to see the bot play in the World Series of Poker for a few million.

~~~
sanswork
>All the actual best players play for millions and have a reputation to
uphold.

All these players are high stakes players. I think you're underestimating the
fun factor. As for reputation. For a poker player having a reputation as being
beatable is a profitable thing to have.

>but they aren't the best of the best.

Who do you think is? Like how many people do you think rank above this group
at HUNL?

~~~
jedberg
Anybody here?
[http://pokerdb.thehendonmob.com/rankings/](http://pokerdb.thehendonmob.com/rankings/)

Or any of these people:
[https://en.wikipedia.org/wiki/List_of_World_Series_of_Poker_...](https://en.wikipedia.org/wiki/List_of_World_Series_of_Poker_Main_Event_champions)

~~~
panorama
I like your posts, but you're misinformed on this topic. The pros selected are
among the best HUNL players in the world right now. Offering the top winners
in a separate variant not only misinterprets how different the games are but
also falsely equivocates being the best with making the most money.

If someone asked you who the best mobile app developers were in the world,
your list should comprise solely of mobile app developers.

~~~
jedberg
You make a fair point. I clearly didn't know what I was talking about. I
edited my original post to reflect that.

~~~
panorama
No worries, thanks for taking the time to listen. Have a great day

------
bko
Does anyone know the technical details of how to train and AI bot for a game
like poker?

I imagine it's just reinforcement learning where the inputs are the actions of
the individual players (hold/fold/raise, timing etc) and the statistical
probabilities in terms of expected cards. Train a neural net to predict
probability of the opponent's hands and act accordingly.

Is it just that the professionals all act similarly enough that the bot can
learn based on other players?

~~~
bcassedy
I'm sure the actual process differs from research group to research group.
Here's how one commercially available AI training system claims to work.

[http://www.pokersnowie.com/about/technology-
training.html](http://www.pokersnowie.com/about/technology-training.html)

And they go into some of the problems that stem from how they've abstracted
the game here:

[http://www.pokersnowie.com/about/weaknesses.html](http://www.pokersnowie.com/about/weaknesses.html)

------
afastow
Sounds like the end of online poker is very near. This version required a
supercomputer and took a long time to decide its actions but those type of
things tend to be quickly improved given enough motivation.

Even if poker sites could somehow perfectly detect automated players(which
they can't of course), highly skilled poker is profitable enough that some
people would be willing to manually execute the actions themselves as directed
by the AI.

~~~
ndh2
Not really. 1v1 is just a small part of poker. Ring games are a whole
different beast.

~~~
afastow
They are more complex for sure but not fundamentally different. I'm amazed the
AI managed to win but now that it's happened it's bound to proliferate quickly
because there's so much money on the line.

------
snksnk
This article is related to no-limit heads-up, which requires a very different
style of play than no-limit full ring. Full ring would be significantly more
difficult to beat by AI. So the title leaves out some very important
information. I am not surprised they beat heads-up; I would have expected it
sooner.

------
georgeecollins
There is a game humans can still beat machines at with ease, Diplomacy(1).
When a machine wins a Diplomacy tournament I know we are finished as anything
except pets.

1\.
[https://en.wikipedia.org/wiki/Diplomacy_(game)](https://en.wikipedia.org/wiki/Diplomacy_\(game\))

~~~
bottled_poe
I'm yet to see a game of Diplomacy every actually end, so that makes it
difficult. The only reason this game is difficult for AI is the social
interaction required to play the game. If the computer was given a means of
making offers and private communication with other players, it would be much
like any other game. Rules, game state, probability, goal state, etc.

------
blueside
I'd really be interested in whether intentionally showing your cards instead
of mucking them was taken into account by the AI. (e.g. showing your poor hand
after a successful bluff)

~~~
xapata
It was not.

------
eliben
Does the computer-ness of the AI give it advantage in poker w.r.t. counting
cards, at which humans are imperfect? Or are humans perfect enough these days?

~~~
strictnein
There's no counting cards in poker. 52 card shuffled deck each time.

~~~
npongratz
It's not "counting cards," but to be minimally successful at certain variants
like seven card stud, I'd argue one must[0] remember cards that have been
folded in the current hand.

But certainly in all HE variants (that I'm aware of), counting and remembering
is unnecessary.

[0] It's necessary, but not sufficient.

------
wslh
Could someone explain how Poker is more challenging that games like Go? Please
eli5.

~~~
adamlett
I'm not qualified to give a comprehensive answer, but I would have to imagine
that it partly has to do with the fact that poker is a game of partial
information. In Go or Chess the current state of the game is known. In poker,
you don't know what cards your opponent hold, so you have to assign
probabilities based on previous actions, and simulate many different
scenarios.

Another part has to do with the fact that both you and your opponent have
surprisingly many legal "moves" at each turn. because not only must you decide
to fold, call or bet, but if you bet, you also have to decide _how much_.

------
karmacondon
Does anyone have link to papers about the technology behind Libratus?

~~~
niketear
the endgame solver used is described in
[http://www.cs.cmu.edu/~noamb/papers/17-AAAI-
Refinement.pdf](http://www.cs.cmu.edu/~noamb/papers/17-AAAI-Refinement.pdf)

the first authors twitter account
[https://twitter.com/polynoamial/](https://twitter.com/polynoamial/)

------
blueside
Unfortunately this fails to impress me, merely because you could theoretically
build a terrible AI and still win.

~~~
TylerE
Just showing your ignorance. That margin of victory over that many hands isn't
luck or coincidence. The pros were absolutely dominated.

~~~
blueside
I disagree and it's not ignorance on my part. Due to the nature of the poker
and heavily weighted component of luck, a terrible AI can theoretically win.

I am sure the AI they built is a mighty and spectacular achievement, but poker
is about the only game I know where the worst player can easily beat the best
player. Prove me wrong and I'll happily accept your insult of "ignorance"

 _edit_ I should have been more clear, as I am extremely impressed with the
AI's results of hands over time. I am referring to the context of a standard
tournament where the loser is eliminated after losing their chips.

~~~
sanswork
In these contests they typically play each hand twice with the sides reversed
to remove the luck factor.

So the deck for hand 1000 might be the same as hand 113853 but they swap who
is the button.

------
MaggieL
Surely nobody is surprised? Perfect knowledge of probabilities, no emotion and
immune to bluffing.

~~~
jasonwatkinspdx
> immune to bluffing

There's no such thing.

An algorithm playing straight hand value based on probabilities is more
susceptible to bluffing, not less. And this is no limit, where a single hand
can swing all the chips.

Any poker AI that isn't a loser is going to have some pretty sophisticated
modeling of the opponent.

~~~
algorias
Having a poker AI that plays deterministic strategies is an obviously terrible
idea, for the precise reason you mention.

It makes much more sense for the strategy space to be the set of probability
distributions over game moves (i.e. mixed strategies).

I _think_ that the optimal mixed strategy for each hand is immune to bluffing
(over many hands it will have larger expected winnings against a bluffer). If
that wasn't the case, there would exist no Bayes-Nash equilibrium for the
game, contradicting Nash's theorem.

~~~
jasonwatkinspdx
I was just making an informal remark, but reading your comment:

> I think that the optimal mixed strategy for each hand is immune to bluffing
> (over many hands it will have larger expected winnings against a bluffer).
> If that wasn't the case, there would exist no Bayes-Nash equilibrium for the
> game, contradicting Nash's theorem.

I believe that's true. I know for sure that heads up limit hold'em has been
solved. That said, I think this context is similar to the iterated prisoners
dilemma contest. There's certain to be an equilibrium, but what's interesting
isn't the perfect strategy in a min/max sense, but rather a slightly
suboptimal strategy that can detect and exploit suboptimal behavior in other
players. It sounds like you know this area well, perhaps you can shed some
light if I'm on the right hunch?

~~~
algorias
Yeah I think you're right. just playing an equilibrium is suboptimal in the
sense that it doesn't extract maximal value from the opponent. You want to be
playing a best response to the opponent's strategy.

That's what you see online poker players do. They model their opponents (in
the sense of labeling them as fun player, too tight, too loose, etc), then try
to predict their hands based on their moves. Otherwise I guess they would be
losing money: poker is zero sum by its nature, and the casino's cut on top of
that makes it negative sum!

