
Bridges Supercomputer Used to Build AI Model for Beating Humans at Poker - jonbaer
https://www.top500.org/news/bridges-supercomputer-used-to-build-ai-model-for-beating-humans-at-poker/
======
Humdeee
30k hands (as opposed to 20k in the first experiment) is usually still not
large enough of a sample size from a player's perspective to overcome up and
downswings. I suppose a compromise must have been made out of respect for the
player's time. I would be very interested in seeing everyone's equity vs
actual winnings at the end.

The pay per hand is very good for any normal mid to higher stakes reg player,
but I wonder if these guys are taking a pay cut to participate... considering
these guys probably charge $500 - $1000 an hour for coaching, and that's aside
from their table winnings. I haven't been in the poker scene for some time
now, and am a bit out of touch.

I wish we could get some real time player commentary. I would love to see how
they adapt with their thought process.

------
nabla9
>Heads-Up No-Limit Texas Hold’em.

Heads-Up is much easier game than poker against multiple opponents.

The ultimate test for AI is multi-way against 6 to 9 professional poker
players. It's completely different game with more layers of complexity.

~~~
spectrum1234
This actually isn't true. Ranges are wider HU which means you need to be more
precise.

Proof: Top HU players can crush 6-9 handed but 6max players get killed HU.

~~~
nabla9
Precision is easy for computer. You don't need AI or supercomputers to
calculate good ranges or pot odds.

In multi-way poker the way you estimate range of another player has effect on
how you estimate the range of other players and how you present yourself.

~~~
spectrum1234
What I mean is mistakes are more mostly HU.

~~~
smallnamespace
AIs and humans have different skill sets. Could be that HU is harder for a
human but easier for AIs that we can write today -- math is not a problem for
a computer but modeling many opponents might be.

------
21
> more than the number of atoms in the universe

Ah, I was scanning the article looking for this phrase. They delivered.

~~~
cjslep
The real news is Michael Feldman should share with us his count of the atoms
in the unobservable universe. I'd be interested in methodology, too.

~~~
archgoon
This is a fairly well known and common estimate.

[https://en.wikipedia.org/wiki/Observable_universe#Matter_con...](https://en.wikipedia.org/wiki/Observable_universe#Matter_content)

~~~
sa1
You shared the estimate for the number of atoms in the _observable_ universe.
afaik, we don't have an estimate for the _unobservable_ universe.

~~~
archgoon
But Michael Feldman did not say that he could do that, he merely said the
universe. Through context, it is obvious that he is restricting to the
observable universe, just like when people say "American" to specifically to
refer to people in the United States.

Sorry, I hadn't realized we were arguing about this level of pedantry.

~~~
cjslep
I just think you missed my very dry wit highlighting how even such an overused
phrase can be technically incorrect, the worst kind of incorrect. Even when
the rest of the more-technical-than-an-overused-phrase article could be
correct.

My apologies, I didn't realize people would take my original response so
seriously.

------
neom
Can you bet on algorithms yet? :)

Should be fun when you're able to look at all the teams that built the
algorithms playing each other, the way they think about building their ML etc
and then place your bets around that or something. Sounds like actually kinda
fun gambling.

~~~
lawless123
Will we be naming them like horses? :-)

Someone will come up with an AI that's really good at betting on other AI's..

~~~
gallerdude
Won't stop me from betting on the one with the best name

------
krick
Can somebody explain why poker is a big deal? I don't play poker and I don't
see why is it complex for an AI. I can see why chess/go could be a big deal:
there's definitely a solution to the game, but the space is too large for it
to be computed, so you have to optimize somehow, which real players somewhat
can. When computers were slow, humans did it better than computers.

Poker, I thought, doesn't really have any solutions, it is largely about luck,
no one really knows what's going on, so aside from some probabilistic
computations it is about guessing what your opponent thinks while not giving
up what you think. Choosing and seeing the behavior. I would think that
playing against AI is generally pointless, since computers don't worry. I
mean, surely there are some general guidelines on strategy, since somebody
plays better that the others, but following it shouldn't be harder for a
computer than for a human, quite the opposite.

So, what is true?

~~~
Cybiote
Poker in a sense, is the most realistic game in that it has multiplayer,
hidden information and stochastic aspects. While chess and go have huge state
spaces (as does no limit poker), they are also fairly straight forward due to
their 2 player perfect information nature. If say you took an algorithm key to
alphago, something like UCT, and tried to apply it to poker, it would diverge
from nash equilibrium. Nash Equilibrium by the way, is the sense in which the
game has solutions.

In poker, you're looking at your expected value per decision (good players
also do this at a meta-level), you're trying to place a bound on the other
players' holdings and trying to bluff at an optimal rate. A good poker bot has
to be capable of all this, which has proved extremely difficult. This is why
it has proven difficult until just about this year, for researchers to be able
to clearly say that at least in the heads up case and given enough compute to
train, bots are at expert human level.

What might turn out interesting is the fact that though headsup is more
difficult for humans and multiplayer is arguably the simpler, the reverse
seems to be true for bots. The state space multiplies such that naive
application of equilibrium play and tree search simply do not cut it.

~~~
krick
I don't understand. Maybe there are some good resources to read, so I could
see it better?

Essentially, you are explaining why poker it hard. This is no surprise, and I
already included it into my question to an extent. The real question is, why
is it _harder_ for computers than for humans? Both seem to be completely
oblivious to what is truly going on due to imperfect information nature of
poker, and both seem to be able to make simple estimations. Computers are
better at multiplying fractions (needed to estimate probability) and computers
are better at not-worrying (needed to bluff well). I'd assume it is humans who
are at disadvantage, and your post doesn't explain (it seems) why is it the
otherwise.

~~~
spectrum1234
As said before because of hidden information the optimal strategy appears to
diverge unless you have more (all) the information in the decision tree.
Basically you can never make a decision with the specific hand you have in
isolation without knowing what you'd do with every other hand (and same for
your opponent). This is very different than chess/go!

So while poker may even have a simpler game tree than chess/go you need to
have more of it upfront to solve the game, and being "off" means much much
greater error. Depth matters less than breadth.

Said another way in chess/go you can brute force get near optimal for the
current decision, then reset and do again for the next decision. But with
poker you have to solve for ALL POSSIBLE decisions (hands) up front (and
inclusive of all possible board situations), and THEN do AGAIN 3 more times
(pre-flop flop turn river) plus the numerous times you are raised means this
is much more than 4 times total.

Example: Bot is out of position with AdAh on AsKsQc facing a big raise. In
isolation its probably best to re-raise and simulations show this makes the
most money. But you also need to account for the times you have AT (no flush
draw) and need to save AT money.

But this logic doesn't stop here. Typically on the river out of position you
will check the majority of your range if your opponent has bet every street
(this is one of the few "proven" strategies because you always want the pot
smaller out of position on average). So, you literally need to have the entire
game solved in order to know HOW TO GET TO THE RIVER in a balanced way with
good and bad hands. And, getting to the river to play profitably likely means
giving up profitability on every other street - this is the diverging from
optimal that default algorithms will be wrong on unless they have the entire
game tree in front of them.

~~~
krick
This again has nothing to do with question in hand. Yes, poker is imperfect
information game, everybody knows that. Why does it make it easier for humans
than for computers? Humans cannot do these computations in the head any better
than computers, quite the contrary.

~~~
spectrum1234
Hard to explain well. Let me try another way.

Thinking through this more right now, it seems like the default way of
computers to solve this penalizes them. Every time they get more depth it
hurts them unless you account for more breadth.

So lets say the programers know this. This STILL probably doesn't help much
because...well idk. What's interesting is Tensor Flow is supposed to be
massively parallel (breadth) and abstract the software/algorithms to do this.
But if you follow this logic the answer must be gradient decent algorithms
also get stuck somewhere... This must have to do with hyper-sensitivity to
really having the full decision tree before optimizing.

The answer seems to be if computers are modeling this exactly as humans do,
(using neural networks which then simplify this problem with heuristics), it
just takes this much computer power to simulate 1 human brain optimizing for
this.

Alternatively if they model it in a more brute force way it must take even
more computer power because this is proving to be less efficient.

------
laegoose1
"four of the most accomplished professional poker players in the world: Jason
Les, Dong Kim, Daniel McAulay and Jimmy Chou"

I am ex-online poker professional and I still follow poker news. These guys do
not play online high-stakes NL Holdem, and online is the place for best
players wrt strategy.

Last time line-up was better. There was Doug Polk who is one of the very top
online players, and he had great results offline too. Btw last time CME held
computer-human match he was pissed that 'tie' result was announced because
human players only beat computer within 90% confidence interval and 'victory'
would be 95% confidence interval as it was explained.

There are dozens of players who play higher stakes, more tournament winnings,
more google hits, higher rankings than these 4. It's a shame CME could not
bring human players that are actually good.

~~~
spectrum1234
Yea but this is splitting hairs. All of these players are better than the best
3-4 years ago.

------
mbroshi
The best no-limit hold-em players in the world are also presumably quite good
at limit hold-em, limit omaha, 7 card stud, etc., not to mention other card
games like bridge. Serious question: When these programs are built to beat
humans in one very particular task, are they any better at beating humans at
similar tasks?

~~~
nabla9
Different games have different difficulties for computers. For average human
player just counting the odds may be hard.

Just different Texas-hold'em variations have different challenges for
computers.

Texas-hold em is very simple if you just play the cards and calculate odds.
Limit hold-em is much easier for computer than no-limit hold em. Head's up no-
limit hold-em is much simpler than multi-way.

------
lawless123
I didn't realize they would need a supercomputer to do this.

~~~
fisherjeff
It sounds like the supercomputer was only used to train the model, which
consumed ~1700 years of CPU time.

~~~
lawless123
Could i cut a few zeros off that if i trained a NN to beat drunk players
online at 1am on a Friday?

~~~
eru
For that, you can just play the pot odds, and don't even need to model the
other players very well.

(The challenge is of course to find table that only have drunken humans.)

