
Iterated Prisoners Dilemma Strategies Dominate Any Evolutionary Opponent (2012) - bookofjoe
https://www.researchgate.net/publication/225054497_Iterated_Prisoners_Dilemma_Contains_Strategies_That_Dominate_Any_Evolutionary_Opponent
======
random32840
Should be: "Iterated Prisoners Dilemma Contains Strategies That Dominate Any
Evolutionary Opponent _at a Game as Simple as the Prisoner 's Dilemma_"

You can write a perfect tic-tac-toe program with relatively simple rules, but
an evolutionary strategy will wipe the floor at Go. This kind of modelling has
value but real life is incredibly complicated, complicated strategies destroy
simple ones as the rules of the game become more complex. Our brains are
extremely expensive organs, and they're built that way for a reason. I think
people are way trigger-happy extrapolating models like this to the real world.

~~~
julienfr112
When games get more complex (Go), the problem with simple strategy is not that
they are less effective, it's that they are not computable.

~~~
random32840
Both are problems. Our brains have a tremendous amount of compute _and_ use
complex, nuanced, adaptive strategies.

The clearest example is how much energy we spend modelling other humans'
thought processes. We spend so much because a simple permutation of tit-for-
tat isn't sufficient - it will lose. It's an arms race to use complex
strategies, an outcome which is specifically precluded in the model. I'd argue
the fact observable reality is so divergent from the model means we should be
skeptical of the applicability of the model.

------
rstuart4133
A far less dry exploration of the same area:
[https://ncase.me/trust/](https://ncase.me/trust/)

------
throwaway55537
Defector: never cooperates

Forgiving: always give, even to those who don't cooperate

Tit-for-tat: cooperates by default but does not reward defectors/freeloaders.
This often the best strategy.

Interesting how this applies to software licenses.

Permissive licenses are clearly forgiving actors.

Copyleft/protective licenses are a gentler version of tit-for-tat.

~~~
mjevans
I don't recall where or when I read it, but there is a strategy that usually
beats Tit-fot-tat.

Tit for tat, but also a small (random) chance of forgiving (giving a break)
anyway. I believe the reason that won is that this allowed for recovery in the
face of understandings and as long as the chance wasn't that large the cost
also wasn't.

~~~
zatel
I don't have the link but theres a great online experience (game, post, idk
what to call it) that refers to what your talking about as copykitten

~~~
zatel
here it is [https://ncase.me/trust/](https://ncase.me/trust/)

------
thedudeabides5
_Only a player with a theory of mind about his opponent can do better, in
which case Iterated Prisoner 's Dilemma is an Ultimatum Game_

Interesting, if true, would the fist linkage between the Prisoner's Dilemma
and the Utlimatum Game.

[https://en.wikipedia.org/wiki/Ultimatum_game](https://en.wikipedia.org/wiki/Ultimatum_game)

~~~
im3w1l
I didn't read the article but I assume the ultimatum consists of "I'll
cooperate x% of the time if you cooperate 100% of the time. Otherise I'll
defect 100%".

Normally in game theory, such statements are not seen as "credible", i.e. you
assume the other person is bluffing and you go on to defect 100% of the time.

~~~
theaeolist
Maybe you should read the article?

~~~
im3w1l
Ok, I did, and it's pretty much it. Punishment is less severe.

~~~
thedudeabides5
Prisoner's Dilemma is about cooperation. "How much do you trust this person"

Ultimatum Game is about bargaining. "At what point are you willing to pay to
punish someone for an unfair deal?"

Still processing the paper, but the implication here would be that you can
link the two models, but the payoff to doing so critically depends on how much
the dominant player is capable of modeling the other player's mind.

Aka smartest model/strategy/robot/person wins.

~~~
im3w1l
It's evolutionarily favourable to accept the ultimatum. They claim that if you
have theory of the other players mind you should reject the ultimatum and hope
the other player gives you a better deal. Giving you a better deal isn't
possible in the original ultimatum game, but this is more of an iterated
ultimatum game.

------
im3w1l
A big reason for cooperating in iterated prisoners dilemma in nature is that
the benefits from cooperating with relatives is huge.

And in a pool of cooperative agents doing some "last-turn-defect" strategy
while theoretically better than cooperate-always, is complicated with small
payoff.

------
scythe
The critical flaw in this strategy is trivial: if you pit it against itself,
both participants refuse to cooperate every time (and, presumably, die).

~~~
random32840
Forgiving tit-for-tat rectifies that flaw.

------
remolueoend
There is a great lecture from Robert Sapolsky/Stanford [2010] about behavioral
evolution / prisoner's dilemma / tit for tat strategy:
[https://youtu.be/Y0Oa4Lp5fLE](https://youtu.be/Y0Oa4Lp5fLE)

------
danaliv
I remember reading, in a book on GAs published in the 90s, that a “tit-for-
tat” strategy* handily beat evolved players.

*Wherein the player does whatever their opponent did in the previous round.

~~~
hazeii
Covered in some detail in Robert Axelrod's "Evolution of Cooperation" from
1984 [0] which is a book resulting from the original paper with W D Hamilton.
Anatol Rappaport submitted "Tit for tat" as a strategy in a computerised
tournament of programs adressing the Prisoners Dilemma, , and it wiped the
floor with the opposition. I don't recall the full details, there was a second
round with some restrictions, Rappaport simply submitted TfT again and it came
out well even with constraints on it.

[0]
[https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation)

~~~
alasdair_
There is actually a better strategy than tit for tat, for iterated prisoner's
dilemma played in a tournament setting where the winner is determined by total
score at the end of all rounds.

The strategy is to submit lots of entrants to the tournament rather than just
one, then play tit for tat against all opponents except those that are in the
clique. The clique players then lose on purpose to a specifically-chosen
player, maximizing that players score.

If you don't want to do it this way and instead care about maximizing the
total scores of the clique as a whole, you can default to every clique member
always cooperating.

Even in cases where the opponent is "blind", you can use a special pattern of
bets to signal that the other player is a member of the clique and then play
your win-maximizing strategy once the signal is detected.

This kind of thing has real-world implications. For example: imagine a poker
game with ten players, all of the same skill level and without a rake. Nine of
the players can collude (say, show each other their hidden cards and make
decisions based on the shared information), making it significantly less
likely than a 10% chance that the targeted player will win. I suspect this is
already done to an extent in extremely high-stakes games (say, $1MM+).

~~~
lonelappde
It's trivial to say that cheating in a competition is a winning strategy as
long as you don't get caught and punished.

~~~
alasdair_
At lease for standard iterated prisoners dilemma AI tournaments, this strategy
was completely legal - I read about it severals years ago when it was first
used to win a prominent one.

The poker one already happens in real life all the time - if friends go to
play at the same table, they often play differently than with a stranger.

------
rurban
That's the classical post-Axelrod right-wing paper (Freeman Dyson of course)
against the Nowak school of cooperative thought.

See eg. [https://www.nature.com/news/physicists-suggest-
selfishness-c...](https://www.nature.com/news/physicists-suggest-selfishness-
can-pay-1.11254) which describes that political struggle going on for the last
40 years. This was eg countered by
[https://www.researchgate.net/publication/236189156_The_Evolu...](https://www.researchgate.net/publication/236189156_The_Evolution_of_Extortion_in_Iterated_Prisoner's_Dilemma_Games)

------
raverbashing
I don't get it why game theorists use so much the Prisoner's Dilemma.

In most real world situations, the payoffs are much different than the PD
ones.

Collaborate, and you may lose or win a little. Defect and for most cases, your
payoff is 0.

~~~
danharaj
It's simple and demonstrates that the Nash equilibria of a game can deviate
from the strategies that produce the best payoff. I think most pop sci
corollaries people try to draw from it are overreaches at best.

