
New Dilemmas for the Prisoner (2013) - zeristor
http://www.americanscientist.org/issues/pub/new-dilemmas-for-the-prisoner
======
elbigbad
This passage is intriguing, does anyone have the full story of how they came
to make the discovery?:

"Last year’s big surprise in Prisoner’s Dilemma research came from two
distinguished polymaths. William H. Press . . . [and] . . . Freeman J. Dyson .
. . The story of how Press and Dyson came to make their discovery is almost as
interesting as the result itself, but I have room here only for the latter."

~~~
jeffhussmann
A brief version of the story is contained in [http://numerical.recipes/whp/on-
iterated-prisoner-dilemma.ht...](http://numerical.recipes/whp/on-iterated-
prisoner-dilemma.htm)

------
xrange
Some further links:

Iterated Prisoner’s Dilemma contains strategies that dominate any evolutionary
opponent

[https://www.ncbi.nlm.nih.gov/pubmed/22615375](https://www.ncbi.nlm.nih.gov/pubmed/22615375)

Extortion and cooperation in the Prisoner’s Dilemma

[http://numerical.recipes/whp/StewartPlotkinExtortion2012.pdf](http://numerical.recipes/whp/StewartPlotkinExtortion2012.pdf)

Zero-Determinant Strategies in the Iterated Prisoner’s Dilemma

[https://golem.ph.utexas.edu/category/2012/07/zerodeterminant...](https://golem.ph.utexas.edu/category/2012/07/zerodeterminant_strategies_in.html)

------
xrange
Anyone have recommendation for good books on game theory or prisoner's
deilemma's? I could see lots of variations on a theme, where you have:

* Precisely finite games (for example, exactly 100 games).

* Finite games where the number of games is random. How does strategy change when you know you'll be playing somewhere between 95-105 games vs. 60-140 games?

* Games where players can "pay" for awards or punishments out of their own fixed pot, instead of the externally set payoffs. What are good strategies for starting out with a small pot, vs. a large pot, and how to best cope with "inequality".

~~~
te
Not a book, and I wish it wasn't so expensive, but this is excellent:
[http://www.thegreatcourses.com/courses/games-people-play-
gam...](http://www.thegreatcourses.com/courses/games-people-play-game-theory-
in-life-business-and-beyond.html)

------
macawfish
This is low key some of the most profound research I've ever seen. It flies in
the face of cold war era paranoid politics but: "the meek shall inherit the
earth."

~~~
SixSigma
Except extortion works for a while. So while the meek wait for their
inheritance, they are being kicked from pillar to post.

------
nordsieck
I'm pretty skeptical of the tournament referenced at the end of the article -
specifically that tit-for-tat did so poorly.

[http://lesswrong.com/lw/7f2/prisoners_dilemma_tournament_res...](http://lesswrong.com/lw/7f2/prisoners_dilemma_tournament_results/)

------
sampsonetics
(I wrote a fun little essay about Press & Dyson and Adami & Hintze for my
scientific writing class last summer. The assignment was to write an essay of
no more than 1000 words, intended for a non-scientific audience. Not sure if
pasting it here is too much for HN etiquette, but it's not published anywhere
else, so here goes. I hadn't read the article linked above when I wrote this,
but it covers very similar ground in much simpler terms. Disclaimer: I didn't
actually interview any of the researchers for the purpose of this short class
assignment, so I was speculating a bit about their subjective states!)

Christoph Adami and Arend Hintze didn't believe what they were reading. The
two evolutionary biologists suspected that something didn't quite add up in a
2012 article by two of the most influential physicists of our time, William
Press and Freeman Dyson.[1] Press and Dyson claimed that they had discovered a
mathematical trick that flew in the face of results from three decades
earlier, which had so far stood the test of time.

The subject of their argument was a paradoxical "game" called the Prisoner's
Dilemma. This game was invented half a century ago to challenge the tenets of
_game theory_ , a branch of mathematics beloved by economists and political
scientists. In the Prisoner's Dilemma, two players face an awkward situation
in which they cannot communicate but their choices affect each other. Each
player must make a simple choice: Will you "cooperate" or will you "defect"?

The game is set up so that it presents a common dilemma faced in real life.
The players will be better off if they both cooperate than if they both
defect. But either player can gain an advantage by defecting when the other
player cooperates. With every move, you are tempted to take advantage of the
other player and afraid that they will take advantage of you, even though you
both know that overall it's better to cooperate.

The Prisoner's Dilemma is reminiscent of any situation in which two
individuals will share equally in the benefits of a shared activity.
Cooperating then means _giving it your all_ and defecting means _slacking
off_. If you both cooperate then you both benefit from your hard work. If you
both defect then there are no benefits to be had. Whenever the _other_ player
cooperates, however, you are better off defecting in order to benefit from
their effort without exerting any yourself. Likewise, whenever the other
player defects, then once again you are better off defecting yourself because
it's not worth your effort to support them.

The paradox of the Prisoner's Dilemma is this: If it always seems better to
defect, no matter what the other player does, how do we ever end up
cooperating? This question, and therefore the Prisoner's Dilemma itself, is of
prime interest to economists, political scientists, biologists, and many
others. The standard answer from game theory was that a "rational" player
would always defect, but some researchers suspected that cooperation might
actually be favored by evolution in the long run.

In a crucial paper in 1981, political scientist Robert Axelrod and
evolutionary biologist William Hamilton reported on a Prisoner's Dilemma
tournament that Axelrod had conducted.[2] Axelrod invited dozens of
researchers to submit computer programs that would play the game thousands of
times. Hoping that the tournament would provide insights into how cooperation
might evolve, he contacted one of its participants, biologist Richard Dawkins,
who introduced him to Hamilton,[3] a biologist at Axelrod's own university who
had published a research paper of his own about the Prisoner's Dilemma.

In the preceding decades, discussion of the Prisoner's Dilemma was largely
philosophical. It was an interesting ethical puzzle, intriguingly simple and
yet maddeningly difficult to reason about. With his computerized tournament,
Axelrod advanced Prisoner's Dilemma research from mere philosophical argument
to proper scientific experiment.

The result was stunning. The submitted programs displayed a wide variety of
strategies -- some preferred to cooperate, some preferred to defect, some
tried to outwit their opponents. But the winner was one of the simplest
programs of all, called Tit-For-Tat. Its strategy was to always start out
cooperating. As long as its opponent likewise cooperated, Tit-For-Tat happily
went on cooperating, too. But as soon as its opponent defected, even for a
single round, Tit-For-Tat followed along. Its choice for each move was
whatever its opponent chose for the preceding move.

In Axelrod and Hamilton's summary of the tournament, they noted that all of
the top programs shared Tit-For-Tat's preference for starting out
cooperatively. The more aggressive programs effectively lost the trust of the
others and missed out on the benefits of cooperation. Cooperation was
vindicated once and for all. Axelrod and others have repeated the tournament
with similar results.

Hence Adami and Hintze's skepticism reading Press and Dyson's claim, 31 years
later, of discovering a new mathematical trick for playing the Prisoner's
Dilemma along with a rigorous proof that it should beat the classic
cooperative strategies. It was a fascinating mathematical theory, but how
could it have gone unnoticed before?

Press and Dyson called their discovery "zero-determinant (ZD) strategies"
because the technique involves causing a mathematical quantity known as a
"determinant" to be zero. By manipulating this determinant, one player can
limit how well the other player performs. The ZD player can choose a
particular score for its opponent or can "extort" an unfair score for itself,
averaged over a large number of rounds. Press and Dyson seemed to have
overthrown cooperation as the star of Prisoner's Dilemma research.

ZD strategies came as a surprise to game theory experts, including Adami and
Hintze. Press and Dyson's analysis was mathematically solid, but since it flew
in the face of Axelrod's tournament results, something was missing. Adami and
Hintze decided to run tournaments similar to Axelrod's, pitting ZD strategies
against cooperative strategies to see how they evolved, publishing their
results in 2013.[4]

Sure enough, coercive ZD strategies excelled in one-on-one competition but
fared poorly in the tournament. Why? In a tournament, ZD strategies are
competing against _each other_ as well as non-ZD strategies. Just as Axelrod
had learned three decades earlier, cooperative strategies reinforce each other
while non-cooperative strategies are marginalized. Cooperation is vindicated
once again!

That's not quite the end of the story, though. It turns out that ZD strategies
can be generous as well as coercive. ZD players can influence their opponents
to accept good scores as easily as bad scores -- and generous ZD strategies do
just fine in a tournament setting. In fact, you're already familiar with one
such strategy: the classic Tit-For-Tat itself is the "most fair" of all ZD
strategies!

[1] "Iterated Prisoner's Dilemma contains strategies that dominate any
evolutionary opponent," _Proceedings of the National Academy of Sciences_ ,
volume 129, issue 26, pages 10409-10413.

[2] "The evolution of cooperation," _Science_ , volume 211, issue 4489, pages
1390-1396.

[3] This introduction via Dawkins was described by Axelrod in 2012: "Launching
'the evolution of cooperation,'" _Journal of Theoretical Biology_ , volume
299, pages 21-24.

[4] "Evolutionary instability of zero-determinant strategies demonstrates that
winning is not everything," _Nature Communications_ , volume 4, article number
2193.

~~~
mratzloff
Nice article! Well-written and informative, tells a story, and uses simple,
clear language. Hope you got an A. :-)

------
zeristor
Reposting requested by HN.

~~~
xrange
Thanks for reposting this, quite interesting.

