doi: 10.1073/pnas.1214834110.
Epub 2013 Apr 9.
Evolution of extortion in Iterated Prisoner's Dilemma games
Affiliations
- PMID: 23572576
- PMCID: PMC3637695
- DOI: 10.1073/pnas.1214834110
Item in Clipboard
Evolution of extortion in Iterated Prisoner's Dilemma games
Proc Natl Acad Sci U S A.
.
Abstract
Iterated games are a fundamental component of economic and evolutionary game theory. They describe situations where two players interact repeatedly and have the ability to use conditional strategies that depend on the outcome of previous interactions, thus allowing for reciprocation. Recently, a new class of strategies has been proposed, so-called "zero-determinant" strategies. These strategies enforce a fixed linear relationship between one's own payoff and that of the other player. A subset of those strategies allows "extortioners" to ensure that any increase in one player's own payoff exceeds that of the other player by a fixed percentage. Here, we analyze the evolutionary performance of this new class of strategies. We show that in reasonably large populations, they can act as catalysts for the evolution of cooperation, similar to tit-for-tat, but that they are not the stable outcome of natural selection. In very small populations, however, extortioners hold their ground. Extortion strategies do particularly well in coevolutionary arms races between two distinct populations. Significantly, they benefit the population that evolves at the slower rate, an example of the so-called "Red King" effect. This may affect the evolution of interactions between host species and their endosymbionts.
Conflict of interest statement
The authors declare no conflict of interest.
Figures
Payoffs 
and 
of players I and II if both players use memory-one strategies in an IPD game (with 
, 
, 
, and 
). In each graph, the strategy of player I is fixed to some p, whereas the strategy q of the coplayer II can vary, sampling the 4D cube of memory-one strategies (the blue dots correspond to 
different realizations of q). (A) In general, the payoff-pairs of the two players cover a 2D area, as here, when player I applies the strategy of WSLS (i.e., 
and 
). (B) However, if player I adopts a ZD strategy, the possible payoff-pairs are restricted to a line. Two special classes of ZD strategies were highlighted by Press and Dyson (10): equalizers [strategies that set the coplayer’s score to a fixed value (the line of payoffs has slope zero)] (C), and extortioners [strategies that guarantee the surplus of player I is the χ-fold of the surplus of player II (i.e., 
, with 
(the line of payoffs has a positive slope and intersects the diagonal at P)] (D).
and of players I and II if both players use memory-one strategies in an IPD game (with , , , and ). In each graph, the strategy of player I is fixed to some p, whereas the strategy q of the coplayer II can vary, sampling the 4D cube of memory-one strategies (the blue dots correspond to different realizations of q). (A) In general, the payoff-pairs of the two players cover a 2D area, as here, when player I applies the strategy of WSLS (i.e., and ). (B) However, if player I adopts a ZD strategy, the possible payoff-pairs are restricted to a line. Two special classes of ZD strategies were highlighted by Press and Dyson (10): equalizers [strategies that set the coplayer’s score to a fixed value (the line of payoffs has slope zero)] (C), and extortioners [strategies that guarantee the surplus of player I is the χ-fold of the surplus of player II (i.e., , with (the line of payoffs has a positive slope and intersects the diagonal at P)] (D).
Reactive strategies (
, 
) for the donation game. All reactive strategies (the square 
) are ZD strategies. The equalizer strategies are those on the segment between “generous TFT” (GTFT) (
, 
) (16) and “Miser” (
, 
) (30), the extortion strategies are those between Miser and TFT (
, 
), and the “compliant” strategies (Discussion and ref. 11) are those between GTFT and TFT.
, ) for the donation game. All reactive strategies (the square ) are ZD strategies. The equalizer strategies are those on the segment between “generous TFT” (GTFT) (, ) (16) and “Miser” (, ) (30), the extortion strategies are those between Miser and TFT (, ), and the “compliant” strategies (Discussion and ref. 11) are those between GTFT and TFT.
Evolutionary competition between some important strategies in the IPD. For various population sizes M, the graphs show the frequency of each strategy in the mutation-selection equilibrium. We consider two mutation regimes: the limit of rare mutations 
(Upper), for which the equilibrium can be calculated analytically using the method of Fudenberg and Imhof (23), and a regime with mutation rate 
(Lower), which is explored by individual-based simulations. For the copying process, we assume that individuals A and B are chosen randomly. A switches to B’s strategy with a probability given by 
, where 
and 
are the corresponding payoff values and 
corresponds to “selection strength” (cf. ref. 21). (A) If All D competes with WSLS, the population is mostly in the defector’s state, independent of population size and the mutation rate. (B and C) However, once 
or TFT is added, WSLS succeeds if populations are sufficiently large. TFT works slightly better than 
. (D and E) Adding All C only leads to minor changes in the stationary distribution, which now slightly favors 
. The parameters are 
, 
, 
, and 
.
(Upper), for which the equilibrium can be calculated analytically using the method of Fudenberg and Imhof (23), and a regime with mutation rate (Lower), which is explored by individual-based simulations. For the copying process, we assume that individuals A and B are chosen randomly. A switches to B’s strategy with a probability given by , where and are the corresponding payoff values and corresponds to “selection strength” (cf. ref. 21). (A) If All D competes with WSLS, the population is mostly in the defector’s state, independent of population size and the mutation rate. (B and C) However, once or TFT is added, WSLS succeeds if populations are sufficiently large. TFT works slightly better than . (D and E) Adding All C only leads to minor changes in the stationary distribution, which now slightly favors . The parameters are , , , and .
Statistics of the evolutionary dynamics for memory-one strategies for a range of different population sizes. We have calculated the relative abundance of extortioners, equalizers, and ZD strategies (i.e., the time spent in a δ-neighborhood) divided by the volume of the intersection of that neighborhood with the set of memory-one strategies (A), the average strategy of the population (B), and the average payoff (C). Extortioners, equalizers, and ZD strategies are only favored for small population sizes. As the population size increases, individuals tend to apply WSLS-like strategies and to cooperate only after mutual cooperation or mutual defection. As a result, the average payoff increases with population size. For the simulations, 
mutant strategies were randomly drawn from the space of memory-one strategies. As in the study by Imhof and Nowak (22), the switch from a monomorphic population using strategy X to a monomorphic population using strategy Y occurred with the probability of fixation of a single Y mutant in a population of X residents. The parameters are 
, 
, 
, and 
.
mutant strategies were randomly drawn from the space of memory-one strategies. As in the study by Imhof and Nowak (22), the switch from a monomorphic population using strategy X to a monomorphic population using strategy Y occurred with the probability of fixation of a single Y mutant in a population of X residents. The parameters are , , , and .
Evolution of extortion in host–symbiont interactions. The graphs show two typical simulation runs for a population of 40 hosts, with each having a subpopulation of 20 symbionts. For each simulation run, one graph (Upper) shows the average payoff for each population, whereas the other graph (Lower) shows the Euclidean distance of each population to the set of extortioners (which can be 1.5275 at most). In the initial population, all individuals cooperate unconditionally. Further evolution depends on the RER. (A) If RER = 1, both species converge toward All D and no population is able to extort the other. (B) For RER = 200, symbionts evolve much more quickly. In the short term, they can thus increase their average payoff by switching to a noncooperative strategy. However, in the long term, hosts apply extortion strategies to force their symbionts to cooperate. Eventually, the hosts’ payoff exceeds 
, whereas the symbionts’ payoff is close to zero. To model the evolutionary process, we followed the method of Damore and Gore (26). Whenever a symbiont reproduces, its offspring remains associated with the same host. Whenever the host reproduces, the new host offspring acquires its symbionts from other hosts (horizontal transmission). Mutations occur with probability 
, by adding Gaussian noise to an entry of the memory-one strategy of the parent (
). The process is run for 2,000 host generations (corresponding to more than 
reproduction events for RER = 1 and more than 
reproduction events for RER = 200). The other parameters are 
, 
, and 
.
, whereas the symbionts’ payoff is close to zero. To model the evolutionary process, we followed the method of Damore and Gore (26). Whenever a symbiont reproduces, its offspring remains associated with the same host. Whenever the host reproduces, the new host offspring acquires its symbionts from other hosts (horizontal transmission). Mutations occur with probability , by adding Gaussian noise to an entry of the memory-one strategy of the parent (). The process is run for 2,000 host generations (corresponding to more than reproduction events for RER = 1 and more than reproduction events for RER = 200). The other parameters are , , and .Similar articles
-
Adaptive dynamics of extortion and compliance.PLoS One. 2013 Nov 1;8(11):e77886. doi: 10.1371/journal.pone.0077886. eCollection 2013. PLoS One. 2013. PMID: 24223739 Free PMC article.
-
Autocratic strategies for iterated games with arbitrary action spaces.Proc Natl Acad Sci U S A. 2016 Mar 29;113(13):3573-8. doi: 10.1073/pnas.1520163113. Epub 2016 Mar 14. Proc Natl Acad Sci U S A. 2016. PMID: 26976578 Free PMC article.
-
From extortion to generosity, evolution in the Iterated Prisoner's Dilemma.Proc Natl Acad Sci U S A. 2013 Sep 17;110(38):15348-53. doi: 10.1073/pnas.1306246110. Epub 2013 Sep 3. Proc Natl Acad Sci U S A. 2013. PMID: 24003115 Free PMC article.
-
Resolving the iterated prisoner's dilemma: theory and reality.J Evol Biol. 2011 Aug;24(8):1628-39. doi: 10.1111/j.1420-9101.2011.02307.x. Epub 2011 May 23. J Evol Biol. 2011. PMID: 21599777 Review.
-
Spatialization and greater generosity in the stochastic Prisoner's Dilemma.Biosystems. 1996;37(1-2):3-17. doi: 10.1016/0303-2647(95)01541-8. Biosystems. 1996. PMID: 8924636 Review.
Cited by
-
Recognising and evaluating the effectiveness of extortion in the Iterated Prisoner's Dilemma.PLoS One. 2024 Jul 26;19(7):e0304641. doi: 10.1371/journal.pone.0304641. eCollection 2024. PLoS One. 2024. PMID: 39058703 Free PMC article.
-
The distorting effects of producer strategies: Why engagement does not reveal consumer preferences for misinformation.Proc Natl Acad Sci U S A. 2024 Mar 5;121(10):e2315195121. doi: 10.1073/pnas.2315195121. Epub 2024 Feb 27. Proc Natl Acad Sci U S A. 2024. PMID: 38412133 Free PMC article.
-
A geometric process of evolutionary game dynamics.J R Soc Interface. 2023 Nov;20(208):20230460. doi: 10.1098/rsif.2023.0460. Epub 2023 Nov 29. J R Soc Interface. 2023. PMID: 38016638 Free PMC article.
-
Multiple thresholds and trajectories of microbial biodiversity predicted across browning gradients by neural networks and decision tree learning.ISME Commun. 2021 Aug 16;1(1):37. doi: 10.1038/s43705-021-00038-8. ISME Commun. 2021. PMID: 37938633 Free PMC article.
-
Zero-determinant strategy in stochastic Stackelberg asymmetric security game.Sci Rep. 2023 Jul 12;13(1):11308. doi: 10.1038/s41598-023-38460-8. Sci Rep. 2023. PMID: 37438579 Free PMC article.
References
-
- Rapoport A, Chammah A. The Prisoner’s Dilemma. Ann Arbor, MI: Univ of Michigan Press; 1965.
-
- Trivers R. The evolution of reciprocal altruism. Q Rev Biol. 1971;46(1):35–57.
-
- Aumann R. Essays in Game Theory and Mathematical Economics in Honor of Oskar Morgenstern. Mannheim, Germany: Wissenschaftsverlag; 1981. Survey of repeated games.
-
- Axelrod R. The Evolution of Cooperation. New York: Basic Books; 1984.
-
- Fudenberg D, Maskin E. The folk theorem in repeated games with discounting or with incomplete information. Econometrica. 1986;50:533–554.
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Other Literature Sources
