Collapse of cooperation in evolving games

Game theory provides a quantitative framework for analyzing the behavior of rational agents. The Iterated Prisoner’s Dilemma in particular has become a standard model for studying cooperation and cheating, with cooperation often emerging as a robust outcome in evolving populations. Here we extend evolutionary game theory by allowing players’ payoffs as well as their strategies to evolve in response to selection on heritable mutations. In nature, many organisms engage in mutually beneficial interactions and individuals may seek to change the ratio of risk to reward for cooperation by altering the resources they commit to cooperative interactions. To study this, we construct a general framework for the coevolution of strategies and payoffs in arbitrary iterated games. We show that, when there is a tradeoff between the benefits and costs of cooperation, coevolution often leads to a dramatic loss of cooperation in the Iterated Prisoner’s Dilemma. The collapse of cooperation is so extreme that the average payoff in a population can decline even as the potential reward for mutual cooperation increases. Depending upon the form of tradeoffs, evolution may even move away from the Iterated Prisoner’s Dilemma game altogether. Our work offers a new perspective on the Prisoner’s Dilemma and its predictions for cooperation in natural populations; and it provides a general framework to understand the coevolution of strategies and payoffs in iterated interactions.Iterated games provide a framework for studying social interactions (1–6) that allows researchers to address pervasive biological problems such as the evolution of cooperation and cheating (2, 7–12). Simple examples such as the Iterated Prisoner’s Dilemma, Snowdrift, and Stag Hunt games (13–18) showcase a startling array of counterintuitive social behaviors, especially when studied in a population replicating under natural selection (16, 19–25). Despite the subject’s long history, a systematic treatment of all evolutionary robust cooperative outcomes for even the simple Iterated Prisoner’s Dilemma has only recently emerged (21, 26–29).Understanding the evolution of strategies in a population under fixed payoffs already poses a steep challenge. To complicate matters further, in many biological settings the payoffs themselves may also depend on the genotypes of the players. Changes to the payoff matrix have been studied in a number of contexts, including one-shot two-player games (13), payoff evolution without strategy evolution (30, 31), under environmental “shocks” to the payoff matrix (32–34), and using continuous games (22, 23, 35). Here we adopt a different approach, and we explicitly study the coevolutionary dynamics between strategies and payoffs in iterated two-player games. We decouple strategy mutations from payoff mutations, and we leverage results on the evolutionary robustness of memory-1 strategies with arbitrary payoff matrices to explore the relationship between payoff evolution and the prevalence of cooperation in a population. We identify a feedback between the costs and benefits of cooperation and the evolutionary robustness of cooperative strategies. Depending on the functional form (35) of the relationship between costs and benefits, this feedback may either reinforce the evolutionary success of cooperation or else precipitate its collapse. In particular, we show that cooperation will always collapse when there are diminishing returns for mutual cooperation.