Global Convergence for Replicator Dynamics of Repeated Snowdrift Games

To understand the emergence and sustainment of cooperative behavior in interacting collectives, we perform global convergence analysis for replicator dynamics of a large, well-mixed population of individuals playing a repeated snowdrift game with four typical strategies, which are always cooperate (ALLC), tit-for-tat (TFT), suspicious tit-for-tat (STFT), and always defect (ALLD). The dynamical model is a 3-D ordinary differential equation (ODE) system that is parameterized by the payoffs of the base game. Instead of routine searches for evolutionarily stable strategies and sets, we expand our analysis to determining the asymptotic behavior of solution trajectories starting from any initial state, and in particular, show that for the full range of payoffs, every trajectory of the system converges to an equilibrium point. What enables us to achieve such comprehensive results is studying the dynamics of two ratios of the state variables, each of which either monotonically increases or decreases in the half-spaces separated by their corresponding planes. The convergence results highlight two findings. First, the inclusion of TFT- and STFT-players, the two types of conditional strategy players in the game, increases the share of cooperators of the overall population compared to the situation when the population consists of only ALLC and ALLD-players. Second, surprisingly enough, regardless of the payoffs, there always exists a set of initial conditions under which ALLC-players do not vanish in the long run, which does not hold for any of the other three types of players.

[1]  Eitan Altman,et al.  Stochastic State Dependent Population Games in Wireless Communication , 2011, IEEE Transactions on Automatic Control.

[2]  David G. Rand,et al.  Direct reciprocity in structured populations , 2012, Proceedings of the National Academy of Sciences.

[3]  David G. Rand,et al.  Static network structure can stabilize human cooperation , 2014, Proceedings of the National Academy of Sciences.

[4]  Jeff S. Shamma,et al.  Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria , 2005, IEEE Transactions on Automatic Control.

[5]  Jason R. Marden,et al.  The Price of Selfishness in Network Coding , 2009, IEEE Transactions on Information Theory.

[6]  R. Axelrod More Effective Choice in the Prisoner's Dilemma , 1980 .

[7]  R. Trivers The Evolution of Reciprocal Altruism , 1971, The Quarterly Review of Biology.

[8]  Eitan Altman,et al.  Constrained Games: The Impact of the Attitude to Adversary's Constraints , 2009, IEEE Transactions on Automatic Control.

[9]  I. Bomze Lotka-Volterra equation and replicator dynamics: A two-dimensional classification , 1983, Biological Cybernetics.

[10]  Franz J Weissing,et al.  Focus on the success of others leads to selfish behavior , 2015, Proceedings of the National Academy of Sciences.

[11]  C. Hauert,et al.  Synergy and discounting of cooperation in social dilemmas. , 2006, Journal of theoretical biology.

[12]  Martin A Nowak,et al.  Evolution of extortion in Iterated Prisoner’s Dilemma games , 2012, Proceedings of the National Academy of Sciences.

[13]  Miroslav Krstic,et al.  Nash Equilibrium Seeking in Noncooperative Games , 2012, IEEE Transactions on Automatic Control.

[14]  Jason R. Marden State based potential games , 2012, Autom..

[15]  Andres Pantoja,et al.  Building Temperature Control Based on Population Dynamics , 2014, IEEE Transactions on Control Systems Technology.

[16]  S. Wiggins Introduction to Applied Nonlinear Dynamical Systems and Chaos , 1989 .

[17]  Tim Roughgarden,et al.  Generalized Efficiency Bounds in Distributed Resource Allocation , 2014, IEEE Trans. Autom. Control..

[18]  Luc-Alain Giraldeau,et al.  The Forager’s Dilemma: Food Sharing and Food Defense as Risk‐Sensitive Foraging Options , 2003, The American Naturalist.

[19]  Larry Samuelson,et al.  Evolutionary stability and lexicographic preferences , 2003, Games Econ. Behav..

[20]  M. Nowak Evolutionary Dynamics: Exploring the Equations of Life , 2006 .

[21]  Hyeong Soo Chang,et al.  Two-person zero-sum Markov games: receding horizon approach , 2003, IEEE Trans. Autom. Control..

[22]  R. Selten,et al.  Gaps in Harley's argument on evolutionarily stable learning rules and in the logic of “tit for tat” , 1984, Behavioral and Brain Sciences.

[23]  Alessandro Astolfi,et al.  Constructive $\epsilon$-Nash Equilibria for Nonzero-Sum Differential Games , 2015, IEEE Transactions on Automatic Control.

[24]  Ning Jia,et al.  Experiments on individual strategy updating in iterated snowdrift game under random rematching. , 2015, Journal of theoretical biology.

[25]  Ming Cao,et al.  Stability analysis for replicator dynamics of evolutionary snowdrift games , 2014, 53rd IEEE Conference on Decision and Control.

[26]  Julián García,et al.  In and out of equilibrium I: Evolution of strategies in repeated games with discounting , 2007, J. Econ. Theory.

[27]  John S. Baras,et al.  Selfish Response to Epidemic Propagation , 2010, IEEE Transactions on Automatic Control.

[28]  Hildeberto Jardón-Kojakhmetov,et al.  Limit sets within curves where trajectories converge to , 2017, Appl. Math. Lett..

[29]  D. Fudenberg,et al.  Evolutionary cycles of cooperation and defection. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Rachid El Azouzi,et al.  Delayed Evolutionary game dynamics with non-uniform interactions in two communities , 2014, 53rd IEEE Conference on Decision and Control.

[31]  Vivek S. Borkar,et al.  Dynamic Cesaro-Wardrop equilibration in networks , 2003, IEEE Trans. Autom. Control..

[32]  Maria Domenica Di Benedetto,et al.  Randomized sampling for large zero-sum games , 2013, Autom..

[33]  Ming Cao,et al.  How Feeling Betrayed Affects Cooperation , 2015, PloS one.

[34]  Sonia Martínez,et al.  Self-Triggered Best-Response Dynamics for Continuous Games , 2015, IEEE Transactions on Automatic Control.

[35]  Ming Cao,et al.  Asynchronous Decision-Making Dynamics Under Best-Response Update Rule in Finite Heterogeneous Populations , 2018, IEEE Transactions on Automatic Control.

[36]  Ming Cao,et al.  Networks of conforming or nonconforming individuals tend to reach satisfactory decisions , 2016, Proceedings of the National Academy of Sciences.

[37]  Bahman Gharesifard,et al.  Evolution of Players' Misperceptions in Hypergames Under Perfect Observations , 2012, IEEE Transactions on Automatic Control.

[38]  Yu Kang,et al.  Neighborhood Diversity Promotes Cooperation in Social Dilemmas , 2018, IEEE Access.

[39]  Milos S. Stankovic,et al.  Distributed Seeking of Nash Equilibria With Applications to Mobile Sensor Networks , 2012, IEEE Transactions on Automatic Control.

[40]  Guangming Xie,et al.  Modified snowdrift games for multi-robot water polo matches , 2012, 2012 24th Chinese Control and Decision Conference (CCDC).

[41]  O. Diekmann,et al.  On the Cyclic Replicator Equation and the Dynamics of Semelparous Populations , 2009, SIAM J. Appl. Dyn. Syst..

[42]  The social costs of punishment , 2012, Behavioral and Brain Sciences.

[43]  Ming Cao,et al.  Analysis and control of strategic interactions in finite heterogeneous populations under best-response update rule , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[44]  C. Hauert,et al.  The Evolutionary Origin of Cooperators and Defectors , 2004, Science.

[45]  Dario Madeo,et al.  Game Interactions and Dynamics on Networked Populations , 2015, IEEE Transactions on Automatic Control.

[46]  Christos A. Ioannou Asymptotic behavior of strategies in the repeated prisoner's dilemma game in the presence of errors , 2014, Artif. Intell. Res..

[47]  Eitan Altman,et al.  Markov Decision Evolutionary Games , 2010, IEEE Transactions on Automatic Control.

[48]  L. Keller,et al.  Human cooperation in social dilemmas: comparing the Snowdrift game with the Prisoner's Dilemma , 2007, Proceedings of the Royal Society B: Biological Sciences.

[49]  Yuzhen Wang,et al.  Algebraic formulation and strategy optimization for a class of evolutionary networked games via semi-tensor product method , 2013, Autom..

[50]  J. Lorberbaum,et al.  No strategy is evolutionarily stable in the repeated prisoner's dilemma. , 1994, Journal of theoretical biology.

[51]  Bruno Sinopoli,et al.  Detection in Adversarial Environments , 2014, IEEE Transactions on Automatic Control.

[52]  Antonio Cabrales,et al.  Three is a crowd in iterated prisoner's dilemmas: experimental evidence on reciprocal behavior , 2012, Scientific Reports.

[53]  Eitan Altman,et al.  State policy couple dynamics in evolutionary games , 2015, 2015 American Control Conference (ACC).

[54]  Ming Cao,et al.  Evolutionary Dynamics of Homophily and Heterophily , 2016, Scientific Reports.

[55]  Jason R. Marden,et al.  Joint Strategy Fictitious Play with Inertia for Potential Games , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[56]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[57]  Jason R. Marden,et al.  Designing Games for Distributed Optimization , 2013, IEEE J. Sel. Top. Signal Process..

[58]  Jason R. Marden,et al.  Decoupling Coupled Constraints Through Utility Design , 2014, IEEE Transactions on Automatic Control.

[59]  M. Nowak,et al.  The evolution of stochastic strategies in the Prisoner's Dilemma , 1990 .

[60]  William H. Sandholm,et al.  Population Games And Evolutionary Dynamics , 2010, Economic learning and social evolution.

[61]  Alexandre M. Bayen,et al.  Stability of Nash equilibria in the congestion game under Replicator dynamics , 2014, 53rd IEEE Conference on Decision and Control.

[62]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[63]  J. Bendor,et al.  Effective Choice in the Prisoner ' s Dilemma , 2007 .

[64]  Immanuel M. Bomze,et al.  Lotka-Volterra equation and replicator dynamics: new issues in classification , 1995, Biological Cybernetics.

[65]  Mary Lou Zeeman,et al.  From local to global behavior in competitive Lotka-Volterra systems , 2002 .

[66]  V. Jansen,et al.  Altruism through beard chromodynamics , 2006, Nature.

[67]  Michael Doebeli,et al.  Spatial structure often inhibits the evolution of cooperation in the snowdrift game , 2004, Nature.

[68]  D. M. V. Hesteren Evolutionary Game Theory , 2017 .