Convergence analysis and control of evolutionary matrix-game dynamics

Networks of decision-making individuals with simple dynamics may give rise to complex and seemingly unpredictable collective behaviors which may have negative consequences such as traffic jams and market crashes or positive outcomes such as volunteer disaster relief and free-market stabilization. Usually on one hand, there is a group task requiring the individuals to cooperate to optimize the collective performances, and on the other, each individual is self-interested and may prefer to prioritize her own interest over the group task, resulting in a social dilemma. Typical research questions in this topic therefore focus on (i) how to predict the long run behavior of the networks and find factors causing non-converging fluctuations in the actions of the individuals; (ii) how to build a model to describe the dynamics as cooperation evolves or is promoted among selfish individuals; and (iii) how to control the number of individuals taking a particular action in a network. We tackle these problems by modeling networks of decision-making individuals using evolutionary matrix-game dynamics, performing convergence analysis to understand the asymptotic behavior of the dynamics, and designing control algorithms to lead the individuals to a desired action.

[1]  George P. Papavassilopoulos,et al.  Games on Large Networks: Information and Complexity , 2017, IEEE Transactions on Automatic Control.

[2]  Ming Cao,et al.  Towards Optimal Control of Evolutionary Games on Networks , 2017, IEEE Transactions on Automatic Control.

[3]  A. Wilhite Network structure, games, and agent dynamics , 2014 .

[4]  David G. Rand,et al.  Social heuristics shape intuitive cooperation , 2014, Nature Communications.

[5]  Thomas R. Shultz,et al.  Stages in the evolution of ethnocentrism , 2008 .

[6]  Christos G. Cassandras,et al.  Introduction to Discrete Event Systems , 1999, The Kluwer International Series on Discrete Event Dynamic Systems.

[7]  Arne Traulsen,et al.  Exploration dynamics in evolutionary games , 2009, Proceedings of the National Academy of Sciences.

[8]  C. Watkins,et al.  The spread of awareness and its impact on epidemic outbreaks , 2009, Proceedings of the National Academy of Sciences.

[9]  Dario Madeo,et al.  Game Interactions and Dynamics on Networked Populations , 2015, IEEE Transactions on Automatic Control.

[10]  Ross A. Hammond,et al.  The Evolution of Ethnocentrism , 2006 .

[11]  Shie Mannor,et al.  Network Formation: Bilateral Contracting and Myopic Dynamics , 2007, IEEE Transactions on Automatic Control.

[12]  Munther A. Dahleh,et al.  On the behavior of threshold models over finite networks , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[13]  H. Ohtsuki,et al.  A simple rule for the evolution of cooperation on graphs and social networks , 2006, Nature.

[14]  Nikolitsa Markou,et al.  Emergence of cooperation in static networks : evolutionary dynamics of coordination games with switching costs , 2013 .

[15]  Jeff S. Shamma,et al.  Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria , 2005, IEEE Transactions on Automatic Control.

[16]  S. Sastry Nonlinear Systems: Analysis, Stability, and Control , 1999 .

[17]  Mary Lou Zeeman,et al.  From local to global behavior in competitive Lotka-Volterra systems , 2002 .

[18]  Martin A. Nowak,et al.  The Evolution of Homophily , 2012, Scientific Reports.

[19]  V. Jansen,et al.  Altruism through beard chromodynamics , 2006, Nature.

[20]  Michael Doebeli,et al.  Spatial structure often inhibits the evolution of cooperation in the snowdrift game , 2004, Nature.

[21]  David G. Rand,et al.  Direct reciprocity in structured populations , 2012, Proceedings of the National Academy of Sciences.

[22]  K. Wärneryd Cheap Talk, Coordination, and Evolutionary Stability , 1993 .

[23]  David G. Rand,et al.  Static network structure can stabilize human cooperation , 2014, Proceedings of the National Academy of Sciences.

[24]  Hyeong Soo Chang,et al.  Two-person zero-sum Markov games: receding horizon approach , 2003, IEEE Trans. Autom. Control..

[25]  Munther A. Dahleh,et al.  On Threshold Models over Finite Networks , 2012, ArXiv.

[26]  Carlos Alós-Ferrer Finite Population Dynamics and Mixed Equilibria , 2003, IGTR.

[27]  Daniela Fischer Differential Equations Dynamical Systems And An Introduction To Chaos , 2016 .

[28]  Christos A. Ioannou Asymptotic behavior of strategies in the repeated prisoner's dilemma game in the presence of errors , 2014, Artif. Intell. Res..

[29]  Eitan Altman,et al.  Markov Decision Evolutionary Games , 2010, IEEE Transactions on Automatic Control.

[30]  H. Peyton Young,et al.  Fast convergence in evolutionary equilibrium selection , 2013, Games Econ. Behav..

[31]  Tamer Basar,et al.  Game-Theoretic Analysis of the Hegselmann-Krause Model for Opinion Dynamics in Finite Dimensions , 2014, IEEE Transactions on Automatic Control.

[32]  Naoki Masuda,et al.  Computational model of collective nest selection by ants with heterogeneous acceptance thresholds , 2015, Royal Society Open Science.

[33]  Dunia López-Pintado,et al.  Contagion and coordination in random networks , 2006, Int. J. Game Theory.

[34]  Gerhard Jäger,et al.  Voronoi languages: Equilibria in cheap-talk games with high-dimensional types and few signals , 2011, Games Econ. Behav..

[35]  L. Keller,et al.  Human cooperation in social dilemmas: comparing the Snowdrift game with the Prisoner's Dilemma , 2007, Proceedings of the Royal Society B: Biological Sciences.

[36]  Jason R. Marden,et al.  The Price of Selfishness in Network Coding , 2009, IEEE Transactions on Information Theory.

[37]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[38]  Alex Arenas,et al.  Traffic-driven epidemic spreading in finite-size scale-free networks , 2009, Proceedings of the National Academy of Sciences.

[39]  D. Helbing,et al.  The future of social experimenting , 2010, Proceedings of the National Academy of Sciences.

[40]  B. Skyrms,et al.  In the light of evolution VIII: Darwinian thinking in the social sciences , 2014, Proceedings of the National Academy of Sciences.

[41]  Ming Cao,et al.  Networks of conforming or nonconforming individuals tend to reach satisfactory decisions , 2016, Proceedings of the National Academy of Sciences.

[42]  N. G. Parke,et al.  Ordinary Differential Equations. , 1958 .

[43]  Mark S. Granovetter Threshold Models of Collective Behavior , 1978, American Journal of Sociology.

[44]  Giulio Cimini,et al.  Dynamics to Equilibrium in Network Games: Individual Behavior and Global Response , 2014, PloS one.

[45]  Tim Roughgarden,et al.  Generalized Efficiency Bounds in Distributed Resource Allocation , 2010, IEEE Transactions on Automatic Control.

[46]  Thomas R. Shultz,et al.  The Evolutionary Dominance of Ethnocentric Cooperation , 2013, J. Artif. Soc. Soc. Simul..

[47]  William H. Sandholm,et al.  Sampling Best Response Dynamics and Deterministic Equilibrium Selection , 2014 .

[48]  Daizhan Cheng,et al.  Modeling, Analysis and Control of Networked Evolutionary Games , 2015, IEEE Transactions on Automatic Control.

[49]  R. Trivers The Evolution of Reciprocal Altruism , 1971, The Quarterly Review of Biology.

[50]  Eitan Altman,et al.  Constrained Games: The Impact of the Attitude to Adversary's Constraints , 2009, IEEE Transactions on Automatic Control.

[51]  I. Bomze Lotka-Volterra equation and replicator dynamics: A two-dimensional classification , 1983, Biological Cybernetics.

[52]  Arne Traulsen,et al.  Human strategy updating in evolutionary games , 2010, Proceedings of the National Academy of Sciences.

[53]  A. Andronov,et al.  Qualitative Theory of Second-order Dynamic Systems , 1973 .

[54]  M. Nowak,et al.  The evolution of stochastic strategies in the Prisoner's Dilemma , 1990 .

[55]  Jorge Cortes,et al.  Distributed Control of Robotic Networks: A Mathematical Approach to Motion Coordination Algorithms , 2009 .

[56]  Franz J Weissing,et al.  Focus on the success of others leads to selfish behavior , 2015, Proceedings of the National Academy of Sciences.

[57]  Abhijit Banerjee,et al.  Neutrally Stable Outcomes in Cheap-Talk Coordination Games , 2000, Games Econ. Behav..

[58]  Marc Lelarge Diffusion and cascading behavior in random networks , 2012, Games Econ. Behav..

[59]  Vincent A. A. Jansen,et al.  Common language or Tower of Babel? On the evolutionary dynamics of signals and their meanings , 2003, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[60]  Martin A Nowak,et al.  Evolution of extortion in Iterated Prisoner’s Dilemma games , 2012, Proceedings of the National Academy of Sciences.

[61]  S. Wiggins Introduction to Applied Nonlinear Dynamical Systems and Chaos , 1989 .

[62]  Krzysztof Ciesielski,et al.  The Poincaré-Bendixson Theorem: from Poincaré to the XXIst century , 2012 .

[63]  D. Tracer,et al.  Social image concerns promote cooperation more than altruistic punishment , 2016, Nature Communications.

[64]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[65]  Antonio Cabrales,et al.  Three is a crowd in iterated prisoner's dilemmas: experimental evidence on reciprocal behavior , 2012, Scientific Reports.

[66]  Luc-Alain Giraldeau,et al.  The Forager’s Dilemma: Food Sharing and Food Defense as Risk‐Sensitive Foraging Options , 2003, The American Naturalist.

[67]  Andrea Montanari,et al.  The spread of innovations in social networks , 2010, Proceedings of the National Academy of Sciences.

[68]  David Carfi,et al.  A Coopetitive Approach to Financial Markets Stabilization and Risk Management , 2012, IPMU.

[69]  Xiao Fan Wang,et al.  Analysis and shifting of stochastically stable equilibria for evolutionary snowdrift games , 2015, Syst. Control. Lett..

[70]  William H. Sandholm,et al.  An Introduction to Dynamo: Diagrams for Evolutionary Game Dynamics , 2013 .

[71]  Yuzhen Wang,et al.  Algebraic formulation and strategy optimization for a class of evolutionary networked games via semi-tensor product method , 2013, Autom..

[72]  Maria Domenica Di Benedetto,et al.  Randomized sampling for large zero-sum games , 2010, 49th IEEE Conference on Decision and Control (CDC).

[73]  G. P. Szegö,et al.  Stability theory of dynamical systems , 1970 .

[74]  J. Lorberbaum,et al.  No strategy is evolutionarily stable in the repeated prisoner's dilemma. , 1994, Journal of theoretical biology.

[75]  T. Shultz,et al.  Why Is Ethnocentrism More Common Than Humanitarianism , 2009 .

[76]  Bruno Sinopoli,et al.  Detection in Adversarial Environments , 2014, IEEE Transactions on Automatic Control.

[77]  G. Szabó,et al.  Evolutionary games on graphs , 2006, cond-mat/0607344.

[78]  J. Sobel,et al.  AN EVOLUTIONARY APPROACH TO PRE-PLAY COMMUNICATION , 1995 .

[79]  Joseph Farrell Cheap Talk, Coordination, and Entry , 1987 .

[80]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[81]  Martin A Nowak,et al.  Evolutionary dynamics in set structured populations , 2009, Proceedings of the National Academy of Sciences.

[82]  Hassan K. Khalil,et al.  Singular perturbation methods in control : analysis and design , 1986 .

[83]  M. Nowak Evolutionary Dynamics: Exploring the Equations of Life , 2006 .

[84]  Franz J Weissing,et al.  Consistent individual differences in human social learning strategies , 2014, Nature Communications.

[85]  Arne Traulsen,et al.  Emergence of stable polymorphisms driven by evolutionary games between mutants , 2012, Nature Communications.

[86]  P. Holmes,et al.  Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields , 1983, Applied Mathematical Sciences.

[87]  Nick Netzer,et al.  The logit-response dynamics , 2010, Games Econ. Behav..

[88]  L. Shapley,et al.  Potential Games , 1994 .

[89]  M. Nowak,et al.  Cooperate without looking: Why we care what people think and not just what they do , 2015, Proceedings of the National Academy of Sciences.

[90]  Ulf Toelch,et al.  Individual consistency and flexibility in human social information use , 2014, Proceedings of the Royal Society B: Biological Sciences.

[91]  Jaume Llibre,et al.  Qualitative Theory of Planar Differential Systems , 2006 .

[92]  M. Nowak,et al.  Evolutionary games and spatial chaos , 1992, Nature.

[93]  Geoffrey A. Hollinger,et al.  Search and pursuit-evasion in mobile robotics , 2011, Auton. Robots.

[94]  Ming Cao,et al.  Towards control of evolutionary games on networks , 2014, 53rd IEEE Conference on Decision and Control.

[95]  W. Press,et al.  Iterated Prisoner’s Dilemma contains strategies that dominate any evolutionary opponent , 2012, Proceedings of the National Academy of Sciences.

[96]  Luís F. C. Alberto,et al.  Characterization of Stability Region for General Autonomous Nonlinear Dynamical Systems , 2012, IEEE Transactions on Automatic Control.

[97]  William H. Sandholm,et al.  Population Games And Evolutionary Dynamics , 2010, Economic learning and social evolution.

[98]  Alexandre M. Bayen,et al.  Stability of Nash equilibria in the congestion game under Replicator dynamics , 2014, 53rd IEEE Conference on Decision and Control.

[99]  J. Bendor,et al.  Effective Choice in the Prisoner ' s Dilemma , 2007 .

[100]  Xiaohua Xia,et al.  Evolutionary game theoretic demand-side management and control for a class of networked smart grid , 2016, Autom..

[101]  Sonia Martínez,et al.  Self-Triggered Best-Response Dynamics for Continuous Games , 2015, IEEE Transactions on Automatic Control.

[102]  Kirill S Korolev,et al.  Public good diffusion limits microbial mutualism. , 2015, Physical review letters.

[103]  S. Strogatz Exploring complex networks , 2001, Nature.

[104]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[105]  John M McNamara,et al.  Reputation can enhance or suppress cooperation through positive feedback , 2015, Nature Communications.

[106]  Bahman Gharesifard,et al.  Evolution of Players' Misperceptions in Hypergames Under Perfect Observations , 2012, IEEE Transactions on Automatic Control.

[107]  Ning Jia,et al.  Experiments on individual strategy updating in iterated snowdrift game under random rematching. , 2015, Journal of theoretical biology.

[108]  M. Spong,et al.  Integral manifolds of singularly perturbed systems with application to rigid-link flexible-joint multibody systems , 2000 .

[109]  John S. Baras,et al.  Selfish Response to Epidemic Propagation , 2010, IEEE Transactions on Automatic Control.

[110]  K. Schlag Why Imitate, and If So, How?, : A Boundedly Rational Approach to Multi-armed Bandits , 1998 .

[111]  D. Fudenberg,et al.  Evolutionary cycles of cooperation and defection. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[112]  Rachid El Azouzi,et al.  Delayed Evolutionary game dynamics with non-uniform interactions in two communities , 2014, 53rd IEEE Conference on Decision and Control.

[113]  Vivek S. Borkar,et al.  Dynamic Cesaro-Wardrop equilibration in networks , 2003, IEEE Trans. Autom. Control..

[114]  Ura Taro,et al.  Sur le courant exterieur a une region invariante; Theoreme de Bendixson , 1960 .

[115]  Thomas R. Shultz,et al.  Ethnocentrism Maintains Cooperation, but Keeping One's Children Close Fuels It , 2011, CogSci.

[116]  Miroslav Krstic,et al.  Nash Equilibrium Seeking in Noncooperative Games , 2012, IEEE Transactions on Automatic Control.

[117]  Jason R. Marden State based potential games , 2012, Autom..

[118]  R. Axelrod More Effective Choice in the Prisoner's Dilemma , 1980 .

[119]  Sanjay P. Bhat,et al.  Topological properties of asymptotically stable sets , 2010 .

[120]  J. Palis,et al.  Geometric theory of dynamical systems : an introduction , 1984 .

[121]  H. Peyton Young,et al.  Rapid innovation diffusion in social networks , 2014, Proceedings of the National Academy of Sciences.

[122]  Martin A. Nowak,et al.  Evolution of cooperation by phenotypic similarity , 2008, Proceedings of the National Academy of Sciences.

[123]  Akihiko Yanase,et al.  Dynamic Games of Environmental Policy in a Global Economy: Taxes versus Quotas , 2007 .

[124]  L. Perko Global families of limit cycles of planar analytic systems , 1990 .

[125]  Jason R. Marden,et al.  Designing games for distributed optimization , 2011, IEEE Conference on Decision and Control and European Control Conference.

[126]  Daniel I. S. Rosenbloom,et al.  Imitation dynamics of vaccination behaviour on social networks , 2011, Proceedings of the Royal Society B: Biological Sciences.

[127]  Daichi Kimura,et al.  Coevolutionary networks with homophily and heterophily. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[128]  Nikolaus Correll,et al.  Multirobot inspection of industrial machinery , 2009 .

[129]  Immanuel M. Bomze,et al.  Lotka-Volterra equation and replicator dynamics: new issues in classification , 1995, Biological Cybernetics.

[130]  J. W. Alexander,et al.  A proof and extension of the Jordan-Brouwer separation theorem , 1922 .

[131]  Steve Chien,et al.  Convergence to approximate Nash equilibria in congestion games , 2007, SODA '07.

[132]  S. Ruan,et al.  Global dynamics and bifurcations in a four-dimensional replicator system , 2012 .

[133]  Josef Hofbauer,et al.  On the occurrence of limit cycles in the Volterra-Lotka equation , 1981 .

[134]  William H. Sandholm,et al.  Almost global convergence to p-dominant equilibrium , 2001, Int. J. Game Theory.

[135]  Eitan Altman,et al.  Stochastic State Dependent Population Games in Wireless Communication , 2011, IEEE Transactions on Automatic Control.

[136]  M. Jackson,et al.  Diffusion of Behavior and Equilibrium Properties in Network Games , 2007 .

[137]  M. Spong Modeling and Control of Elastic Joint Robots , 1987 .

[138]  George J. Pappas,et al.  Analysis and Control of Epidemics: A Survey of Spreading Processes on Complex Networks , 2015, IEEE Control Systems.

[139]  Fredrik Jansson,et al.  Pitfalls in Spatial Modelling of Ethnocentrism: A Simulation Analysis of the Model of Hammond and Axelrod , 2013, J. Artif. Soc. Soc. Simul..

[140]  Erwin Frey,et al.  Evolutionary game theory in growing populations. , 2010, Physical review letters.

[141]  K. Kaski,et al.  Spatial snowdrift game with myopic agents , 2005 .

[142]  B Skyrms,et al.  A dynamic model of social network formation. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[143]  J. Scherpen,et al.  Model reduction of a flexible-joint robot: a port-Hamiltonian approach , 2016 .

[144]  Arne Traulsen,et al.  Mixing times in evolutionary game dynamics. , 2012, Physical review letters.

[145]  Ross A Hammond,et al.  ALTRUISM VIA KIN‐SELECTION STRATEGIES THAT RELY ON ARBITRARY TAGS WITH WHICH THEY COEVOLVE , 2004, Evolution; international journal of organic evolution.

[146]  U. Dieckmann,et al.  Resource heterogeneity can facilitate cooperation , 2013, Nature Communications.

[147]  R J Requejo,et al.  Coexistence of cooperators and defectors in well mixed populations mediated by limiting resources. , 2011, Physical review letters.

[148]  Ángel Sánchez,et al.  Promotion of cooperation on networks? The myopic best response case , 2009, ArXiv.

[149]  Eitan Altman,et al.  State policy couple dynamics in evolutionary games , 2015, 2015 American Control Conference (ACC).

[150]  M. Nowak,et al.  The evolution of eusociality , 2010, Nature.

[151]  Ming Cao,et al.  Evolutionary Dynamics of Homophily and Heterophily , 2016, Scientific Reports.

[152]  Jason R. Marden,et al.  Joint Strategy Fictitious Play with Inertia for Potential Games , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[153]  F. C. Santos,et al.  Scale-free networks provide a unifying framework for the emergence of cooperation. , 2005, Physical review letters.

[154]  Jason R. Marden,et al.  Decoupling Coupled Constraints Through Utility Design , 2014, IEEE Transactions on Automatic Control.

[155]  James D. Meiss,et al.  Differential dynamical systems , 2007, Mathematical modeling and computation.

[156]  Heinrich H. Nax,et al.  A Behavioral Study of 'Noise' in Coordination Games , 2015, J. Econ. Theory.

[157]  H. Ohtsuki,et al.  The replicator equation on graphs. , 2006, Journal of theoretical biology.

[158]  Milos S. Stankovic,et al.  Distributed Seeking of Nash Equilibria With Applications to Mobile Sensor Networks , 2012, IEEE Transactions on Automatic Control.

[159]  Guangming Xie,et al.  Modified snowdrift games for multi-robot water polo matches , 2012, 2012 24th Chinese Control and Decision Conference (CCDC).

[160]  O. Diekmann,et al.  On the Cyclic Replicator Equation and the Dynamics of Semelparous Populations , 2009, SIAM J. Appl. Dyn. Syst..

[161]  The social costs of punishment , 2012, Behavioral and Brain Sciences.

[162]  David G. Rand,et al.  Humans Display a 'Cooperative Phenotype' that is Domain General and Temporally Stable , 2014 .

[163]  Ming Cao,et al.  Analysis and control of strategic interactions in finite heterogeneous populations under best-response update rule , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[164]  I. Bendixson Sur les courbes définies par des équations différentielles , 1901 .

[165]  Ming Cao,et al.  Stability analysis for replicator dynamics of evolutionary snowdrift games , 2014, 53rd IEEE Conference on Decision and Control.

[166]  M. Nowak,et al.  Evolutionary dynamics in structured populations , 2010, Philosophical Transactions of the Royal Society B: Biological Sciences.

[167]  H. Peyton Young,et al.  The dynamics of social innovation , 2011, Proceedings of the National Academy of Sciences.

[168]  Carlos Alós-Ferrer,et al.  Best-Response Dynamics in a Birth-Death Model of Evolution in Games , 2010, IGTR.

[169]  Jiro Egawa A Remark on the Flow near a Compact Invariant Set , 1973 .

[170]  L. Perko Differential Equations and Dynamical Systems , 1991 .

[171]  Ming Cao,et al.  How Feeling Betrayed Affects Cooperation , 2015, PloS one.