From Population Games to Payoff Dynamics Models: A Passivity-Based Approac

This tutorial article describes a dynamical systems framework rooted in evolutionary game principles to characterize non-cooperative strategic interactions among large populations of bounded rationality agents. It also overviews recent results that use passivity notions to characterize the stability of Nash-like equilibria. In our framework, each agent belongs to a population that prescribes to its members a strategy set and a strategy revision protocol. A so-called social state registers the proportions of agents in every population adopting each strategy and a pre-selected dynamic payoff mechanism, specified by a payoff dynamics model (PDM), determines the payoff as a causal map of the social state. According to the framework, each agent must take up a strategy at a time, which it can repeatedly revise over time based on its current strategy, and information about the payoff and social state available to it. The PDM class considered in our framework can model precisely or approximately prevalent dynamic behaviors such as inertia and delays that are inherent to learning and network effects, which cannot be captured using conventional memoryless payoff mechanisms (often referred to as population games).We organize the article in two main parts. The first introduces basic concepts prevailing in existing approaches in which a population game determines the payoff, while the second considers rather general PDM classes, of which every population game is a particular case. The latter expounds a passivity-based methodology to characterize convergence of the social state to Nash-like equilibria.

[1]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[2]  P. Taylor,et al.  Evolutionarily Stable Strategies and Game Dynamics , 1978 .

[3]  Rida Laraki,et al.  Higher order game dynamics , 2012, J. Econ. Theory.

[4]  Josef Hofbauer,et al.  Stable Games , 2007, 2007 46th IEEE Conference on Decision and Control.

[5]  William H. Sandholm,et al.  Learning in Games via Reinforcement and Regularization , 2014, Math. Oper. Res..

[6]  C. Cannings,et al.  The Generalized War of Attrition , 1997 .

[7]  Lacra Pavel,et al.  A Passivity-Based Approach to Nash Equilibrium Seeking Over Networks , 2017, IEEE Transactions on Automatic Control.

[8]  Jeff S. Shamma,et al.  Passivity and Evolutionary Game Dynamics , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[9]  T. L. Vincent,et al.  Evolution and control system design. The evolutionary game , 2000 .

[10]  Eitan Altman,et al.  Evolutionary Games in Wireless Networks , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  M. K. Jensen Aggregative games and best-reply potentials , 2010 .

[12]  William H. Sandholm,et al.  Excess payoff dynamics and other well-behaved evolutionary dynamics , 2005, J. Econ. Theory.

[13]  Josef Hofbauer,et al.  Stable games and their dynamics , 2009, J. Econ. Theory.

[14]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[15]  Josef Hofbauer,et al.  Evolution in games with randomly disturbed payoffs , 2007, J. Econ. Theory.

[16]  Lacra Pavel,et al.  On Passivity and Reinforcement Learning in Finite Games , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[17]  William H. Sandholm,et al.  Pairwise Comparison Dynamics and Evolutionary Foundations for Nash Equilibrium , 2009, Games.

[18]  Jeff S. Shamma,et al.  Distributed Dynamic Reinforcement of Efficient Outcomes in Multiagent Coordination and Network Formation , 2011, Dynamic Games and Applications.

[19]  Dirk Brockmann Networks: An Introduction; Networks, Crowds, and Markets: Reasoning About a Highly Connected World , 2011 .

[20]  William H. Sandholm,et al.  ON THE GLOBAL CONVERGENCE OF STOCHASTIC FICTITIOUS PLAY , 2002 .

[21]  William H. Sandholm,et al.  Evolution and equilibrium under inexact information , 2003, Games Econ. Behav..

[22]  T. Kurtz Solutions of ordinary differential equations as limits of pure jump markov processes , 1970, Journal of Applied Probability.

[23]  J. Hofbauer From Nash and Brown to Maynard Smith: Equilibria, Dynamics and ESS , 2001 .

[24]  William H. Sandholm,et al.  Potential Games with Continuous Player Sets , 2001, J. Econ. Theory.

[25]  T. Koopmans,et al.  Studies in the Economics of Transportation. , 1956 .

[26]  J. Willems Dissipative dynamical systems part I: General theory , 1972 .

[27]  P. Lions,et al.  Mean field games , 2007 .

[28]  Jörgen W. Weibull,et al.  The Mass-Action Interpretation of Nash Equilibrium , 1995 .

[29]  P. Moylan Implications of passivity in a class of nonlinear systems , 1974 .

[30]  L. Shapley,et al.  Potential Games , 1994 .

[31]  M. Benaïm Dynamics of stochastic approximation algorithms , 1999 .

[32]  W. Arthur,et al.  The Economy as an Evolving Complex System II , 1988 .

[33]  Michael J. Smith,et al.  The Stability of a Dynamic Model of Traffic Assignment - An Application of a Method of Lyapunov , 1984, Transp. Sci..

[34]  G. Arslan,et al.  Anticipatory Learning in General Evolutionary Games , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[35]  David M. Kreps,et al.  Learning Mixed Equilibria , 1993 .

[36]  J. Neumann,et al.  SOLUTIONS OF GAMES BY DIFFERENTIAL EQUATIONS , 1950 .

[37]  George H. Hines,et al.  Equilibrium-independent passivity: A new definition and numerical certification , 2011, Autom..

[38]  William H. Sandholm,et al.  Population Games And Evolutionary Dynamics , 2010, Economic learning and social evolution.

[39]  Jeff S. Shamma,et al.  Payoff Dynamics Model and Evolutionary Dynamics Model: Feedback and Convergence to Equilibria , 2019, 1903.02018.

[40]  Nuno C. Martins,et al.  Information structures in optimal decentralized control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[41]  Peter E. Caines,et al.  The NCE (Mean Field) Principle With Locality Dependent Cost Interactions , 2010, IEEE Transactions on Automatic Control.

[42]  Chris Arney,et al.  Networks, Crowds, and Markets: Reasoning about a Highly Connected World (Easley, D. and Kleinberg, J.; 2010) [Book Review] , 2013, IEEE Technology and Society Magazine.

[43]  P. Moylan,et al.  The stability of nonlinear dissipative systems , 1976 .

[44]  William H. Sandholm,et al.  Local stability of strict equilibria under evolutionary game dynamics , 2014 .

[45]  Jeff S. Shamma,et al.  Population games, stable games, and passivity , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[46]  M. Feldman,et al.  Local dispersal promotes biodiversity in a real-life game of rock–paper–scissors , 2002, Nature.

[47]  Nicanor Quijano,et al.  The Role of Population Games and Evolutionary Dynamics in Distributed Control Systems: The Advantages of Evolutionary Game Theory , 2017, IEEE Control Systems.

[48]  J. G. Wardrop,et al.  Some Theoretical Aspects of Road Traffic Research , 1952 .

[49]  M. Benaïm,et al.  Deterministic Approximation of Stochastic Evolution in Games , 2003 .