Persistent patterns: multi-agent learning beyond equilibrium and utility

We propose an analytic framework for multi-agent learning that, unlike standard approaches, is not connected to convergence to an equilibrium concept nor to payoff guarantees for the agents. We view multi-agent systems as reservoirs that allow for the long term survival of rich spatiotemporal correlations (i.e., patterns) amongst the agents' behaviors. Our aim is to develop abstractions that allow us to capture details about the possible limit behaviors of such systems. Our approach is based on the contrast between weakly and strongly persistent properties. Informally, a property is weakly persistent if for each starting point there exist limit points that satisfy it. A property is strongly persistent if it is satisfied by all limit points. In the case of non-converging dynamics the set of weakly persistent properties can be significantly richer than that of the strongly persistent properties reflecting topological properties of the system limit sets in a concise and algorithmically tractable manner.

[1]  Shie Mannor,et al.  Multi-agent learning for engineers , 2007, Artif. Intell..

[2]  P. Taylor,et al.  Evolutionarily Stable Strategies and Game Dynamics , 1978 .

[3]  K. Schmitt,et al.  Permanence and the dynamics of biological systems. , 1992, Mathematical biosciences.

[4]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[5]  James D. Meiss,et al.  Differential dynamical systems , 2007, Mathematical modeling and computation.

[6]  Éva Tardos,et al.  Beyond the Nash Equilibrium Barrier , 2011, ICS.

[7]  L. Barreira.  Poincaré recurrence:. old and new , 2006 .

[8]  G. Teschl.  Ordinary Differential Equations and Dynamical Systems , 2012 .

[9]  Leslie Lamport,et al.  Proving the Correctness of Multiprocess Programs , 1977, IEEE Transactions on Software Engineering.

[10]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[11]  Jean-Pierre Bourguignon,et al.  Mathematische Annalen , 1893 .

[12]  Jeff S. Shamma,et al.  Optimization Despite Chaos: Convex Relaxations to Complex Limit Sets via Poincaré Recurrence , 2014, SODA.

[13]  Bowen Alpern,et al.  Recognizing safety and liveness , 2005, Distributed Computing.

[14]  I. Bendixson.  Sur les courbes définies par des équations différentielles , 1901 .

[15]  Peter Stone,et al.  Multiagent learning is not the answer. It is the question , 2007, Artif. Intell..

[16]  J. Neumann.  Zur Theorie der Gesellschaftsspiele , 1928 .

[17]  J. Robinson.  AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[18]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  Yang Cai,et al.  On minmax theorems for multiplayer games , 2011, SODA '11.

[21]  Michael P. Wellman,et al.  Foundations of multi-agent learning: Introduction to the special issue , 2007, Artif. Intell..

[22]  Yoav Shoham,et al.  If multi-agent learning is the answer, what is the question? , 2007, Artif. Intell..

[23]  J. Hofbauer.  Evolutionary dynamics for bimatrix games: A Hamiltonian system? , 1996, Journal of mathematical biology.

[24]  Éva Tardos,et al.  Multiplicative updates outperform generic no-regret learning in congestion games: extended abstract , 2009, STOC '09.

[25]  G. P. Szegö,et al.  Stability theory of dynamical systems , 1970 .

[26]  William H. Sandholm,et al.  Population Games And Evolutionary Dynamics , 2010, Economic learning and social evolution.

[27]  E. Akin,et al.  Evolutionary dynamics of zero-sum games , 1984, Journal of mathematical biology.

[28]  William H. Sandholm,et al.  An Introduction to Dynamo: Diagrams for Evolutionary Game Dynamics , 2013 .