Optimization Despite Chaos: Convex Relaxations to Complex Limit Sets via Poincaré Recurrence

It is well understood that decentralized systems can, through network interactions, give rise to complex behavior patterns that do not reflect their equilibrium properties. The challenge of any analytic investigation is to identify and characterize persistent properties despite the inherent irregularities of such systems and to do so efficiently. We develop a novel framework to address this challenge. Our setting focuses on evolutionary dynamics in network extensions of zero-sum games. Such dynamics have been shown analytically to exhibit chaotic behavior which traditionally has been thought of as an overwhelming obstacle to algorithmic inquiry. We circumvent these issues as follows: First, we combine ideas from dynamical systems and game theory to produce topological characterizations of system trajectories. Trajectories capture the time evolution of the system given an initial starting state. They are complex, and do not necessarily converge to limit points or even limit cycles. We provide tractable approximations of such limit sets. These relaxed descriptions involve simplices, and can be computed in polynomial time. Next, we apply standard optimization techniques to compute extremal values of system features (e.g. expected utility of an agent) within these relaxations. Finally, we use information theoretic conservation laws along with Poincare recurrence theory to argue about tightness and optimality of our relaxation techniques.

[1]  G. P. Szegö,et al.  Stability theory of dynamical systems , 1970 .

[2]  S. Vajda Some topics in two-person games , 1971 .

[3]  H. I. Freedman,et al.  Mathematical analysis of some three-species food-chain models , 1977 .

[4]  T. Hallam Structural sensitivity of grazing formulations in nutrient controlled plankton models , 1978 .

[5]  T. Gard Persistence in food webs: holling-type food chains , 1980 .

[6]  H. I. Freedman,et al.  Persistence in models of three interacting predator-prey populations , 1984 .

[7]  E. Akin,et al.  Evolutionary dynamics of zero-sum games , 1984, Journal of mathematical biology.

[8]  Eitan Zemel,et al.  Nash and correlated equilibria: Some complexity considerations , 1989 .

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  K. Schmitt,et al.  Permanence and the dynamics of biological systems. , 1992, Mathematical biosciences.

[11]  J. Jordan Three Problems in Learning Mixed-Strategy Nash Equilibria , 1993 .

[12]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[13]  J. Hofbauer,et al.  Fictitious Play, Shapley Polygons and the Replicator Equation , 1995 .

[14]  J. Hofbauer Evolutionary dynamics for bimatrix games: A Hamiltonian system? , 1996, Journal of mathematical biology.

[15]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[16]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[17]  Eizo Akiyama,et al.  Chaos in learning a simple two-person game , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[18]  L. Barreira Poincaré recurrence:. old and new , 2006 .

[19]  James D. Meiss,et al.  Differential dynamical systems , 2007, Mathematical modeling and computation.

[20]  Christos H. Papadimitriou,et al.  On a Network Generalization of the Minmax Theorem , 2009, ICALP.

[21]  C. Shalizi Dynamics of Bayesian Updating with Dependent Data and Misspecified Models , 2009, 0901.1342.

[22]  Josef Hofbauer,et al.  Time Average Replicator and Best-Reply Dynamics , 2009, Math. Oper. Res..

[23]  Éva Tardos,et al.  Multiplicative updates outperform generic no-regret learning in congestion games: extended abstract , 2009, STOC '09.

[24]  G. Karev Replicator Equations and the Principle of Minimal Production of Information , 2009, Bulletin of mathematical biology.

[25]  Christos H. Papadimitriou,et al.  On Learning Algorithms for Nash Equilibria , 2010, SAGT.

[26]  Yang Cai,et al.  On minmax theorems for multiplayer games , 2011, SODA '11.

[27]  Georgios Piliouras,et al.  Beating the best Nash without regret , 2011, SECO.

[28]  Éva Tardos,et al.  Beyond the Nash Equilibrium Barrier , 2011, ICS.

[29]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..