Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle

Abstract. We consider stochastic dynamic games in large population conditions where multiclass agents are weakly coupled via their individual dynamics and costs. We approach this large population game problem by the so-called Nash Certainty Equivalence (NCE) Principle which leads to a decentralized control synthesis. The McKean-Vlasov NCE method presented in this paper has a close connection with the statistical physics of large particle systems: both identify a consistency relationship between the individual agent (or particle) at the microscopic level and the mass of individuals (or particles) at the macroscopic level. The overall game is decomposed into (i) an optimal control problem whose Hamilton-Jacobi-Bellman (HJB) equation determines the optimal control for each individual and which involves a measure corresponding to the mass effect, and (ii) a family of McKean-Vlasov (M-V) equations which also depend upon this measure. We designate the NCE Principle as the property that the resulting scheme is consistent (or soluble), i.e. the prescribed control laws produce sample paths which produce the mass effect measure. By construction, the overall closed-loop behaviour is such that each agent’s behaviour is optimal with respect to all other agents in the game theoretic Nash sense.

[1]  J. G. Wardrop,et al.  Some Theoretical Aspects of Road Traffic Research , 1952 .

[2]  J. Gillis,et al.  Probability and Related Topics in Physical Sciences , 1960 .

[3]  H. McKean,et al.  A CLASS OF MARKOV PROCESSES ASSOCIATED WITH NONLINEAR PARABOLIC EQUATIONS , 1966, Proceedings of the National Academy of Sciences of the United States of America.

[4]  M. Hp.  A class of markov processes associated with nonlinear parabolic equations. , 1966 .

[5]  R. Dobrushin.  Prescribing a System of Random Variables by Conditional Distributions , 1970 .

[6]  D. Haar,et al.  Statistical Physics , 1971, Nature.

[7]  L. Shapley,et al.  Values of Non-Atomic Games , 1974 .

[8]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[9]  R. Aumann.  Values of Markets with a Continuum of Traders , 1975 .

[10]  P. Varaiya.  N-player stochastic differential games , 1976 .

[11]  D. E. Matthews.  Evolution and the Theory of Games , 1977 .

[12]  N. Z. Shapiro,et al.  Values of Large Games, I: A Limit Theorem , 1978, Math. Oper. Res..

[13]  L. S. Shapley,et al.  Values of Large Games II: Oceanic Games , 1978, Math. Oper. Res..

[14]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[15]  G. Papavassilopoulos.  On the linear-quadratic-Gaussian Nash game with one-step delay observation sharing pattern , 1982 .

[16]  D. Dawson.  Critical dynamics and fluctuations for a mean-field model of cooperative behavior , 1983 .

[17]  E. Green.  Continuum and Finite-Player Noncooperative Models of Competition , 1984 .

[18]  Val E. Lambson.  Self-enforcing collusion in large dynamic markets , 1984 .

[19]  Robert J. Weber,et al.  Distributional Strategies for Games with Incomplete Information , 1985, Math. Oper. Res..

[20]  Alain Haurie,et al.  On the relationship between Nash - Cournot and Wardrop equilibria , 1983, Networks.

[21]  J. Szep,et al.  Games with incomplete information , 1985 .

[22]  J. Friedman.  Game theory with applications to economics , 1986 .

[23]  J. Gärtner,et al.  Large deviations from the mckean-vlasov limit for weakly interacting diffusions , 1987 .

[24]  D. Chandler,et al.  Introduction To Modern Statistical Mechanics , 1987 .

[25]  A. Sznitman.  Topics in propagation of chaos , 1991 .

[26]  L. Daemen,et al.  Equilibrium and nonequilibrium statistical mechanics of a nonlinear model of DNA , 1991 .

[27]  A. Bensoussan.  Stochastic Control of Partially Observable Systems , 1992 .

[28]  G. Erickson.  Differential game models of advertising competition , 1995 .

[29]  D. Talay,et al.  Nonlinear self-stabilizing processes – I Existence, invariant probability, propagation of chaos , 1998 .

[30]  S. Benachour,et al.  Nonlinear self-stabilizing processes – II: Convergence to invariant probability , 1998 .

[31]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[32]  Edmund J. Collins,et al.  Optimality Models in Behavioral Biology , 2001, SIAM Rev..

[33]  Zhen Liu,et al.  Fixed Point Methods for the Simulation of the Sharing of a Local Loop by a Large Number of Interacting TCP Connections , 2001 .

[34]  M. A. Khan,et al.  Non-Cooperative Games with Many Players , 2002 .

[35]  P. Caines,et al.  Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[36]  E. Balder.  An equilibrium existence result for games with incomplete information and indeterminate outcomes , 2004 .

[37]  John C. Harsanyi,et al.  Games with Incomplete Information Played by "Bayesian" Players, I-III: Part I. The Basic Model& , 2004, Manag. Sci..

[38]  P. Caines,et al.  Large-population cost-coupled LQG problems: generalizations to non-uniform individuals , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[39]  Peter E. Caines,et al.  Uplink power adjustment in wireless communication systems: a stochastic control analysis , 2004, IEEE Transactions on Automatic Control.

[40]  P. Caines,et al.  Nash Equilibria for Large-Population Linear Stochastic Systems of Weakly Coupled Agents , 2005 .

[41]  D. Morale,et al.  An interacting particle system modelling aggregation behavior: from individuals to populations , 2005, Journal of mathematical biology.

[42]  P. Varaiya,et al.  Differential games , 1971 .

[43]  Minyi Huang,et al.  Nash Strategies and Adaptation for Decentralized Games Involving Weakly-coupled Agents , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[44]  Minyi Huang,et al.  Nash Certainty Equivalence in Large Population Stochastic Dynamic Games: Connections with the Physics of Interacting Particle Systems , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[45]  Peter E. Caines,et al.  Distributed Multi-Agent Decision-Making with Partial Observations: Asymtotic Nash Equilibria , 2006 .

[46]  Minyi Huang,et al.  Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria , 2007, IEEE Transactions on Automatic Control.