Mean Field Stochastic Control in Radial Loss Networks: A Paradigm for Tractable Distributed Network Admission Control

The computational intractability of the dynamic programming (DP) equations associated with optimal admission and routing in stochastic loss networks of any non-trivial size (Ma et al., 2006, 2008) leads one to consider suboptimal decentralized game theoretic formulations of the problem. The special class of radial networks with a central core of infinite capacity is considered, and it is shown (under adequate assumptions) that an associated decentralized call admission control problem becomes tractable asymptotically, as the size of network grows to infinity. This is achieved by following a methodology first explored by M. Huang et al. (2003, 2006-2008) in the context of large scale dynamic games for sets of weakly coupled linear stochastic control systems. At the established Nash equilibrium, each agent reacts optimally with respect to the average trajectory of the mass of all other agents; this trajectory is approximated by a deterministic infinite population limit (associated with the mean field or ensemble statistics of the random agents) which is the solution of a particular fixed point problem. This framework has connections with the mean field models studied by Lasry and Lions (2006, 2007) and close connections with the notion of oblivious equilibrium proposed by Weintraub, Benkard, and Van Roy (2005, 2008) via a mean field approximation.

[1]  A. Mullin,et al.  Mathematical Theory of Connecting Networks and Telephone Traffic. , 1966 .

[2]  Peter E. Caines,et al.  Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle , 2006, Commun. Inf. Syst..

[3]  F. Kelly Routing in circuit-switched networks: optimization, shadow prices and decentralization , 1988, Advances in Applied Probability.

[4]  E. Çinlar On the superposition of m-dimensional point processes , 1968, Journal of Applied Probability.

[5]  S. Zachary,et al.  Loss networks , 2009, 0903.0640.

[6]  John N. Tsitsiklis,et al.  Call admission control and routing in integrated services networks using neuro-dynamic programming , 2000, IEEE Journal on Selected Areas in Communications.

[7]  Peter E. Caines,et al.  Control of Admission and Routing in Loss Networks: Hybrid Dynamic Programming Equations , 2010, IEEE Transactions on Automatic Control.

[8]  Jorma T. Virtamo,et al.  Polynomial cost approximations in markov decision theory based call admission control , 2001, TNET.

[9]  Peter E. Caines,et al.  On the Viscosity Solutions of Hybrid HJB Equations Arising in Loss Network System Control , 2007 .

[10]  Minyi Huang,et al.  Nash Certainty Equivalence in Large Population Stochastic Dynamic Games: Connections with the Physics of Interacting Particle Systems , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[11]  Eitan Altman,et al.  On optimal call admission control in resource-sharing system , 2001, IEEE Trans. Commun..

[12]  Peter E. Caines,et al.  Large population games in radial loss networks: Computationally tractable equilibria for distributed network admission control , 2009, 2009 International Conference on Game Theory for Networks.

[13]  Peter E. Caines,et al.  A locality generalization of the NCE (Mean Field) principle: Agent specific cost interactions , 2008, 2008 47th IEEE Conference on Decision and Control.

[14]  Benjamin Van Roy,et al.  Oblivious Equilibrium: A Mean Field Approximation for Large-Scale Dynamic Games , 2005, NIPS.

[15]  R. V. Gamkrelidze,et al.  Principles of optimal control theory , 1977 .

[16]  P. Lions,et al.  Jeux à champ moyen. II – Horizon fini et contrôle optimal , 2006 .

[17]  C. L. Benkard,et al.  Markov Perfect Industry Dynamics with Many Firms , 2005 .

[18]  Peter E. Caines,et al.  An Invariance Principle in Large Population Stochastic Dynamic Games , 2007, J. Syst. Sci. Complex..

[19]  P. Lions,et al.  Mean field games , 2007 .

[20]  Zbigniew Dziong,et al.  Call admission and routing in multi-service loss networks , 1994, IEEE Trans. Commun..

[21]  R. Weber,et al.  Optimal control of service rates in networks of queues , 1987, Advances in Applied Probability.

[22]  E. Altman,et al.  On submodular value functions and complex dynamic programming , 1998 .

[23]  B. Hajek Optimal control of two interacting service stations , 1982, 1982 21st IEEE Conference on Decision and Control.

[24]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[25]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[26]  Benjamin Van Roy,et al.  Markov Perfect Industry Dynamics with Many Firms , 2005 .

[27]  Minyi Huang,et al.  Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria , 2007, IEEE Transactions on Automatic Control.

[28]  P. Caines,et al.  Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[29]  F. Kelly Blocking probabilities in large circuit-switched networks , 1986, Advances in Applied Probability.

[30]  Peter E. Caines,et al.  Distributed control for radial loss network systems via the ash Certainty Equivalence (mean field) principle , 2008, 2008 47th IEEE Conference on Decision and Control.

[31]  Christos G. Cassandras,et al.  Adaptive call admission control in circuit-switched networks , 2002, IEEE Trans. Autom. Control..