Stochastic population games with individual independent states and coupled constraints

This paper studies non-cooperative population games with several individual states and independent Markov process. Each member of each class of the population has (i) its own state (ii) its actions in each state, (iii) an instantaneous reward which depends on its state and the population's profile, (iv) a time-average (coupled) constraints. We apply this model to battery-dependent power control in wireless networks with several types of renewable energies. We show that the game has an equilibrium in stationary strategies under ergodic assumptions and we present a class of evolutionary game dynamics which converge to stationary equilibria.

[1]  Dusit Niyato,et al.  Sleep and Wakeup Strategies in Solar-Powered Wireless Sensor/Mesh Networks: Performance Analysis and Optimization , 2007, IEEE Transactions on Mobile Computing.

[2]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[3]  Eitan Altman,et al.  Constrained Markov Games: Nash Equilibria , 2000 .

[4]  Nicolas Vieille,et al.  Stochastic Games with a Single Controller and Incomplete Information , 2002, SIAM J. Control. Optim..

[5]  Eitan Altman,et al.  Evolutionary games with random number of interacting players applied to access control , 2008, 2008 6th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks and Workshops.

[6]  Masayuki Takahashi.  Equilibrium points of stochastic non-cooperative $n$-person games , 1964 .

[7]  János Flesch,et al.  Stochastic games with additive transitions , 2007, Eur. J. Oper. Res..

[8]  S. Sorin.  Asymptotic properties of a non-zero sum stochastic game , 1986 .

[9]  Eitan Altman,et al.  Battery-state dependent power control as a dynamic game , 2008, 2008 6th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks and Workshops.

[10]  Eitan Altman,et al.  Evolutionary Power Control Games in Wireless Networks , 2008, Networking.

[11]  N. Vieille.  Stochastic Games: recent results , 2001 .

[12]  A. Hordijk,et al.  Contraction Conditions for Average and α-Discount Optimality in Countable State Markov Games with Unbounded Rewards , 1997, Math. Oper. Res..

[13]  A. M. Fink,et al.  Equilibrium in a stochastic $n$-person game , 1964 .

[14]  Nicolas Vieille,et al.  Chapter 48 Stochastic games: Recent results , 2002 .

[15]  Nicolas Vieille,et al.  Two-player stochastic games II: The case of recursive games , 2000 .

[16]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[17]  Eitan Altman,et al.  Zero-sum constrained stochastic games with independent state processes , 2005, Math. Methods Oper. Res..

[18]  Yezekael Hayel,et al.  A Stochastic Evolutionary Game of Energy Management in a Distributed Aloha Network , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.