A strategic learning algorithm for state-based games

Learning algorithm design for state-based games is investigated. A heuristic uncoupled learning algorithm, which is a two memory better reply with inertia dynamics, is proposed. Under certain reasonable conditions it is proved that for any initial state, if all agents in the state-based game follow the proposed learning algorithm, the action state pair converges almost surely to an action invariant set of recurrent state equilibria. The design relies on global and local searches with finite memory, inertia, and randomness. Finally, existence of time-efficient universal learning algorithm is studied. A class of state-based games is presented to show that there is no universal learning algorithm converging to a recurrent state equilibrium.

[1]  Shengwei Mei,et al.  State-based potential game approach for distributed economic dispatch problem in smart grid , 2016, 2016 IEEE Power and Energy Society General Meeting (PESGM).

[2]  Jason R. Marden.  Selecting efficient correlated equilibria through distributed learning , 2017, Games Econ. Behav..

[3]  Ming Cao,et al.  Asynchronous Decision-Making Dynamics Under Best-Response Update Rule in Finite Heterogeneous Populations , 2018, IEEE Transactions on Automatic Control.

[4]  Mohammad Sadegh Talebi,et al.  Uncoupled Learning Rules for Seeking Equilibria in Repeated Plays: An Overview , 2013, ArXiv.

[5]  J. Jordan.  Three Problems in Learning Mixed-Strategy Nash Equilibria , 1993 .

[6]  Jason R. Marden,et al.  Achieving Pareto Optimality Through Distributed Learning , 2014, SIAM J. Control. Optim..

[7]  M. Johansson,et al.  Distributed Optimization and Games: A Tutorial Overview , 2010 .

[8]  H. Peyton Young,et al.  Strategic Learning and Its Limits , 2004 .

[9]  Yishay Mansour,et al.  How long to equilibrium? The communication complexity of uncoupled equilibrium procedures , 2010, Games Econ. Behav..

[10]  Nan Xiao,et al.  Distributed consensus in noncooperative congestion games: An application to road pricing , 2013, 2013 10th IEEE International Conference on Control and Automation (ICCA).

[11]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[12]  Walid Saad,et al.  Game-Theoretic Methods for the Smart Grid: An Overview of Microgrid Systems, Demand-Side Management, and Smart Grid Communications , 2012, IEEE Signal Processing Magazine.

[13]  Wei Ren,et al.  Game theory control solution for sensor coverage problem in unknown environment , 2014, 53rd IEEE Conference on Decision and Control.

[14]  H. Peyton Young,et al.  Learning by trial and error , 2009, Games Econ. Behav..

[15]  S. Hart,et al.  A simple adaptive procedure leading to correlated equilibrium , 2000 .

[16]  Jason R. Marden.  State based potential games , 2012, Autom..

[17]  Jeff S. Shamma,et al.  Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria , 2005, IEEE Transactions on Automatic Control.

[18]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[19]  H. Peyton Young,et al.  Learning efficient Nash equilibria in distributed systems , 2012, Games Econ. Behav..

[20]  J. French,et al.  A formal theory of social power. , 1956, Psychological review.

[21]  Nicanor Quijano,et al.  Game-Theoretical Methods in Control of Engineering Systems: An Introduction to the Special Issue , 2017, IEEE Control Systems.

[22]  Jason R. Marden,et al.  Game Theory and Distributed Control , 2015 .