A stochastic multi-channel spectrum access game with incomplete information

To ensure continuous functioning and satisfactory performance, a wireless communication system has to not only learn and adapt to the unknown and ever-changing wireless environment, but also strategically deal with the usually unfamiliar peers. Incomplete information stochastic game (SG) is a promising model for the corresponding analysis and strategy design. In this work, an exemplary multi-channel spectrum access game (SAG) with unknown environment dynamics and limited information of the other player is considered to illustrate the proposed solution for the corresponding incomplete information SG. To find the best communication strategy in the face of uncertainty, a joint reinforcement learning and type identification algorithm is developed, which is provably convergent under certain technical conditions. Numerical results show that using the proposed algorithm, a wireless user can gradually achieve the same performance as that in the corresponding complete information game.

[1]  Anthony Ephremides,et al.  Jamming games in wireless networks with incomplete information , 2011, IEEE Communications Magazine.

[2]  Nicolas Vieille,et al.  Stochastic Games with a Single Controller and Incomplete Information , 2002, SIAM J. Control. Optim..

[3]  Quanyan Zhu,et al.  Distributed strategic learning with application to network security , 2011, Proceedings of the 2011 American Control Conference.

[4]  Dusit Niyato,et al.  Coalition-Based Cooperative Packet Delivery under Uncertainty: A Dynamic Bayesian Coalitional Game , 2013, IEEE Transactions on Mobile Computing.

[5]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[6]  Anthony Ephremides,et al.  MAC games for distributed wireless network security with incomplete information of selfish and malicious user types , 2009, 2009 International Conference on Game Theory for Networks.

[7]  Stephen B. Wicker,et al.  Random Access Games: Selfish Nodes with Incomplete Information , 2007, MILCOM 2007 - IEEE Military Communications Conference.

[8]  K. J. Ray Liu,et al.  An anti-jamming stochastic game for cognitive radio networks , 2011, IEEE Journal on Selected Areas in Communications.

[9]  Dong In Kim,et al.  Game Theoretic Approaches for Multiple Access in Wireless Networks: A Survey , 2011, IEEE Communications Surveys & Tutorials.

[10]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[11]  Cristina Comaniciu,et al.  A Bayesian game approach for intrusion detection in wireless ad hoc networks , 2006, GameNets '06.

[12]  Peng Ning,et al.  Improving learning and adaptation in security games by exploiting information asymmetry , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[13]  Yang Xiao,et al.  Game Theory for Network Security , 2013, IEEE Communications Surveys & Tutorials.

[14]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[15]  Francis Minhthang Bui,et al.  A Game-Theoretic Framework for Robust Optimal Intrusion Detection in Wireless Sensor Networks , 2014, IEEE Transactions on Information Forensics and Security.

[16]  John M. Cioffi,et al.  Increase in capacity of multiuser OFDM system using dynamic subchannel allocation , 2000, VTC2000-Spring. 2000 IEEE 51st Vehicular Technology Conference Proceedings (Cat. No.00CH37026).

[17]  Michael L. Littman,et al.  Friend-or-Foe Q-learning in General-Sum Games , 2001, ICML.