Resource Allocation for Multi-User Cognitive Radio Systems Using Multi-agent Q-Learning

Cognitive Radio (CR) is a new generation of wireless communication system that enables unlicensed users to exploit underutilized licensed spectrum to optimize the radio spectrum utilization. The resource allocation is difficult to achieve in a dynamic distributed environment, in which CR users take decisions to select a channel without negotiation, and react to the environmental changes. This paper focuses on using a multi-agent reinforcementlearning (MARL), Q-learning algorithm, on channels selection decision by secondary users in 2×2 and 3×3 cognitive radio system. Numerical results, obtained with MATLAB, demonstrate that resource allocation is realized without any negotiation between secondary and primary users. In this work, the analogy between the numerical and simulated results is also noted.

[1]  Liaoyuan Zeng,et al.  Spectrum efficiency optimization in multiuser Ultra Wideband cognitive radio networks , 2010, 2010 7th International Symposium on Wireless Communication Systems.

[2]  Xuemin Shen,et al.  Adaptive Admission-Control and Channel-Allocation Policy in Cooperative Ad Hoc Opportunistic Spectrum Networks , 2010, IEEE Transactions on Vehicular Technology.

[3]  Balasubramaniam Natarajan,et al.  Modeling fairness in resource allocation for secondary users in a competitive cognitive radio network , 2010, 2010 Wireless Telecommunications Symposium (WTS).

[4]  Tian Tian,et al.  Methods on Further Improving the Spectrum Management , 2007, 2007 International Symposium on Electromagnetic Compatibility.

[5]  Pawel A. Dmochowski,et al.  Analysis and implementation of reinforcement learning on a GNU Radio cognitive radio platform , 2010, 2010 Proceedings of the Fifth International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[6]  Xu Mao,et al.  Biologically-Inspired Distributed Spectrum Access for Cognitive Radio Network , 2010, 2010 6th International Conference on Wireless Communications Networking and Mobile Computing (WiCOM).

[7]  Ben Polak,et al.  Fictitious play in 2×2 games: A geometric proof of convergence , 1994 .

[8]  Youyun Xu,et al.  A Q-Learning based sensing task selection scheme for cognitive radio networks , 2009, 2009 International Conference on Wireless Communications & Signal Processing.

[9]  Donya He,et al.  Cognitive Radio and RF Communications Design Optimization using Genetic Algorithms , 2007, MILCOM 2007 - IEEE Military Communications Conference.

[10]  Zhenyu Zhang,et al.  Intelligent cognitive radio: Research on learning and evaluation of CR based on Neural Network , 2007, 2007 ITI 5th International Conference on Information and Communications Technology.

[11]  Anni Cai,et al.  Evolutionary algorithms for radio resource management in cognitive radio network , 2009, 2009 IEEE 28th International Performance Computing and Communications Conference.

[12]  Dorothy Okello,et al.  Dynamic spectrum allocation in multiuser wireless networks , 2011, 2011 IST-Africa Conference Proceedings.

[13]  Husheng Li,et al.  Multi-agent Q-learning of channel selection in multi-user cognitive radio systems: A two by two case , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[14]  Stefan Mangold,et al.  Spectrum agile radio: radio resource measurements for opportunistic spectrum usage , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[15]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[16]  Peng Wang,et al.  Joint spectrum allocation and scheduling in multi-radio multi-channel cognitive radio wireless networks , 2010, 2010 IEEE Sarnoff Symposium.

[17]  Fang Ye,et al.  Spectrum detection model for cognitive radio networks , 2010, 2010 IEEE Fifth International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA).

[18]  Joseph Mitola,et al.  Cognitive Radio An Integrated Agent Architecture for Software Defined Radio , 2000 .

[19]  Cheng Wu,et al.  Spectrum management of cognitive radio using multi-agent reinforcement learning , 2010, AAMAS.

[20]  Chengshu Li,et al.  Dynamic Channel Selection Algorithm for Cognitive Radios , 2008, 2008 4th IEEE International Conference on Circuits and Systems for Communications.

[21]  Xiaodong Wang,et al.  Optimal Radio Allocation for Multi-radio Cognitive Wireless Networks , 2006 .

[22]  Zhenyu Zhang,et al.  Application research of evolution in cognitive radio based on GA , 2008, 2008 3rd IEEE Conference on Industrial Electronics and Applications.

[23]  Zheng Zhou,et al.  Optimization of Collaborative Spectrum Sensing for Cognitive Radio , 2008, 2008 IEEE International Conference on Networking, Sensing and Control.

[24]  Hong Jiang,et al.  Modeling of Learning Inference and Decision-Making Engine in Cognitive Radio , 2010, 2010 Second International Conference on Networks Security, Wireless Communications and Trusted Computing.

[25]  F. Lahouti,et al.  Spectral Efficiency Optimized Adaptive Transmission for Interfering Cognitive Radios , 2009, 2009 IEEE International Conference on Communications Workshops.

[26]  K. Moessner,et al.  Collaborative Spectrum Sensing for Cognitive Radio , 2009, 2009 IEEE International Conference on Communications Workshops.

[27]  M. Zorzi,et al.  Learning and Adaptation in Cognitive Radios Using Neural Networks , 2008, 2008 5th IEEE Consumer Communications and Networking Conference.

[28]  Zengyou Sun,et al.  Study of Cognitive Radio Spectrum Detection in OFDM System , 2010, 2010 Asia-Pacific Conference on Wearable Computing Systems.

[29]  Christophe J. Le Martret,et al.  Parameter determination of secondary user cognitive radio network using genetic algorithm , 2009, 2009 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing.

[30]  Fangwen Fu,et al.  Detection of Spectral Resources in Cognitive Radios Using Reinforcement Learning , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[31]  Cheng Wu,et al.  Learning-Based Spectrum Selection in Cognitive Radio Ad Hoc Networks , 2010, WWIC.

[32]  R. Knopp,et al.  Cognitive radio Research and Implementation Challenges , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[33]  Venugopal V. Veeravalli,et al.  Algorithms for Dynamic Spectrum Access With Learning for Cognitive Radio , 2008, IEEE Transactions on Signal Processing.

[34]  Joseph Mitola,et al.  Software Radio Architecture: Object-Oriented Approaches to Wireless Systems Engineering , 2000 .