Performance Analysis of Reinforcement Learning for Achieving Context Awareness and Intelligence in Mobile Cognitive Radio Networks

Cognitive Radio (CR) is a key technology for improving the utilization level of the overall radio spectrum in wireless communications. It is able to sense and change its transmission and reception parameters adaptively according to spectrum availability in different spectrum channels. The Cognition Cycle (CC) is a state machine that is embodied in each CR host that defines the mechanisms related to achieving context awareness and intelligence including observation, learning, and action selection. The CC is the key element in the design of various applications in CR networks such as Dynamic Channel Selection (DCS), scheduling and congestion control. In this paper, Reinforcement Learning (RL) is employed to implement the CC in mobile CR networks. Previous works consider static networks with homogeneous channels. This paper analyzes the performance of RL as an approach to achieve context awareness and intelligence in regard to DCS in mobile CR networks with heterogeneous channels. Our contribution in this paper is to show whether RL is an appropriate tool to implement the CC. The results presented in this paper show that RL is a promising approach.

[1]  Fangwen Fu,et al.  Detection of Spectral Resources in Cognitive Radios Using Reinforcement Learning , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[2]  E. Moulines,et al.  Dynamic spectrum access with non-stationary Multi-Armed Bandit , 2008, 2008 IEEE 9th Workshop on Signal Processing Advances in Wireless Communications.

[3]  I.N. Vukovic,et al.  Delay analysis of different backoff algorithms in IEEE 802.11 , 2004, IEEE 60th Vehicular Technology Conference, 2004. VTC2004-Fall. 2004.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Kok-Lim Alvin Yau,et al.  Achieving context awareness and intelligence in Cognitive Radio Networks using reinforcement learning for multi-state applications , 2010 .

[6]  Oriol Sallent,et al.  A Self-Organized Spectrum Assignment Strategy in Next Generation OFDMA Networks Providing Secondary Spectrum Access , 2009, 2009 IEEE International Conference on Communications.

[7]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[8]  C.-C. Jay Kuo,et al.  A Cognitive MAC Protocol Using Statistical Channel Allocation for Wireless Ad-Hoc Networks , 2007, 2007 IEEE Wireless Communications and Networking Conference.

[9]  Ian F. Akyildiz,et al.  A survey on spectrum management in cognitive radio networks , 2008, IEEE Communications Magazine.

[10]  Yenumula B. Reddy.  Detecting Primary Signals for Efficient Utilization of Spectrum Using Q-Learning , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[11]  Joseph Mitola,et al.  Cognitive radio: making software radios more personal , 1999, IEEE Wirel. Commun..

[12]  Kok-Lim Alvin Yau,et al.  Applications of Reinforcement Learning to Cognitive Radio Networks , 2010, 2010 IEEE International Conference on Communications Workshops.

[13]  Kok-Lim Alvin Yau,et al.  A context-aware and Intelligent Dynamic Channel Selection scheme for cognitive radio networks , 2009, 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[14]  Timothy J. O'Shea,et al.  Applications of Machine Learning to Cognitive Radio Networks , 2007, IEEE Wireless Communications.