Learning mechanisms for achieving context awareness and intelligence in Cognitive Radio networks

Providing that licensed or Primary Users (PUs) are oblivious to the presence of unlicensed or Secondary Users (SUs), Cognitive Radio (CR) enables the SUs to use underutilized licensed spectrum (or white spaces) opportunistically and temporarily conditional on the interference to the PUs being below an acceptable level. Context awareness and intelligence enable the SU to sense for and use the underutilized licensed spectrum in an efficient manner. This paper investigates various learning mechanisms for achieving context awareness and intelligence with respect to Dynamic Channel Selection (DCS) in CR networks. The learning mechanisms are Adaptation (Adapt), Window (Win), Adaptation-Window (AdaptWin), and Reinforcement Learning (RL). The DCS scheme helps SU base station to select channel adaptively for data transmission to its SU host in static and mobile centralized CR networks. The purpose is to enhance quality of service, particularly throughput and delay (in terms of number of channel switches), in the presence of channel heterogeneity. Our contribution is to investigate simple and yet pragmatic learning mechanisms for CR networks. Simulation results reveal that RL, AdaptWin and Win achieve approximately similar and the best possible network performance, followed by Adapt, and finally Random, which does not apply learning and serves as baseline.

[1]  Kok-Lim Alvin Yau,et al.  Achieving context awareness and intelligence in Cognitive Radio Networks using reinforcement learning for multi-state applications , 2010 .

[2]  David Grace,et al.  Cognitive radio with reinforcement learning applied to heterogeneous multicast terrestrial communication systems , 2009, 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[3]  E. Moulines,et al.  Dynamic spectrum access with non-stationary Multi-Armed Bandit , 2008, 2008 IEEE 9th Workshop on Signal Processing Advances in Wireless Communications.

[4]  Yenumula B. Reddy.  Detecting Primary Signals for Efficient Utilization of Spectrum Using Q-Learning , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[5]  Kok-Lim Alvin Yau,et al.  Achieving Context Awareness and Intelligence in Cognitive Radio Networks using Reinforcement Learning for Stateful Applications , 2010 .

[6]  D. Grace,et al.  Performance of cognitive radio reinforcement spectrum sharing using different weighting factors , 2008, 2008 Third International Conference on Communications and Networking in China.

[7]  Oriol Sallent,et al.  A Self-Organized Spectrum Assignment Strategy in Next Generation OFDMA Networks Providing Secondary Spectrum Access , 2009, 2009 IEEE International Conference on Communications.

[8]  Fangwen Fu,et al.  Detection of Spectral Resources in Cognitive Radios Using Reinforcement Learning , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Kok-Lim Alvin Yau,et al.  A context-aware and Intelligent Dynamic Channel Selection scheme for cognitive radio networks , 2009, 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications.