Reinforced HMM based learning model of dynamic spectrum allocation in UHF - ISM band of 902–928 MHz for Cognitive radio

A reinforced learning model based on HMM following urn and ball concept is suggested in this paper. Earlier works on HMM for Cognitive radio is towards predicting the channel state based on the activity of the primary user and on occupying the next available channel based on the power spectral density measurements. This work applies HMM techniques on real time spectral measurements and reinforces initial training based on the transition probabilities calculated from the measured power levels with the learning methodology based on the survival duration and packet loss data. The model is validated over the spectral measurements taken in the UHF-ISM band 902-928 MHz and shown to improve the reliability of the link decreasing the setting up delay of the link by selecting the best possible channel based on the interference level and the data on survival duration. The model is also analyzed with increased occupancy of the spectrum as more and more emerging users start occupying the band.

[1]  Hazem H. Refai,et al.  Cognitive radio architecture for rapidly deployable heterogeneous wireless networks , 2010, IEEE Transactions on Consumer Electronics.

[2]  D. Grace,et al.  Performance of cognitive radio reinforcement spectrum sharing using different weighting factors , 2008, 2008 Third International Conference on Communications and Networking in China.

[3]  C.W. Bostian,et al.  Online modeling of wireless channels with hidden Markov models and channel impulse responses for cognitive radios , 2004, 2004 IEEE MTT-S International Microwave Symposium Digest (IEEE Cat. No.04CH37535).

[4]  Jae Moung Kim,et al.  HMM-based Adaptive Frequency-Hopping Cognitive Radio System to Reduce Interference Time and to Improve Throughput , 2010, KSII Trans. Internet Inf. Syst..

[5]  Dhananjay Kumar,et al.  Spectrum opportunity in UHF — ISM band of 902–928 MHz for cognitive radio , 2011, 2011 Third International Conference on Advanced Computing.

[6]  Jie Liu,et al.  A Learning Spectrum Hole Prediction Model for Cognitive Radio Systems , 2010, 2010 10th IEEE International Conference on Computer and Information Technology.

[7]  Sang-Won Kim,et al.  HMM Based Channel Status Predictor for Cognitive Radio , 2007, 2007 Asia-Pacific Microwave Conference.

[8]  Joseph Mitola,et al.  Cognitive Radio An Integrated Agent Architecture for Software Defined Radio , 2000 .

[9]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[10]  David Grace,et al.  Cognitive radio with reinforcement learning applied to heterogeneous multicast terrestrial communication systems , 2009, 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[11]  Dhananjay Kumar,et al.  Radio resource management in OFDMA-CRN considering primary user activity and detection scenario , 2011, Proceedings of ITU Kaleidoscope 2011: The Fully Networked Human? - Innovations for Future Networks and Services (K-2011).

[12]  B. T. Maharaj,et al.  Performance of a Hidden Markov channel occupancy model for cognitive radio , 2011, IEEE Africon '11.

[13]  K. J. Ray Liu,et al.  Advances in cognitive radio networks: A survey , 2011, IEEE Journal of Selected Topics in Signal Processing.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  An He,et al.  A Survey of Artificial Intelligence for Cognitive Radios , 2010, IEEE Transactions on Vehicular Technology.

[16]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[17]  Brian L. Mark,et al.  Hidden Markov process based dynamic spectrum access for cognitive radio , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[18]  Preston Marshall.  Quantitative Analysis of Cognitive Radio and Network Performance , 2010, Mobile communications series.

[19]  Robert C. Qiu,et al.  Prediction of channel state for cognitive radio using higher-order hidden Markov model , 2010, Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon).