A Survey on Machine-Learning Techniques in Cognitive Radios

In this survey paper, we characterize the learning problem in cognitive radios (CRs) and state the importance of artificial intelligence in achieving real cognitive communications systems. We review various learning problems that have been studied in the context of CRs classifying them under two main categories: Decision-making and feature classification. Decision-making is responsible for determining policies and decision rules for CRs while feature classification permits identifying and classifying different observation models. The learning algorithms encountered are categorized as either supervised or unsupervised algorithms. We describe in detail several challenging learning issues that arise in cognitive radio networks (CRNs), in particular in non-Markovian environments and decentralized networks, and present possible solution methods to address them. We discuss similarities and differences among the presented algorithms and identify the conditions under which each of the techniques may be applied.

[1]  Dave Cavalcanti,et al.  Spectrum Sensing for Dynamic Spectrum Access of TV Bands , 2007, 2007 2nd International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[2]  Eitan Altman,et al.  A Hybrid Approach for Radio Resource Management in Heterogeneous Cognitive Networks , 2011, IEEE Journal on Selected Areas in Communications.

[3]  A.R. Hammons,et al.  A common lexicon and design issues surrounding cognitive radio networks operating in the presence of jamming , 2008, MILCOM 2008 - 2008 IEEE Military Communications Conference.

[4]  Vikram Krishnamurthy Decentralized Spectrum Access Amongst Cognitive Radios—An Interacting Multivariate Global Game-Theoretic Approach , 2009, IEEE Transactions on Signal Processing.

[5]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[6]  Apurva N. Mody,et al.  Machine Learning based Cognitive Communications in White as Well as the Gray Space , 2007, MILCOM 2007 - IEEE Military Communications Conference.

[7]  Geoffrey Ye Li,et al.  Detection Timing and Channel Selection for Periodic Spectrum Sensing in Cognitive Radio , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[8]  Venugopal V. Veeravalli,et al.  Cooperative Sensing for Primary Detection in Cognitive Radio , 2008, IEEE Journal of Selected Topics in Signal Processing.

[9]  Ananthram Swami,et al.  A Survey of Dynamic Spectrum Access: Signal Processing and Networking Perspectives , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[10]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[11]  Sudharman K. Jayaweera,et al.  Optimal and Low-Complexity Algorithms for Dynamic Spectrum Access in Centralized Cognitive Radio Networks with Fading Channels , 2011, 2011 IEEE 73rd Vehicular Technology Conference (VTC Spring).

[12]  Andrea J. Goldsmith,et al.  Breaking Spectrum Gridlock With Cognitive Radios: An Information Theoretic Perspective , 2009, Proceedings of the IEEE.

[13]  George Iosifidis,et al.  Challenges in auction theory driven spectrum management , 2011, IEEE Communications Magazine.

[14]  E. Postma,et al.  Evolutionary Learning Outperforms Reinforcement Learning on Non-Markovian Tasks , 2005 .

[15]  Wei Yuan,et al.  Threshold-Learning in Local Spectrum Sensing of Cognitive Radio , 2009, VTC Spring 2009 - IEEE 69th Vehicular Technology Conference.

[16]  R. M. Buehrer,et al.  Game theoretic analysis of a network of cognitive radios , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[17]  J. J. Popoola,et al.  A Novel Modulation-Sensing Method , 2011, IEEE Vehicular Technology Magazine.

[18]  Scott Sanner,et al.  Practical solution techniques for first-order MDPs , 2009, Artif. Intell..

[19]  Tetsuji Ogawa,et al.  Speaker Clustering Based on Utterance-Oriented Dirichlet Process Mixture Model , 2011, INTERSPEECH.

[20]  Behrouz Farhang-Boroujeny,et al.  Multicarrier communication techniques for spectrum sensing and communication in cognitive radios , 2008, IEEE Communications Magazine.

[21]  T. Costlow Cognitive radios will adapt to users , 2003 .

[22]  Hong Jiang,et al.  Modeling of Learning Inference and Decision-Making Engine in Cognitive Radio , 2010, 2010 Second International Conference on Networks Security, Wireless Communications and Trusted Computing.

[23]  Sudharman K. Jayaweera,et al.  Optimal Myopic Sensing and Dynamic Spectrum Access in Centralized Secondary Cognitive Radio Networks with Low-Complexity Implementations , 2011, 2011 IEEE 73rd Vehicular Technology Conference (VTC Spring).

[24]  Q. Zhao,et al.  Decentralized cognitive mac for dynamic spectrum access , 2005, First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005..

[25]  Ian F. Akyildiz,et al.  A survey on spectrum management in cognitive radio networks , 2008, IEEE Communications Magazine.

[26]  Wei Yuan,et al.  Joint Power and Rate Control in Cognitive Radio Networks: A Game-Theoretical Approach , 2008, 2008 IEEE International Conference on Communications.

[27]  Quanyan Zhu,et al.  No-Regret Learning in Collaborative Spectrum Sensing with Malicious Nodes , 2010, 2010 IEEE International Conference on Communications.

[28]  Min Han,et al.  Prediction of chaotic time series based on the recurrent predictor neural network , 2004, IEEE Transactions on Signal Processing.

[29]  Vijay K. Bhargava,et al.  Cognitive Wireless Communication Networks , 2007 .

[30]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[31]  Zhu Han,et al.  Distributive Opportunistic Spectrum Access for Cognitive Radio using Correlated Equilibrium and No-Regret Learning , 2007, 2007 IEEE Wireless Communications and Networking Conference.

[32]  Sudharman K. Jayaweera,et al.  Multidimensional Dirichlet Process-Based Non-Parametric Signal Classification for Autonomous Self-Learning Cognitive Radios , 2013, IEEE Transactions on Wireless Communications.

[33]  Sudharman K. Jayaweera,et al.  Ieee Transactions on Wireless Communications, Accepted for Publication Asymmetric Cooperative Communications Based Spectrum Leasing via Auctions in Cognitive Radio Networks , 2022 .

[34]  Haris Volos,et al.  Cognitive Engine Design for Link Adaptation: An Application to Multi-Antenna Systems , 2010, IEEE Transactions on Wireless Communications.

[35]  Vikram Krishnamurthy,et al.  Game Theoretic Rate Adaptation for Spectrum-Overlay Cognitive Radio Networks , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[36]  D. Kazakos,et al.  On-Line Threshold Learning for Neyman-Pearson Distributed Detection , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[37]  Mohsen Guizani,et al.  Opportunistic Exploitation of Bandwidth Resources through Reinforcement Learning , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[38]  Hai-Yuan Liu,et al.  A Modulation Type Recognition Method Using Wavelet Support Vector Machines , 2009, 2009 2nd International Congress on Image and Signal Processing.

[39]  Christos G. Christodoulou,et al.  Radiobots: The autonomous, self-learning future cognitive radios , 2011, 2011 International Symposium on Intelligent Signal Processing and Communications Systems (ISPACS).

[40]  H. Robbins A Stochastic Approximation Method , 1951 .

[41]  Qing Zhao,et al.  Decentralized dynamic spectrum access for cognitive radios: cooperative design of a non-cooperative game , 2009, IEEE Transactions on Communications.

[42]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[43]  David G. Daut,et al.  Signature Based Spectrum Sensing Algorithms for IEEE 802.22 WRAN , 2007, 2007 IEEE International Conference on Communications.

[44]  Yueming Cai,et al.  A learner based on neural network for cognitive radio , 2010, 2010 IEEE 12th International Conference on Communication Technology.

[45]  Feifei Gao,et al.  Optimization of Cooperative Spectrum Sensing in Cognitive Radio , 2011, IEEE Transactions on Vehicular Technology.

[46]  D. Cabric Addressing feasibility of cognitive radios , 2008, IEEE Signal Processing Magazine.

[47]  Sudharman K. Jayaweera,et al.  Wideband Spectrum Sensing and Non-Parametric Signal Classification for Autonomous Self-Learning Cognitive Radios , 2012, IEEE Transactions on Wireless Communications.

[48]  Simon Haykin,et al.  Spectrum Sensing for Cognitive Radio , 2009, Proceedings of the IEEE.

[49]  Zhiyong Feng,et al.  Centralized channel and power allocation for cognitive radio networks: A Q-learning solution , 2010, 2010 Future Network & Mobile Summit.

[50]  Guan Yu,et al.  Document clustering via dirichlet process mixture model with feature selection , 2010, KDD.

[51]  An He,et al.  A Survey of Artificial Intelligence for Cognitive Radios , 2010, IEEE Transactions on Vehicular Technology.

[52]  Christos G. Christodoulou,et al.  Radiobots: Architecture, Algorithms and Realtime Reconfigurable Antenna Designs for Autonomous, Self-learning Future Cognitive Radios , 2011 .

[53]  Stefan Schaal,et al.  Policy Gradient Methods for Robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[54]  Dong Chen,et al.  Multiuser Power and Channel Allocation Algorithm in Cognitive Radio , 2007, 2007 International Conference on Parallel Processing (ICPP 2007).

[55]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[56]  Geoffrey Ye Li,et al.  Spatial Spectrum Holes for Cognitive Radio with Directional Transmission , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[57]  Alagan Anpalagan,et al.  Opportunistic Spectrum Access in Unknown Dynamic Environment: A Game-Theoretic Stochastic Learning Solution , 2012, IEEE Transactions on Wireless Communications.

[58]  Behrouz Farhang-Boroujeny,et al.  Filter Bank Spectrum Sensing for Cognitive Radios , 2008, IEEE Transactions on Signal Processing.

[59]  Geoffrey Ye Li,et al.  Cooperative Spectrum Sensing in Cognitive Radio, Part II: Multiuser Networks , 2007, IEEE Transactions on Wireless Communications.

[60]  K. J. Ray Liu,et al.  Advances in cognitive radio networks: A survey , 2011, IEEE Journal of Selected Topics in Signal Processing.

[61]  V. Tarokh,et al.  Cognitive radio networks , 2008, IEEE Signal Processing Magazine.

[62]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[63]  Dandan Zhang,et al.  SVM-Based Spectrum Sensing in Cognitive Radio , 2011, 2011 7th International Conference on Wireless Communications, Networking and Mobile Computing.

[64]  Chandra R. Murthy,et al.  Cyclostationary-Based Architectures for Spectrum Sensing in IEEE 802.22 WRAN , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[65]  George E. Monahan,et al.  A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 2007 .

[66]  Mischa Dohler,et al.  Docitive networks: an emerging paradigm for dynamic spectrum management [Dynamic Spectrum Management] , 2010, IEEE Wireless Communications.

[67]  Ananthram Swami,et al.  Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors , 2007, IEEE Transactions on Information Theory.

[68]  Shigeo Abe Support Vector Machines for Pattern Classification , 2010, Advances in Pattern Recognition.

[69]  T. Ferguson A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[70]  Muhammad Imran Taj,et al.  Cognitive Radio Spectrum E volution Prediction using A rtificial Neural Networks based Multivariate T ime Series Modelling , 2011, EW.

[71]  Mahamod Ismail,et al.  Development of a cognitive radio decision engine using multi-objective hybrid genetic algorithm , 2009, 2009 IEEE 9th Malaysia International Conference on Communications (MICC).

[72]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[73]  Emin Orhan Dirichlet Processes , 2012 .

[74]  Simon Haykin,et al.  Cognitive radio: brain-empowered wireless communications , 2005, IEEE Journal on Selected Areas in Communications.

[75]  Brian M. Sadler,et al.  A Survey of Dynamic Spectrum Access , 2007, IEEE Signal Processing Magazine.

[76]  Shlomo Zilberstein,et al.  Dynamic Programming for Partially Observable Stochastic Games , 2004, AAAI.

[77]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[78]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[79]  Maria-Gabriella Di Benedetto,et al.  A Survey on MAC Strategies for Cognitive Radio Networks , 2012, IEEE Communications Surveys & Tutorials.

[80]  K. J. Ray Liu,et al.  Evolutionary cooperative spectrum sensing game: how to collaborate? , 2010, IEEE Transactions on Communications.

[81]  Geoffrey Ye Li,et al.  Simplified Relay Selection and Power Allocation in Cooperative Cognitive Radio Systems , 2011, IEEE Transactions on Wireless Communications.

[82]  K. J. Ray Liu,et al.  An anti-jamming stochastic game for cognitive radio networks , 2011, IEEE Journal on Selected Areas in Communications.

[83]  Venugopal V. Veeravalli,et al.  Algorithms for Dynamic Spectrum Access With Learning for Cognitive Radio , 2008, IEEE Transactions on Signal Processing.

[84]  Peter L. Bartlett,et al.  Infinite-Horizon Policy-Gradient Estimation , 2001, J. Artif. Intell. Res..

[85]  Thomas Dean Atwood,et al.  RF channel characterization for cognitive radio using support vector machines , 2010 .

[86]  Yonghong Zeng,et al.  Power Control in Cognitive Radios under Cooperative and Non-Cooperative Spectrum Sensing , 2011, IEEE Transactions on Wireless Communications.

[87]  Biing-Hwang Juang,et al.  Signal Processing in Cognitive Radio , 2009, Proceedings of the IEEE.

[88]  N. Mandayam,et al.  Demand responsive pricing and competitive spectrum allocation via a spectrum server , 2005, First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005..

[89]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[90]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[91]  Zhu Han,et al.  Replacement of spectrum sensing in cognitive radio , 2009, IEEE Transactions on Wireless Communications.

[92]  David Grace,et al.  Efficient exploration in reinforcement learning-based cognitive radio spectrum sharing , 2011, IET Commun..

[93]  Khaled Ben Letaief,et al.  Cooperative Communications for Cognitive Radio Networks , 2009, Proceedings of the IEEE.

[94]  Joseph Mitola,et al.  Cognitive Radio Architecture Evolution , 2009, Proceedings of the IEEE.

[95]  Young Min Kim,et al.  An Alternative Energy Detection Using Sliding Window for Cognitive Radio System , 2008, 2008 10th International Conference on Advanced Communication Technology.

[96]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[97]  Carlos Mosquera,et al.  A dynamic spectrum leasing (DSL) framework for spectrum sharing in cognitive radio networks , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[98]  Yenumula B. Reddy Detecting Primary Signals for Efficient Utilization of Spectrum Using Q-Learning , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[99]  Dusit Niyato,et al.  A Neural Network Based Spectrum Prediction Scheme for Cognitive Radio , 2010, 2010 IEEE International Conference on Communications.

[100]  Haibo He,et al.  MAC protocol classification in a cognitive radio network , 2010, The 19th Annual Wireless and Optical Communications Conference (WOCC 2010).

[101]  E. Thorndike “Animal Intelligence” , 1898, Nature.

[102]  M. Zorzi,et al.  Learning and Adaptation in Cognitive Radios Using Neural Networks , 2008, 2008 5th IEEE Consumer Communications and Networking Conference.

[103]  Stuart German,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1988 .

[104]  Fangwen Fu,et al.  Stochastic Game Formulation for Cognitive Radio Networks , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[105]  Cristina Comaniciu,et al.  Adaptive Channel Allocation Spectrum Etiquette for Cognitive Radio Networks , 2005, First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005..

[106]  J. Nicholas Laneman,et al.  Sequence Detection Algorithms for PHY-Layer Sensing in Dynamic Spectrum Access Networks , 2011, IEEE Journal of Selected Topics in Signal Processing.

[107]  Sudharman K. Jayaweera,et al.  Dynamic spectrum leasing in cognitive radio networks via primary-secondary user power control games , 2009, IEEE Transactions on Wireless Communications.

[108]  Berna Sayraç,et al.  Semi Dynamic Parameter Tuning for Optimized Opportunistic Spectrum Access , 2008, 2008 IEEE 68th Vehicular Technology Conference.

[109]  Oriol Sallent,et al.  Introduction to IEEE P1900.4 Activities , 2008, IEICE Trans. Commun..

[110]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[111]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[112]  Mihaela van der Schaar,et al.  Spectrum Access Games and Strategic Learning in Cognitive Radio Networks for Delay-Critical Applications , 2009, Proceedings of the IEEE.

[113]  Sudharman K. Jayaweera Learning to Thrive in a Leasing Market : An Auctioning Framework for Distributed Dynamic Spectrum Leasing ( D-DSL ) , 2010 .

[114]  Fangwen Fu,et al.  Detection of Spectral Resources in Cognitive Radios Using Reinforcement Learning , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[115]  Carlo S. Regazzoni,et al.  Spectrum sensing: A distributed approach for cognitive terminals , 2007, IEEE Journal on Selected Areas in Communications.

[116]  Sofie Pollin,et al.  Identifying Spectrum Usage by Unknown Systems using Experiments in Machine Learning , 2009, 2009 IEEE Wireless Communications and Networking Conference.

[117]  Carlos Mosquera,et al.  Dynamic Spectrum Leasing: A New Paradigm for Spectrum Sharing in Cognitive Radio Networks , 2010, IEEE Transactions on Vehicular Technology.

[118]  Yonghong Zeng,et al.  Maximum-Minimum Eigenvalue Detection for Cognitive Radio , 2007, 2007 IEEE 18th International Symposium on Personal, Indoor and Mobile Radio Communications.

[119]  Rong Zheng,et al.  Repeated Auctions with Bayesian Nonparametric Learning for Spectrum Access in Cognitive Radio Networks , 2011, IEEE Transactions on Wireless Communications.

[120]  Hong Jiang,et al.  Design of optimal engine for cognitive radio parameters based on the DUGA , 2010, The 3rd International Conference on Information Sciences and Interaction Sciences.

[121]  Sudharman K. Jayaweera,et al.  Distributed Reinforcement Learning based MAC protocols for autonomous cognitive secondary users , 2011, 2011 20th Annual Wireless and Optical Communications Conference (WOCC).

[122]  Thomas R. Shultz,et al.  Connectionist Models of Reinforcement, Imitation, and Instruction in Learning to Solve Complex Problems , 2009, IEEE Transactions on Autonomous Mental Development.

[123]  Sudharman K. Jayaweera,et al.  Optimal Myopic Sensing and Dynamic Spectrum Access in Cognitive Radio Networks with Low-Complexity Implementations , 2012, IEEE Transactions on Wireless Communications.

[124]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[125]  T. Ferguson Prior Distributions on Spaces of Probability Measures , 1974 .

[126]  Rajarathnam Chandramouli,et al.  Human behavior inspired cognitive radio network design , 2008, IEEE Communications Magazine.

[127]  C.G. Christodoulou,et al.  Signal classification with an SVM-FFT approach for feature extraction in cognitive radio , 2009, 2009 SBMO/IEEE MTT-S International Microwave and Optoelectronics Conference (IMOC).

[128]  Joseph Mitola,et al.  Cognitive radio: making software radios more personal , 1999, IEEE Wirel. Commun..

[129]  Geoffrey Ye Li,et al.  Cooperative Spectrum Sensing in Cognitive Radio, Part I: Two User Networks , 2007, IEEE Transactions on Wireless Communications.

[130]  Timothy J. O'Shea,et al.  Applications of Machine Learning to Cognitive Radio Networks , 2007, IEEE Wireless Communications.

[131]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[132]  Ying-Chang Liang,et al.  Covariance Based Signal Detections for Cognitive Radio , 2007, 2007 2nd IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[133]  Georgios B. Giannakis,et al.  A Wavelet Approach to Wideband Spectrum Sensing for Cognitive Radios , 2006, 2006 1st International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[134]  Junhui Zhao,et al.  Power control based on the Asynchronous Distributed Pricing Algorithm in cognitive radios , 2010, 2010 IEEE Youth Conference on Information, Computing and Telecommunications.

[135]  Michele Zorzi,et al.  A Neural Network Based Cognitive Controller for Dynamic Channel Selection , 2009, 2009 IEEE International Conference on Communications.

[136]  Brian Choi,et al.  Distributed Spectrum Sensing for Cognitive Radio Systems , 2007, 2007 Information Theory and Applications Workshop.

[137]  Yang Yang,et al.  Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks , 2009, 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[138]  Marina Petrova,et al.  Multi-class classification of analog and digital signals in cognitive radios using Support Vector Machines , 2010, 2010 7th International Symposium on Wireless Communication Systems.

[139]  Mischa Dohler,et al.  Learning from experts in cognitive radio networks: The docitive paradigm , 2010, 2010 Proceedings of the Fifth International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[140]  Trevor Darrell Reinforcement Learning of Active Recognition Behaviors , 1997, NIPS 1997.

[141]  Junde Song,et al.  Signal Classification Based on Spectral Correlation Analysis and SVM in Cognitive Radio , 2008, 22nd International Conference on Advanced Information Networking and Applications (aina 2008).

[142]  R.G. Wendorf,et al.  A Channel-Change Game for Multiple Interfering Cognitive Wireless Networks , 2006, MILCOM 2006 - 2006 IEEE Military Communications conference.

[143]  Awais Khawar,et al.  Robust Signal Classification Using Unsupervised Learning , 2011, IEEE Transactions on Wireless Communications.

[144]  Georgios B. Giannakis,et al.  Statistical tests for presence of cyclostationarity , 1994, IEEE Trans. Signal Process..

[145]  Mohsen Guizani,et al.  Opportunistic Bandwidth Sharing Through Reinforcement Learning , 2010, IEEE Transactions on Vehicular Technology.

[146]  No-Regret learning for simultaneous power control and channel allocation in cognitive radio networks , 2012, 2012 Computing, Communications and Applications Conference.

[147]  Kang G. Shin,et al.  Cognitive radios for dynamic spectrum access: from concept to reality , 2010, IEEE Wireless Communications.

[148]  K. J. Ray Liu,et al.  Evolutionary Game Framework for Behavior Dynamics in Cooperative Spectrum Sensing , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[149]  Husheng Li,et al.  Multi-agent Q-learning of channel selection in multi-user cognitive radio systems: A two by two case , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[150]  M. Escobar Estimating Normal Means with a Dirichlet Process Prior , 1994 .

[151]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[152]  Wenbo Wang,et al.  Noncooperative Power Control Game with Exponential Pricing for Cognitive Radio Network , 2007, 2007 IEEE 65th Vehicular Technology Conference - VTC2007-Spring.

[153]  Yourong Lu,et al.  Channel and Modulation Selection Based on Support Vector Machines for Cognitive Radio , 2006, 2006 International Conference on Wireless Communications, Networking and Mobile Computing.

[154]  Umberto Spagnolini,et al.  Spectrum Leasing to Cooperating Secondary Ad Hoc Networks , 2008, IEEE Journal on Selected Areas in Communications.

[155]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[156]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[157]  Wei Lin,et al.  Artificial Neural Network Based Spectrum Sensing Method for Cognitive Radio , 2010, 2010 6th International Conference on Wireless Communications Networking and Mobile Computing (WiCOM).

[158]  Sudharman K. Jayaweera,et al.  Blind cyclostationary feature detection based spectrum sensing for autonomous self-learning cognitive radios , 2012, 2012 IEEE International Conference on Communications (ICC).

[159]  Amir Ghasemi,et al.  Spectrum sensing in cognitive radio networks: requirements, challenges and design trade-offs , 2008, IEEE Communications Magazine.

[160]  L. Morales-Tirado,et al.  A Hybrid Cognitive Engine for Improving Coverage in 3G Wireless Networks , 2009, 2009 IEEE International Conference on Communications Workshops.

[161]  Qian Zhang,et al.  Cooperative relay to improve diversity in cognitive radio networks , 2009, IEEE Commun. Mag..

[162]  H. Vincent Poor,et al.  Collaborative Cyclostationary Spectrum Sensing for Cognitive Radio Systems , 2009, IEEE Transactions on Signal Processing.

[163]  Ananthram Swami,et al.  Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP framework , 2007, IEEE Journal on Selected Areas in Communications.

[164]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[165]  Tan Yee Fan,et al.  A Tutorial on Support Vector Machine , 2009 .

[166]  Hüseyin Arslan,et al.  A survey of spectrum sensing algorithms for cognitive radio applications , 2009, IEEE Communications Surveys & Tutorials.

[167]  Zhi Ding,et al.  Opportunistic spectrum access in cognitive radio networks , 2008, IJCNN.

[168]  Shiwen Mao,et al.  Performance Evaluation of Cognitive Radios: Metrics, Utility Functions, and Methodology , 2009, Proceedings of the IEEE.

[169]  Joseph Mitola,et al.  Cognitive Radio An Integrated Agent Architecture for Software Defined Radio , 2000 .

[170]  Fernando Paganini,et al.  Mechanism-based resource allocation for multimedia transmission over spectrum agile wireless networks , 2007, IEEE Journal on Selected Areas in Communications.

[171]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[172]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[173]  Youyun Xu,et al.  A Q-Learning based sensing task selection scheme for cognitive radio networks , 2009, 2009 International Conference on Wireless Communications & Signal Processing.

[174]  Ana Galindo-Serrano,et al.  Distributed Q-Learning for Aggregated Interference Control in Cognitive Radio Networks , 2010, IEEE Transactions on Vehicular Technology.

[175]  Carlos Mosquera,et al.  Efficient Dynamic Spectrum Sharing in Cognitive Radio Networks: Centralized Dynamic Spectrum Leasing (C-DSL) , 2010, IEEE Transactions on Wireless Communications.

[176]  Christos Christodoulou,et al.  Support Vector Machines for Antenna Array Processing and Electromagnetics , 2006, Support Vector Machines for Antenna Array Processing and Electromagnetics.

[177]  Yang Han,et al.  Jointly cooperative decode-and-forward relaying for secondary spectrum access , 2012, 2012 46th Annual Conference on Information Sciences and Systems (CISS).

[178]  Ryszard S. Michalski,et al.  Learning and Cognition , 1995, WOCFAI.

[179]  H. Vincent Poor,et al.  Reinforcement learning based distributed multiagent sensing policy for cognitive radio networks , 2011, 2011 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN).

[180]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[181]  Marceau Coupechoux,et al.  An Auction Framework for Spectrum Allocation with Interference Constraint in Cognitive Radio Networks , 2010, 2010 Proceedings IEEE INFOCOM.

[182]  Paul Cotae,et al.  Cognitive Radio: Time Domain Spectrum Allocation using Game Theory , 2007, 2007 IEEE International Conference on System of Systems Engineering.

[183]  Dong In Kim,et al.  Game Theoretic Approaches for Multiple Access in Wireless Networks: A Survey , 2011, IEEE Communications Surveys & Tutorials.

[184]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[185]  Kok-Lim Alvin Yau,et al.  Applications of Reinforcement Learning to Cognitive Radio Networks , 2010, 2010 IEEE International Conference on Communications Workshops.

[186]  Bart De Schutter,et al.  Multi-Agent Reinforcement Learning: A Survey , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[187]  Martin A. Riedmiller,et al.  Evaluation of Policy Gradient Methods and Variants on the Cart-Pole Benchmark , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[188]  Dexian Zhang,et al.  Dynamic spectrum access for cognitive radio systems with repeated games , 2010, 2010 IEEE International Conference on Wireless Communications, Networking and Information Security.

[189]  M.J. Sherman,et al.  Survey of IEEE standards supporting cognitive radio and dynamic spectrum access , 2008, MILCOM 2008 - 2008 IEEE Military Communications Conference.

[190]  Hong Jiang,et al.  Design of Learning Engine Based on Support Vector Machine in Cognitive Radio , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[191]  K. B. Letaief,et al.  Optimization of cooperative spectrum sensing with energy detection in cognitive radio networks , 2009, IEEE Transactions on Wireless Communications.

[192]  Vijay K. Bhargava,et al.  Design of OMC-MAC: An Opportunistic Multi-Channel MAC with QoS Provisioning for Distributed Cognitive Radio Networks , 2011, IEEE Transactions on Wireless Communications.

[193]  Xiaohu You,et al.  Efficient Channel Estimation for MIMO Single-Carrier Block Transmission With Dual Cyclic Timeslot Structure , 2007, IEEE Transactions on Communications.

[194]  D. Freedman On the Asymptotic Behavior of Bayes' Estimates in the Discrete Case , 1963 .