Learning and Reasoning in Cognitive Radio Networks

Cognitive radio networks challenge the traditional wireless networking paradigm by introducing concepts firmly stemmed into the Artificial Intelligence (AI) field, i.e., learning and reasoning. This fosters optimal resource usage and management allowing a plethora of potential applications such as secondary spectrum access, cognitive wireless backbones, cognitive machine-to-machine etc. The majority of overview works in the field of cognitive radio networks deal with the notions of observation and adaptations, which are not a distinguished cognitive radio networking aspect. Therefore, this paper provides insight into the mechanisms for obtaining and inferring knowledge that clearly set apart the cognitive radio networks from other wireless solutions.

[1]  Cheng Wu,et al.  Spectrum management of cognitive radio using multi-agent reinforcement learning , 2010, AAMAS.

[2]  S. Hart,et al.  A simple adaptive procedure leading to correlated equilibrium , 2000 .

[3]  Daniel Denkovski,et al.  Policy enforced spectrum sharing for unaware secondary systems , 2011, CogART '11.

[4]  Jeffrey H. Reed,et al.  Convergence of cognitive radio networks , 2004, 2004 IEEE Wireless Communications and Networking Conference (IEEE Cat. No.04TH8733).

[5]  Kok-Lim Alvin Yau,et al.  Performance Analysis of Reinforcement Learning for Achieving Context Awareness and Intelligence in Mobile Cognitive Radio Networks , 2011, 2011 IEEE International Conference on Advanced Information Networking and Applications.

[6]  Dusit Niyato,et al.  Dynamics of Network Selection in Heterogeneous Wireless Networks: An Evolutionary Game Approach , 2009, IEEE Transactions on Vehicular Technology.

[7]  Allen B. MacKenzie,et al.  Game Theory for Wireless Engineers (Synthesis Lectures on Communications) , 2006 .

[8]  Manuela Veloso,et al.  An Analysis of Stochastic Game Theory for Multiagent Reinforcement Learning , 2000 .

[9]  Matteo Cesana,et al.  On Spectrum Selection Games in Cognitive Radio Networks , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[10]  David Grace,et al.  Efficient exploration in reinforcement learning-based cognitive radio spectrum sharing , 2011, IET Commun..

[11]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[12]  Joanne E McEntee.  Information is available , 2008 .

[13]  John E. Hummel,et al.  Finding similarity in a model of relational reasoning , 2009, Cognitive Systems Research.

[14]  Yoav Shoham,et al.  Multi-Agent Reinforcement Learning:a critical survey , 2003 .

[15]  Linda Doyle,et al.  Learning Nash Equilibria in Distributed Channel Selection for Frequency-Agile Radios , 2012 .

[16]  Nelson Luis Saldanha da Fonseca,et al.  Identifying Relevant Cross-Layer Interactions in Cognitive Processes , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[17]  Harsh K. Verma,et al.  Analysis of Decision Making Operation In Cognitive Radio Using Fuzzy Logic System , 2010 .

[18]  Linda Doyle,et al.  An Encapsulation for Reasoning, Learning, Knowledge Representation, and Reconfiguration Cognitive Radio Elements , 2006, 2006 1st International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[19]  Ian F. Akyildiz,et al.  Reinforcement learning-based cooperative sensing in cognitive radio ad hoc networks , 2010, 21st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications.

[20]  Gerald Tesauro,et al.  Extending Q-Learning to General Adaptive Multi-Agent Systems , 2003, NIPS.

[21]  Daniel Denkovski,et al.  Novel Policy Reasoning Architecture for Cognitive Radio Environments , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[22]  Kok-Lim Alvin Yau,et al.  A context-aware and Intelligent Dynamic Channel Selection scheme for cognitive radio networks , 2009, 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[23]  Ananthram Swami,et al.  Distributed Algorithms for Learning and Cognitive Medium Access with Logarithmic Regret , 2010, IEEE Journal on Selected Areas in Communications.

[24]  Allen B. MacKenzie,et al.  The price of ignorance: distributed topology control in cognitive networks , 2010, IEEE Transactions on Wireless Communications.

[25]  V. V. Phansalkar,et al.  Decentralized Learning of Nash Equilibria in Multi-Person Stochastic Games With Incomplete Information , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[26]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[27]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[28]  Fabrizio Granelli,et al.  Towards a Model for Quantitative Reasoning in Cognitive Nodes , 2009, 2009 IEEE Globecom Workshops.

[29]  An He,et al.  Development of a case-based reasoning cognitive engine for IEEE 802.22 WRAN applications , 2009, MOCO.

[30]  Mihaela van der Schaar,et al.  Learning to Compete for Resources in Wireless Stochastic Games , 2009, IEEE Transactions on Vehicular Technology.

[31]  Cristina Comaniciu,et al.  Adaptive Channel Allocation Spectrum Etiquette for Cognitive Radio Networks , 2005, First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005..

[32]  Joseph Mitola,et al.  Cognitive Radio An Integrated Agent Architecture for Software Defined Radio , 2000 .

[33]  Carolyn L. Talcott,et al.  CoRaL--Policy Language and Reasoning Techniques for Spectrum Policies , 2007, Eighth IEEE International Workshop on Policies for Distributed Systems and Networks (POLICY'07).

[34]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[35]  Mihaela van der Schaar,et al.  Spectrum Access Games and Strategic Learning in Cognitive Radio Networks for Delay-Critical Applications , 2009, Proceedings of the IEEE.

[36]  Dusit Niyato,et al.  Competitive spectrum sharing in cognitive radio networks: a dynamic game approach , 2008, IEEE Transactions on Wireless Communications.

[37]  Mohammad S. Obaidat,et al.  Adaptive wireless networks using learning automata , 2011, IEEE Wireless Communications.

[38]  Hyun-Jin Choi,et al.  An ontology-based reasoning approach towards energy-aware smart homes , 2011, 2011 IEEE Consumer Communications and Networking Conference (CCNC).

[39]  Jeffrey H. Reed,et al.  Survey of cognitive radio architectures , 2010, Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon).

[40]  Dusit Niyato,et al.  Competitive Pricing for Spectrum Sharing in Cognitive Radio Networks: Dynamic Game, Inefficiency of Nash Equilibrium, and Collusion , 2008, IEEE Journal on Selected Areas in Communications.

[41]  Vikram Krishnamurthy,et al.  Transmission control in cognitive radio as a Markovian dynamic game: Structural result on randomized threshold policies , 2010, IEEE Transactions on Communications.

[42]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[43]  Bart Kosko,et al.  Fuzzy Cognitive Maps , 1986, Int. J. Man Mach. Stud..

[44]  Samson Lasaulce,et al.  Learning equilibria with partial information in decentralized wireless networks , 2011, IEEE Communications Magazine.

[45]  Behnam Bahrak,et al.  BRESAP: A Policy Reasoner for Processing Spectrum Access Policies Represented by Binary Decision Diagrams , 2010, 2010 IEEE Symposium on New Frontiers in Dynamic Spectrum (DySPAN).

[46]  Ahmed Karmouch,et al.  GENp2-3: Circumscriptive Context Reasoning for Automated Network Management Operations , 2006, IEEE Globecom 2006.

[47]  James Ivers,et al.  Reasoning Frameworks , 2005 .

[48]  A. Rubinstein.  Instinctive and Cognitive Reasoning: A Study of Response Times , 2006 .

[49]  H. Peyton Young,et al.  Learning by trial and error , 2009, Games Econ. Behav..

[50]  Leemon C. Baird,et al.  Residual Algorithms: Reinforcement Learning with Function Approximation , 1995, ICML.

[51]  Qusay H. Mahmoud,et al.  Cognitive Networks: Towards Self-Aware Networks , 2007 .

[52]  Hamed S. Al-Raweshidy,et al.  Competitive Spectrum Sharing in Wireless Networks: A Dynamic Non-cooperative Game Approach , 2009, WMNC/PWC.

[53]  D. Sibley.  A Cognitive Framework for Reasoning with Scientific Models , 2009 .

[54]  K. J. Ray Liu,et al.  Evolutionary Game Framework for Behavior Dynamics in Cooperative Spectrum Sensing , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[55]  Samson Lasaulce,et al.  Game Theory and Learning for Wireless Networks: Fundamentals and Applications , 2011 .

[56]  Agnar Aamodt,et al.  Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches , 1994, AI Commun..

[57]  Allen B. MacKenzie,et al.  Joint Power and Channel Minimization in Topology Control: A Cognitive Network Approach , 2007, 2007 IEEE International Conference on Communications.

[58]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[59]  S. Musman.  Using parallel distributed reasoning for monitoring computing networks , 2010, 2010 - MILCOM 2010 MILITARY COMMUNICATIONS CONFERENCE.

[60]  Pin-Han Ho,et al.  Extended Knowledge-Based Reasoning Approach to Spectrum Sensing for Cognitive Radio , 2010, IEEE Transactions on Mobile Computing.

[61]  Michael E. Theologou,et al.  Enhancing cognitive radio systems with robust reasoning , 2008 .

[62]  Ana Galindo-Serrano,et al.  Distributed Q-Learning for Aggregated Interference Control in Cognitive Radio Networks , 2010, IEEE Transactions on Vehicular Technology.

[63]  L. Doyle,et al.  Impact of Cognitive Radio: Recognition and Informed Exploitation of Grey Spectrum Opportunities , 2012, IEEE Vehicular Technology Magazine.

[64]  Daniel Friend,et al.  Cognitive Networks: Foundations to Applications , 2009 .

[65]  K. J. Ray Liu,et al.  Repeated open spectrum sharing game with cheat-proof strategies , 2009, IEEE Transactions on Wireless Communications.

[66]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[67]  Balakrishnan Chandrasekaran,et al.  What are ontologies, and why do we need them? , 1999, IEEE Intell. Syst..