Reinforcement Learning-Based Spectrum Management for Cognitive Radio Networks: A Literature Review and Case Study

[1]  Lisandro Zambenedetti Granville,et al.  Improving reinforcement learning algorithms for dynamic spectrum allocation in cognitive sensor networks , 2013, 2013 IEEE Wireless Communications and Networking Conference (WCNC).

[2]  Xianfu Chen,et al.  Stochastic Power Adaptation with Multiagent Reinforcement Learning for Cognitive Wireless Mesh Networks , 2013, IEEE Transactions on Mobile Computing.

[3]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[4]  Luiz A. DaSilva,et al.  Complexity of Spectrum Activity and Benefits of Reinforcement Learning for Dynamic Channel Selection , 2013, IEEE Journal on Selected Areas in Communications.

[5]  Bart De Schutter,et al.  Multi-Agent Reinforcement Learning: A Survey , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[6]  Zhu Han,et al.  A Survey on Applications of Model-Free Strategy Learning in Cognitive Wireless Networks , 2015, IEEE Communications Surveys & Tutorials.

[7]  Pawel A. Dmochowski,et al.  Analysis and implementation of reinforcement learning on a GNU Radio cognitive radio platform , 2010, 2010 Proceedings of the Fifth International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[8]  Wahidah Hashim,et al.  A reinforcement learning-based routing scheme for cognitive radio ad hoc networks , 2014, 2014 7th IFIP Wireless and Mobile Networking Conference (WMNC).

[9]  Ian F. Akyildiz,et al.  Reinforcement learning-based cooperative sensing in cognitive radio ad hoc networks , 2010, 21st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications.

[10]  Apurva N. Mody,et al.  IEEE Standards Supporting Cognitive Radio and Networks, Dynamic Spectrum Access, and Coexistence , 2008, IEEE Communications Magazine.

[11]  Marco Di Felice,et al.  SEARCH: A routing protocol for mobile cognitive radio ad-Hoc networks , 2009, 2009 IEEE Sarnoff Symposium.

[12]  Cheng-Xiang Wang,et al.  Reinforcement learning approaches and evaluation criteria for opportunistic spectrum access , 2014, 2014 IEEE International Conference on Communications (ICC).

[13]  Ian F. Akyildiz,et al.  Cooperation Reliability Based on Reinforcement Learning for Cognitive Radio Networks , 2010, 2010 Fifth IEEE Workshop on Networking Technologies for Software Defined Radio Networks (SDR).

[14]  Daniel Kudenko,et al.  Reinforcement learning of coordination in cooperative multi-agent systems , 2002, AAAI/IAAI.

[15]  Tamer A. ElBatt,et al.  Distributed Cooperative Q-Learning for Power Allocation in Cognitive Femtocell Networks , 2012, 2012 IEEE Vehicular Technology Conference (VTC Fall).

[16]  Ian F. Akyildiz,et al.  Optimal spectrum sensing framework for cognitive radio networks , 2008, IEEE Transactions on Wireless Communications.

[17]  Zhiyong Feng,et al.  Centralized channel and power allocation for cognitive radio networks: A Q-learning solution , 2010, 2010 Future Network & Mobile Summit.

[18]  Amit Konar,et al.  Multi-agent reinfocement learning for stochastic power management in cognitive radio network , 2016, 2016 International Conference on Microelectronics, Computing and Communications (MicroCom).

[19]  Timothy J. O'Shea,et al.  Applications of Machine Learning to Cognitive Radio Networks , 2007, IEEE Wireless Communications.

[20]  Maaz Rehan,et al.  Effect of reinforcement learning on routing of cognitive radio ad-hoc networks , 2015, 2015 International Symposium on Mathematical Sciences and Computing Research (iSMSC).

[21]  Kok-Lim Alvin Yau,et al.  Context-awareness and intelligence in Distributed Cognitive Radio Networks: A Reinforcement Learning approach , 2010, 2010 Australian Communications Theory Workshop (AusCTW).

[22]  Luciano Bononi,et al.  Adaptive Sensing Scheduling and Spectrum Selection in Cognitive Wireless Mesh Networks , 2011, 2011 Proceedings of 20th International Conference on Computer Communications and Networks (ICCCN).

[23]  Anjali Agarwal,et al.  Resource adaptations for revenue optimization in cognitive mesh network using reinforcement learning , 2010, 2010 IEEE Globecom Workshops.

[24]  Ana Galindo-Serrano,et al.  Distributed Q-Learning for Aggregated Interference Control in Cognitive Radio Networks , 2010, IEEE Transactions on Vehicular Technology.

[25]  José Ferreira de Rezende,et al.  Channel sensing order for cognitive radio networks using reinforcement learning , 2011, 2011 IEEE 36th Conference on Local Computer Networks.

[26]  Kok-Lim Alvin Yau,et al.  Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues , 2012, J. Netw. Comput. Appl..

[27]  Mischa Dohler,et al.  Learning from experts in cognitive radio networks: The docitive paradigm , 2010, 2010 Proceedings of the Fifth International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[28]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[29]  H. Vincent Poor,et al.  Exploiting spatial diversity in multiagent reinforcement learning based spectrum sensing , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[30]  Daniel Kudenko,et al.  Reinforcement Learning Approaches to Coordination in Cooperative Multi-agent Systems , 2002, Adaptive Agents and Multi-Agents Systems.

[31]  Masoud Sabaei,et al.  Non-cooperative reinforcement learning based routing in cognitive radio networks , 2017, Comput. Commun..

[32]  Kok-Lim Alvin Yau,et al.  Achieving Context Awareness and Intelligence in Distributed Cognitive Radio Networks: A Payoff Propagation Approach , 2011, 2011 IEEE Workshops of International Conference on Advanced Information Networking and Applications.

[33]  Kok-Lim Alvin Yau,et al.  Route Selection for Multi-Hop Cognitive Radio Networks Using Reinforcement Learning: An Experimental Study , 2016, IEEE Access.

[34]  Fatih Alagöz,et al.  Self-evaluating reinforcement learning based spectrum management for cognitive Ad Hoc networks , 2013, The International Conference on Information Networking 2013 (ICOIN).

[35]  Liljana Gavrilovska,et al.  Learning and Reasoning in Cognitive Radio Networks , 2013, IEEE Communications Surveys & Tutorials.

[36]  Sudharman K. Jayaweera,et al.  A Survey on Machine-Learning Techniques in Cognitive Radios , 2013, IEEE Communications Surveys & Tutorials.

[37]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[38]  David Grace,et al.  Efficient exploration in reinforcement learning-based cognitive radio spectrum sharing , 2011, IET Commun..

[39]  Luciano Bononi,et al.  Cooperation and communication in Cognitive radio networks based on TV spectrum experiments , 2011, 2011 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks.

[40]  Joseph Mitola,et al.  Cognitive Radio An Integrated Agent Architecture for Software Defined Radio , 2000 .

[41]  R. Agrawal Sample mean based index policies by O(log n) regret for the multi-armed bandit problem , 1995, Advances in Applied Probability.

[42]  Kok-Lim Alvin Yau,et al.  Enhancing network performance in Distributed Cognitive Radio Networks using single-agent and multi-agent Reinforcement Learning , 2010, IEEE Local Computer Network Conference.

[43]  Husheng Li,et al.  Multi-objective reinforcement learning based routing in cognitive radio networks: Walking in a random maze , 2012, 2012 International Conference on Computing, Networking and Communications (ICNC).

[44]  Chen-Khong Tham,et al.  Reinforcement learning and CMAC-based adaptive routing for MANETs , 2004, Proceedings. 2004 12th IEEE International Conference on Networks (ICON 2004) (IEEE Cat. No.04EX955).

[45]  H. Vincent Poor,et al.  Multiagent Reinforcement Learning Based Spectrum Sensing Policies for Cognitive Radio Networks , 2013, IEEE Journal of Selected Topics in Signal Processing.

[46]  Husheng Li,et al.  Multi-Agent Q-Learning for Competitive Spectrum Access in Cognitive Radio Systems , 2010, 2010 Fifth IEEE Workshop on Networking Technologies for Software Defined Radio Networks (SDR).

[47]  Mohsen Guizani,et al.  Opportunistic Bandwidth Sharing Through Reinforcement Learning , 2010, IEEE Transactions on Vehicular Technology.

[48]  Ian F. Akyildiz,et al.  NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey , 2006, Comput. Networks.

[49]  Christophe Moy,et al.  Reinforcement learning demonstrator for opportunistic spectrum access on real radio signals , 2015, 2015 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN).

[50]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[51]  Daesik Hong,et al.  QoS-guaranteed transmission mode selection for efficient resource utilization in multi-hop cellular networks , 2008, IEEE Transactions on Wireless Communications.

[52]  Mei Song,et al.  Reinforcement Learning Based Auction Algorithm for Dynamic Spectrum Access in Cognitive Radio Networks , 2010, 2010 IEEE 72nd Vehicular Technology Conference - Fall.

[53]  Jim Dowling,et al.  Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[54]  P. Dayan,et al.  Reinforcement learning: The Good, The Bad and The Ugly , 2008, Current Opinion in Neurobiology.

[55]  Fangwen Fu,et al.  Detection of Spectral Resources in Cognitive Radios Using Reinforcement Learning , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[56]  Santosh Pandey,et al.  IEEE 802.11af: a standard for TV white space spectrum sharing , 2013, IEEE Communications Magazine.

[57]  Kok-Lim Alvin Yau,et al.  Achieving Efficient and Optimal Joint Action in Distributed Cognitive Radio Networks Using Payoff Propagation , 2010, 2010 IEEE International Conference on Communications.

[58]  Randy Paffenroth,et al.  Multi-objective reinforcement learning-based deep neural networks for cognitive space communications , 2017, 2017 Cognitive Communications for Aerospace Applications Workshop (CCAA).

[59]  Oriol Sallent,et al.  Distributed spectrum management based on reinforcement learning , 2009, 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[60]  Suzan Bayhan,et al.  CooperativeQ: Energy-efficient channel access based on cooperative reinforcement learning , 2015, 2015 IEEE International Conference on Communication Workshop (ICCW).

[61]  Linda Doyle,et al.  Learning Nash Equilibria in Distributed Channel Selection for Frequency-Agile Radios , 2012 .

[62]  Kobi Cohen,et al.  Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access , 2017, IEEE Transactions on Wireless Communications.

[63]  Kok-Lim Alvin Yau,et al.  Applications of Reinforcement Learning to Cognitive Radio Networks , 2010, 2010 IEEE International Conference on Communications Workshops.

[64]  Mohammad S. Obaidat,et al.  A Reinforcement learning-based cognitive MAC protocol , 2015, 2015 IEEE International Conference on Communications (ICC).

[65]  Luciano Bononi,et al.  End-to-end protocols for Cognitive Radio Ad Hoc Networks: An evaluation study , 2011, Perform. Evaluation.

[66]  Yang Yang,et al.  Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks , 2009, 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications.

[67]  Hüseyin Arslan,et al.  A survey of spectrum sensing algorithms for cognitive radio applications , 2009, IEEE Communications Surveys & Tutorials.

[68]  L. Luo,et al.  Editorial overview , 2008, Current Opinion in Neurobiology.

[69]  Robin J. Evans,et al.  Reinforcement learning based secondary user transmissions in cognitive radio networks , 2013, 2013 IEEE Globecom Workshops (GC Wkshps).

[70]  Cheng Wu,et al.  Learning-Based Spectrum Selection in Cognitive Radio Ad Hoc Networks , 2010, WWIC.

[71]  Yusun Chang,et al.  Learning through Reinforcement for Repeated Power Control Game in Cognitive Radio Networks , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[72]  Yenumula B. Reddy Detecting Primary Signals for Efficient Utilization of Spectrum Using Q-Learning , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[73]  Nazanin Rahnavard,et al.  A Learning-Based QoE-Driven Spectrum Handoff Scheme for Multimedia Transmissions over Cognitive Radio Networks , 2014, IEEE Journal on Selected Areas in Communications.

[74]  Cheng Wu,et al.  Spectrum management of cognitive radio using multi-agent reinforcement learning , 2010, AAMAS.

[75]  Francesca Cuomo,et al.  Routing in cognitive radio networks: Challenges and solutions , 2011, Ad Hoc Networks.

[76]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.