RescueNet: Reinforcement-learning-based communication framework for emergency networking

Abstract A new paradigm for emergency networking is envisioned to enable reliable and high data-rate wireless multimedia communication among public safety agencies in licensed spectrum while causing only acceptable levels of disruption to incumbent network communication. The novel concept of mission policies , which specify the Quality of Service (QoS) requirements of the incumbent networks as well as of the emergency networks involved in rescue and recovery missions, is introduced. The use of mission policies, which vary over time and space, enables graceful degradation in the QoS of the incumbent networks (only when necessary) based on mission policy specifications. A Multi-Agent Reinforcement Learning (MARL)-based cross-layer communication framework, “RescueNet,” is proposed for self-adaptation of nodes in emergency networks based on this new paradigm. In addition to addressing the research challenges posed by the non-stationarity of the problem, the novel idea of knowledge sharing among the agents of different ages (either bootstrapping or selective exploration strategies or both) is introduced to improve significantly the performance of the proposed solution in terms of convergence time and conformance to the mission policies.

[1]  Ian F. Akyildiz,et al.  CRAHNs: Cognitive radio ad hoc networks , 2009, Ad Hoc Networks.

[2]  H. Vincent Poor,et al.  Multiagent Reinforcement Learning Based Spectrum Sensing Policies for Cognitive Radio Networks , 2013, IEEE Journal of Selected Topics in Signal Processing.

[3]  Michael L. Littman,et al.  Value-function reinforcement learning in Markov games , 2001, Cognitive Systems Research.

[4]  Karina Mabell Gomez,et al.  Enabling disaster-resilient 4G mobile communication networks , 2014, IEEE Communications Magazine.

[5]  J.-C. Renaud,et al.  Multi-Agent Systems on Sensor Networks: A Distributed Reinforcement Learning Approach , 2005, 2005 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[6]  Hanif D. Sherali,et al.  Optimal Spectrum Sharing for Multi-Hop Software Defined Radio Networks , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[7]  David S. Broomhead,et al.  Codes for spread spectrum applications generated using chaotic dynamical systems , 1999 .

[8]  Leonard Kleinrock,et al.  Optimal Transmission Ranges for Randomly Distributed Packet Radio Terminals , 1984, IEEE Trans. Commun..

[9]  Yunnan Wu,et al.  KNOWS: Cognitive Radio Networks Over White Spaces , 2007, 2007 2nd IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[10]  Chen-Khong Tham,et al.  Coordinated Sensing Coverage in Sensor Networks using Distributed Reinforcement Learning , 2006, 2006 14th IEEE International Conference on Networks.

[11]  Coleman Bazelon Too many goals: Problems with the 700 MHz auction , 2009, Inf. Econ. Policy.

[12]  Kang G. Shin,et al.  In-band spectrum sensing in cognitive radio networks: energy detection or feature detection? , 2008, MobiCom '08.

[13]  Andreas Pitsillides,et al.  Explore and exploit in wireless ad hoc emergency response networks , 2014, 2014 IEEE International Conference on Communications (ICC).

[14]  Luciano Bononi,et al.  Adaptive Sensing Scheduling and Spectrum Selection in Cognitive Wireless Mesh Networks , 2011, 2011 Proceedings of 20th International Conference on Computer Communications and Networks (ICCCN).

[15]  Lei Ding,et al.  Cross-Layer Routing and Dynamic Spectrum Allocation in Cognitive Radio Ad Hoc Networks , 2010, IEEE Transactions on Vehicular Technology.

[16]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[17]  Ana Galindo-Serrano,et al.  Distributed Q-Learning for Aggregated Interference Control in Cognitive Radio Networks , 2010, IEEE Transactions on Vehicular Technology.

[18]  Joseph Mitola,et al.  Cognitive radio: making software radios more personal , 1999, IEEE Wirel. Commun..

[19]  I.G.M.M. Niemegeers,et al.  Cognitive radio emergency networks - requirements and design , 2005, First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005..

[20]  Wei Liu,et al.  Joint On-Demand Routing and Spectrum Assignment in Cognitive Radio Networks , 2007, 2007 IEEE International Conference on Communications.

[21]  Victor C. M. Leung,et al.  A novel cooperative communication protocol for QoS provisioning in wireless sensor networks , 2009, 2009 5th International Conference on Testbeds and Research Infrastructures for the Development of Networks & Communities and Workshops.

[22]  Dario Pompili,et al.  Learning-based framework for policy-aware cognitive radio emergency networking , 2013, 2013 IEEE Global Communications Conference (GLOBECOM).

[23]  Dario Pompili,et al.  A Multimedia Cross-Layer Protocol for Underwater Acoustic Sensor Networks , 2010, IEEE Transactions on Wireless Communications.

[24]  Seungjoon Lee,et al.  Network function virtualization: Challenges and opportunities for innovations , 2015, IEEE Communications Magazine.

[25]  Lennard G. Kruger The First Responder Network (FirstNet) and Next-Generation Communications for Public Safety: Issues for Congress [January 26, 2017] , 2017 .

[26]  Terence D. Todd,et al.  Secondary Wireless Mesh Network Design Using Leased Frequency Spectra , 2010, 2010 IEEE Wireless Communication and Networking Conference.

[27]  Luciano Bononi,et al.  Cooperation and communication in Cognitive radio networks based on TV spectrum experiments , 2011, 2011 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks.

[28]  Victor C. M. Leung,et al.  Cooperative Communications with Relay Selection for QoS Provisioning in Wireless Sensor Networks , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[29]  R. Chandramouli,et al.  Stochastic Learning Solution for Distributed Discrete Power Control Game in Wireless Data Networks , 2008, IEEE/ACM Transactions on Networking.

[30]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[31]  Wei Zhou,et al.  DistressNet: a wireless ad hoc and sensor network architecture for situation management in disaster response , 2010, IEEE Communications Magazine.

[32]  Kenichi Mase How to deliver your message from/to a disaster area , 2011, IEEE Communications Magazine.

[33]  H. Vincent Poor,et al.  Cognitive Medium Access: Exploration, Exploitation, and Competition , 2007, IEEE Transactions on Mobile Computing.

[34]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[35]  Andrew W. Moore,et al.  Distributed Value Functions , 1999, ICML.

[36]  Linda K. Moore The First Responder Network and Next-Generation Communications for Public Safety: Issues for Congress , 2014 .