Multi-Autonomous Robot Enhanced Ad-Hoc Network under Uncertain and Vulnerable Environment

This paper studies the problem of real-time routing in a multi-autonomous robot enhanced network at uncertain and vulnerable tactical edge. Recent network protocols, such as opportunistic mobile network routing protocols, engaged social network in communication network that can increase the interoperability by using social mobility and opportunistic carry and forward routing algorithms. However, in practical harsh environment such as a battlefield, the uncertainty of social mobility and complexity of vulnerable environment due to unpredictable physical and cyberattacks from enemy, would seriously affect the effectiveness and practicality of these emerging network protocols. This paper presents a GT-SaREMANET (Game Theoretic Situation-aware Robot Enhanced Mobile Adhoc Network) routing protocol that adopt the online reinforcement learning technique to supervise the mobility of multi-robots as well as handle the uncertainty and potential physical and cyber attack at tactical edge. Firstly, a set of game theoretic mission oriented metrics has been introduced to describe the interrelation among network quality, multi-robot mobility as well as potential attacking activities. Then, a distributed multi-agent game theoretic reinforcement learning algorithm has been developed. It will not only optimize GT-SaRE-MANET routing protocol and the mobility of multirobots online, but also effectively avoid the physical and/or cyber-attacks from enemy by using the game theoretic mission oriented metrics. The effectiveness of proposed design has been demonstrated through computer aided simulations and hardware experiments. key words: reinforcement learning, game theory, mobile ad-hoc network, mission oriented metrics, multi-agent systems

[1]  Scott Burleigh,et al.  Networking in Interstellar Dimensions: Communicating With TRAPPIST-1 , 2019, IEEE Transactions on Aerospace and Electronic Systems.

[2]  Tomoyuki Ohta,et al.  An Inter-Domain Routing Protocol Based on Autonomous Clustering for Heterogeneous Mobile Ad Hoc Networks , 2015, IEICE Trans. Commun..

[3]  Kevin R. Fall,et al.  A delay-tolerant network architecture for challenged internets , 2003, SIGCOMM '03.

[4]  Michael L. Littman,et al.  A Distributed Reinforcement Learning Scheme for Network Routing , 1993 .

[5]  Guilin Chen,et al.  Throughput Capacity Study for MANETs with Erasure Coding and Packet Replication , 2015, IEICE Trans. Commun..

[6]  Amin Vahdat,et al.  Epidemic Routing for Partially-Connected Ad Hoc Networks , 2009 .

[7]  Boleslaw K. Szymanski,et al.  Multilayer MANET routing with social-cognitive learning , 2017, MILCOM 2017 - 2017 IEEE Military Communications Conference (MILCOM).

[8]  Robert W. Heath,et al.  MmWave Vehicle-to-Infrastructure Communication: Analysis of Urban Microcellular Networks , 2017, IEEE Transactions on Vehicular Technology.

[9]  Marco Conti,et al.  Opportunistic networking: data forwarding in disconnected mobile ad hoc networks , 2006, IEEE Communications Magazine.

[10]  Marília Curado,et al.  A reinforcement learning-based routing for delay tolerant networks , 2013, Eng. Appl. Artif. Intell..

[11]  Pin-Han Ho,et al.  ARBR: Adaptive reinforcement-based routing for DTN , 2010, 2010 IEEE 6th International Conference on Wireless and Mobile Computing, Networking and Communications.

[12]  Philippe Jacquet,et al.  Optimized Link State Routing Protocol (OLSR) , 2003, RFC.

[13]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[14]  Matthew Sprinkle Design Considerations in a Modern Land Mobile Radio System , 2003 .

[15]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[16]  Shimon Whiteson,et al.  Learning to Communicate with Deep Multi-Agent Reinforcement Learning , 2016, NIPS.

[17]  Pan Hui,et al.  BUBBLE Rap: Social-Based Forwarding in Delay-Tolerant Networks , 2008, IEEE Transactions on Mobile Computing.

[18]  Boleslaw K. Szymanski,et al.  Exploiting Friendship Relations for Efficient Routing in Mobile Social Networks , 2012, IEEE Transactions on Parallel and Distributed Systems.

[19]  Tanveer A. Zia,et al.  A multi-hop cross layer decision based routing for VANETs , 2015, Wirel. Networks.

[20]  Niccolo Leo Caldararo Wild Fire, Home Survival, Defensive Space and Self-Fulfilling Prophesy , 2008 .

[21]  Mads Haahr,et al.  Social network analysis for routing in disconnected delay-tolerant MANETs , 2007, MobiHoc '07.

[22]  Yasuhiko Matsunaga,et al.  Relay Selection Scheme Based on Path Throughput for Device-to-Device Communication in Public Safety LTE , 2018, IEICE Trans. Commun..

[23]  Hideki Tode,et al.  A Design of Wide Area MANET by Dynamic Linkage with IP-Based Infrastructure , 2009, IEICE Trans. Commun..