A Double Q-Learning Routing in Delay Tolerant Networks

Delay tolerant networks (DTNs) are wireless mobile networks, where the nodes are sparse and end-to-end connectivity is rare. The intermittent connectivity in DTNs makes it challenging to efficiently deliver messages. Research results have shown that the routing protocol based on reinforcement learning can achieve a reasonable balance between routing performance and cost. However, how to predict the next hop of messages more accurately is still open. In this paper, Double Q-Learning Routing (DQLR) protocol is proposed, which investigates the routing selection of the next hop in a distributed manner and solves the overestimation problem by Double Q-Learning algorithm. Further, the intermediate value and dynamic reward mechanisms are proposed to adapt node mobility and network topology change, which improve the network performance. The simulation results show that DQLR protocol can increase the delivery ratio with a low overhead.

[1]  Song Guo,et al.  Big Data Meet Green Challenges: Big Data Toward Green Applications , 2016, IEEE Systems Journal.

[2]  Hado van Hasselt,et al.  Double Q-learning , 2010, NIPS.

[3]  Rabin K. Patra,et al.  Routing in a delay tolerant network , 2004, SIGCOMM '04.

[4]  Katia Obraczka,et al.  A survey on congestion control for delay and disruption tolerant networks , 2015, Ad Hoc Networks.

[5]  Yunfeng Lin,et al.  Performance modeling of network coding in epidemic routing , 2007, MobiOpp '07.

[6]  Pan Hui,et al.  BUBBLE Rap: Social-Based Forwarding in Delay-Tolerant Networks , 2008, IEEE Transactions on Mobile Computing.

[7]  Cherry Wakayama,et al.  Utilizing kinematics and selective sweeping in reinforcement learning-based routing algorithms for underwater networks , 2015, Ad Hoc Networks.

[8]  Xianfu Chen,et al.  Energy-Efficiency Oriented Traffic Offloading in Wireless Networks: A Brief Survey and a Learning Approach for Heterogeneous Cellular Networks , 2015, IEEE Journal on Selected Areas in Communications.

[9]  Giancarlo Fortino,et al.  WSNs-assisted opportunistic network for low-latency message forwarding in sparse settings , 2019, Future Gener. Comput. Syst..

[10]  Marília Curado,et al.  A reinforcement learning-based routing for delay tolerant networks , 2013, Eng. Appl. Artif. Intell..

[11]  Yi Yang,et al.  Big Data Meet Cyber-Physical Systems: A Panoramic Survey , 2018, IEEE Access.

[12]  M. Bellafkih,et al.  AntProPHET: A new routing protocol for delay tolerant networks , 2014, Proceedings of 2014 Mediterranean Microwave Symposium (MMS2014).

[13]  Celimuge Wu,et al.  Distributed Reinforcement Learning Approach for Vehicular Ad Hoc Networks , 2010, IEICE Trans. Commun..

[14]  Subir Biswas,et al.  Learning based gain-aware content dissemination in delay tolerant networks , 2017, 2017 9th International Conference on Communication Systems and Networks (COMSNETS).

[15]  Tzu-Chieh Tsai,et al.  NCCU Trace: social-network-aware mobility trace , 2015, IEEE Communications Magazine.

[16]  Nikhil Kumar Singh,et al.  Survey of Routing in Delay Tolerant Networks , 2017 .

[17]  Pin-Han Ho,et al.  ARBR: Adaptive reinforcement-based routing for DTN , 2010, 2010 IEEE 6th International Conference on Wireless and Mobile Computing, Networking and Communications.

[18]  Jörg Ott,et al.  Working day movement model , 2008, MobilityModels '08.

[19]  Jie Wu,et al.  Predict and relay: an efficient routing in disruption-tolerant networks , 2009, MobiHoc '09.

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Zhensheng Zhang,et al.  Routing in intermittently connected mobile ad hoc networks and delay tolerant networks: overview and challenges , 2006, IEEE Communications Surveys & Tutorials.