Adaptive Q-Routing with random echo and route memory

Mobile ad hoc networks require routing algorithms that provide high performance in terms of delivery times of packets for dynamically changing topologies under various load conditions. A routing algorithm is proposed which is based on adaptive Q-routing technique with Full Echo extension. The proposed algorithm, called Adaptive Q-routing with Random Echo and Route Memory (AQRERM), has the improved performance in terms of overshoot and settling time of the learning. It also greatly improves stability of routing under conditions of high load for the benchmark example.

[1]  Charles E. Perkins,et al.  Ad-hoc on-demand distance vector routing , 1999, Proceedings WMCSA'99. Second IEEE Workshop on Mobile Computing Systems and Applications.

[2]  Devika Subramanian,et al.  Ants and Reinforcement Learning: A Case Study in Routing in Dynamic Networks , 1997, IJCAI.

[3]  Rahul Desai,et al.  MANET with Q Routing Protocol , 2013 .

[4]  Alex Hinds,et al.  A Review of Routing Protocols for Mobile Ad-Hoc NETworks (MANET) , 2013 .

[5]  Yuliya Shilova,et al.  Full Echo Q-routing with adaptive learning rates: A reinforcement learning approach to network routing , 2016, 2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW).

[6]  Michael L. Littman,et al.  Learning-based route management in wireless ad hoc networks , 2008 .

[7]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[8]  Douglas R. Heisterkamp,et al.  Simulated Annealing Based Hierarchical Q-Routing: A Dynamic Routing Protocol , 2011, 2011 Eighth International Conference on Information Technology: New Generations.

[9]  Badieh Traboulsi,et al.  MANET with the Q-Routing Protocol , 2012 .

[10]  Niyati Gupta,et al.  Improved Route Selection Approaches using Q-learning framework for 2D NoCs , 2015, MES@ISCA.

[11]  Leonid Peshkin,et al.  Reinforcement learning for adaptive routing , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[12]  Mohamed Othman,et al.  An adaptive routing algorithm: enhanced confidence-based Q routing algorithm in network traffic , 2004 .

[13]  Said Hoceini,et al.  K-Shortest Paths Q-Routing: A New QoS Routing Algorithm in Telecommunication Networks , 2005, ICN.

[14]  Shailesh Kumar and Risto Miikkulainen.  Dual Reinforcement Q-Routing: An On-Line Adaptive Routing Algorithm , 1997 .

[15]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[16]  Dit-Yan Yeung,et al.  Predictive Q-Routing: A Memory-based Reinforcement Learning Approach to Adaptive Traffic Control , 1995, NIPS.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Salim Chikhi,et al.  A Survey of Reinforcement Learning Based Routing Protocols for Mobile Ad-Hoc Networks , 2011, WiMo/CoNeCo.

[19]  Mohamad Elzohbi.  Flexible and Scalable Routing Approach for Mobile Ad Hoc Networks by Function Approximation of Q-Learning , 2016 .

[20]  J. Cid-Sueiro,et al.  Q-Probabilistic Routing in Wireless Sensor Networks , 2007, 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information.

[21]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .