Preventing instability in full echo Q-routing with adaptive learning rates

A routing algorithm based on Q-routing paradigm is proposed for ad-hoc dynamically changing networks. The technique derived from Full Echo approach is used to enhance exploration capacity and prevent instability of routing under high load conditions. The performance of routing is increased by random polling of neighbors according to the local estimates of the average delivery time in the network.

[1]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[2]  Yuliya Shilova,et al.  Full Echo Q-routing with adaptive learning rates: A reinforcement learning approach to network routing , 2016, 2016 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (EIConRusNW).

[3]  Leonid Peshkin,et al.  Reinforcement learning for adaptive routing , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[4]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[5]  Douglas R. Heisterkamp,et al.  Simulated Annealing Based Hierarchical Q-Routing: A Dynamic Routing Protocol , 2011, 2011 Eighth International Conference on Information Technology: New Generations.

[6]  Said Hoceini,et al.  K-Shortest Paths Q-Routing: A New QoS Routing Algorithm in Telecommunication Networks , 2005, ICN.

[7]  John A. Stankovic.  Wireless Sensor Networks , 2008, Computer.

[8]  Dit-Yan Yeung,et al.  Predictive Q-Routing: A Memory-based Reinforcement Learning Approach to Adaptive Traffic Control , 1995, NIPS.

[9]  Badieh Traboulsi,et al.  MANET with the Q-Routing Protocol , 2012 .

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Shailesh Kumar and Risto Miikkulainen.  Dual Reinforcement Q-Routing: An On-Line Adaptive Routing Algorithm , 1997 .

[12]  Antonio Iera,et al.  The Internet of Things: A survey , 2010, Comput. Networks.

[13]  Devika Subramanian,et al.  Ants and Reinforcement Learning: A Case Study in Routing in Dynamic Networks , 1997, IJCAI.