A dynamic route change mechanism for mobile ad hoc networks

A mobile ad hoc network (MANET) is a self-configuring network of mobile devices connected by wireless links. Frequent topology changes and limited bandwidth make communication in MANETs particularly challenging. We present a MANET routing protocol that can efficiently handle network mobility by a way of preemptively switching to a better route before the current route fails. The protocol uses a distributed Q-learning algorithm to infer network status information and takes in to consideration link stability and bandwidth efficiency while selecting a route. We study the performance of this protocol through simulation and demonstrate its advantages over existing protocols.

[1]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[2]  Charles E. Perkins,et al.  Ad hoc On-Demand Distance Vector (AODV) Routing , 2001, RFC.

[3]  Alexander Zelinsky,et al.  Q-Learning in Continuous State and Action Spaces , 1999, Australian Joint Conference on Artificial Intelligence.

[4]  Chiu-Kuo Liang,et al.  An ad hoc on-demand routing protocol with high packet delivery fraction , 2004, 2004 IEEE International Conference on Mobile Ad-hoc and Sensor Systems (IEEE Cat. No.04EX975).

[5]  Subir Biswas,et al.  Neighborhood Route Diffusion for Packet Salvaging in Networks with High Mobility , 2008, 2008 IEEE International Performance, Computing and Communications Conference.

[6]  Javier A. Barria,et al.  Reinforcement Learning for Resource Allocation in LEO Satellite Networks , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Celimuge Wu,et al.  A MANET protocol considering link stability and bandwidth efficiency , 2009, 2009 International Conference on Ultra Modern Telecommunications & Workshops.

[8]  Jim Dowling,et al.  Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  Shervin Erfani,et al.  Survey of multipath routing protocols for mobile ad hoc networks , 2009, J. Netw. Comput. Appl..

[10]  Celimuge Wu,et al.  An enhanced reinforcement routing protocol for inter-vehicular unicast application , 2008, EuroIMSA 2008.

[11]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[12]  Javier A. Barria,et al.  A reinforcement learning ticket-based probing path discovery scheme for MANETs , 2004, Ad Hoc Networks.

[13]  Leslie Pack Kaelbling,et al.  Mobilized ad-hoc networks: a reinforcement learning approach , 2004 .

[14]  Javier A. Barria,et al.  QoS Routing in MANETS with Imprecise Information Using Actor-Critic Reinforcement Learning , 2007, 2007 IEEE Wireless Communications and Networking Conference.

[15]  Yong Wang,et al.  Predicting link quality using supervised learning in wireless sensor networks , 2007, MOCO.

[16]  Mahesh K. Marina,et al.  On-demand multipath distance vector routing in ad hoc networks , 2001, Proceedings Ninth International Conference on Network Protocols. ICNP 2001.

[17]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[18]  Luigi Paura,et al.  Bio-inspired link quality estimation for wireless mesh networks , 2009, 2009 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks & Workshops.

[19]  Tei-Wei Kuo,et al.  A Maximum-Residual Multicast Protocol for Large-Scale Mobile Ad Hoc Networks , 2009, IEEE Transactions on Mobile Computing.

[20]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.