A Learning-based Adaptive Routing Tree for Wireless Sensor Networks

One of the most common communication patterns in sensor networks is routing data to a base station, while the base station can be either static or mobile. Even in static cases, a static spanning tree may not survive for a long time due to failures of sensor nodes. In this paper, we present an adaptive spanning tree routing mechanism, using real-time reinforcement learning strategies. We demonstrate via simulation that without additional control packets for tree maintenance, adaptive spanning trees can maintain the "best" connectivity to the base station, in spite of node failures or mobility of the base station. And by using a general constraint-based routing specification, one can apply the same strategy to achieve load balancing and to control network congestion effectively in real time.

[1]  Ying Zhang,et al.  Smart Routing with Learning-Based QoS-Aware Meta-strategies , 2004, QofIS.

[2]  Deborah Estrin,et al.  Geographical and Energy Aware Routing: a recursive data dissemination protocol for wireless sensor networks , 2002 .

[3]  A. Arora,et al.  Routing on a Logical Grid in Sensor Networks , 2004 .

[4]  Songwu Lu,et al.  A Robust Data Delivery Protocol for Large Scale Sensor Networks , 2003, IPSN.

[5]  Satish K. Tripathi,et al.  Signal stability-based adaptive routing (SSA) for ad hoc mobile networks , 1997, IEEE Wirel. Commun..

[6]  Brad Karp,et al.  GPSR: greedy perimeter stateless routing for wireless networks , 2000, MobiCom '00.

[7]  Deborah Estrin,et al.  Highly-resilient, energy-efficient multipath routing in wireless sensor networks , 2001, MOCO.

[8]  Chai-Keong Toh,et al.  A novel distributed routing protocol to support ad-hoc mobile computing , 1996, Conference Proceedings of the 1996 IEEE Fifteenth Annual International Phoenix Conference on Computers and Communications.

[9]  Ying Zhang,et al.  Message-initiated constraint-based routing for wireless ad-hoc sensor networks , 2004, First IEEE Consumer Communications and Networking Conference, 2004. CCNC 2004..

[10]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[11]  Sergio D. Servetto,et al.  Constrained random walks on random graphs: routing algorithms for large scale wireless sensor networks , 2002, WSNA '02.

[12]  Ying Zhang,et al.  Adaptive tree: a learning-based meta-routing strategy for sensor networks , 2006, CCNC 2006. 2006 3rd IEEE Consumer Communications and Networking Conference, 2006..

[13]  Bruce H. Krogh,et al.  Energy-efficient surveillance system using wireless sensor networks , 2004, MobiSys '04.

[14]  Ying Zhang,et al.  Information-Directed Routing in Sensor Networks Using Real-Time Reinforcement Learning , 2006 .

[15]  David E. Culler,et al.  Taming the underlying challenges of reliable multihop routing in sensor networks , 2003, SenSys '03.

[16]  Ying Zhang,et al.  Search-based Adaptive Routing Strategies for Sensor Networks , 2004 .

[17]  M. S. Corson,et al.  A highly adaptive distributed routing algorithm for mobile wireless networks , 1997, Proceedings of INFOCOM '97.

[18]  Deborah Estrin,et al.  Directed diffusion: a scalable and robust communication paradigm for sensor networks , 2000, MobiCom '00.

[19]  Elizabeth M. Belding-Royer,et al.  A review of current routing protocols for ad hoc mobile wireless networks , 1999, IEEE Wirel. Commun..

[20]  Risto Miikkulainen,et al.  Confidence-based Q-Routing: An on-line adaptive network routing algorithm , 1998 .

[21]  Robert Poor.  Gradient Routing in Ad Hoc Networks , 2000 .

[22]  David A. Maltz,et al.  Dynamic Source Routing in Ad Hoc Wireless Networks , 1994, Mobidata.

[23]  Ying Zhang,et al.  Improvements on Ant Routing for Sensor Networks , 2004, ANTS Workshop.

[24]  Reid G. Simmons,et al.  Complexity Analysis of Real-Time Reinforcement Learning , 1993, AAAI.

[25]  Feng Zhao,et al.  Scalable Information-Driven Sensor Querying and Routing for Ad Hoc Heterogeneous Sensor Networks , 2002, Int. J. High Perform. Comput. Appl..

[26]  Ying Zhang,et al.  Combs, needles, haystacks: balancing push and pull for discovery in large-scale sensor networks , 2004, SenSys '04.

[27]  R. D. Rockwell,et al.  Smart Packets for active networks , 1999, 1999 IEEE Second Conference on Open Architectures and Network Programming. Proceedings. OPENARCH '99 (Cat. No.99EX252).

[28]  Ying Zhang,et al.  High-Level Sensor Network Simulations for Routing Performance Evaluations , 2006 .

[29]  Ying Zhang,et al.  Constrained flooding: a robust and efficient routing framework for wireless sensor networks , 2006, 20th International Conference on Advanced Information Networking and Applications - Volume 1 (AINA'06).

[30]  Seungjoon Lee,et al.  Efficient geographic routing in multihop wireless networks , 2005, MobiHoc '05.

[31]  Ramesh Govindan,et al.  Understanding packet delivery performance in dense wireless sensor networks , 2003, SenSys '03.

[32]  Charles E. Perkins,et al.  Ad-hoc on-demand distance vector routing , 1999, Proceedings WMCSA'99. Second IEEE Workshop on Mobile Computing Systems and Applications.

[33]  Devika Subramanian,et al.  Ants and Reinforcement Learning: A Case Study in Routing in Dynamic Networks , 1997, IJCAI.

[34]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.