A Multi-Agent Framework for Packet Routing in Wireless Sensor Networks

Wireless sensor networks (WSNs) have been widely investigated in recent years. One of the fundamental issues in WSNs is packet routing, because in many application domains, packets have to be routed from source nodes to destination nodes as soon and as energy efficiently as possible. To address this issue, a large number of routing approaches have been proposed. Although every existing routing approach has advantages, they also have some disadvantages. In this paper, a multi-agent framework is proposed that can assist existing routing approaches to improve their routing performance. This framework enables each sensor node to build a cooperative neighbour set based on past routing experience. Such cooperative neighbours, in turn, can help the sensor to effectively relay packets in the future. This framework is independent of existing routing approaches and can be used to assist many existing routing approaches. Simulation results demonstrate the good performance of this framework in terms of four metrics: average delivery latency, successful delivery ratio, number of live nodes and total sensing coverage.

[1]  Cauligi S. Raghavendra,et al.  PEGASIS: Power-efficient gathering in sensor information systems , 2002, Proceedings, IEEE Aerospace Conference.

[2]  Martin Reisslein,et al.  Towards Efficient Wireless Video Sensor Networks: A Survey of Existing Node Architectures and Proposal for A Flexi-WVSNP Design , 2011, IEEE Communications Surveys & Tutorials.

[3]  Bing-Hong Liu,et al.  Virtual-Coordinate-Based Delivery-Guaranteed Routing Protocol in Wireless Sensor Networks , 2008, IEEE/ACM Transactions on Networking.

[4]  Tommi S. Jaakkola,et al.  Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms , 2000, Machine Learning.

[5]  Giorgio Valentini,et al.  Dynamic multi-objective routing algorithm: a multi-objective routing algorithm for the simple hybrid routing protocol on wireless sensor networks , 2010, IET Commun..

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  Kristina Lerman,et al.  Resource allocation in the grid using reinforcement learning , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[8]  A.G. Bakirtzis,et al.  A Reinforcement Learning Algorithm for Market Participants in FTR Auctions , 2007, 2007 IEEE Lausanne Power Tech.

[9]  Majid Nili Ahmadabadi,et al.  Optimal Local Basis: A Reinforcement Learning Approach for Face Recognition , 2009, International Journal of Computer Vision.

[10]  Jiandong Li,et al.  A Survey on Routing Protocols for Large-Scale Wireless Sensor Networks , 2011, Sensors.

[11]  JAMAL N. AL-KARAKI,et al.  Routing techniques in wireless sensor networks: a survey , 2004, IEEE Wireless Communications.

[12]  Sajal K. Das,et al.  R3E: Reliable Reactive Routing Enhancement for Wireless Sensor Networks , 2014, IEEE Transactions on Industrial Informatics.

[13]  Michele Zorzi,et al.  ALBA-R: Load-Balancing Geographic Routing Around Connectivity Holes in Wireless Sensor Networks , 2014, IEEE Transactions on Parallel and Distributed Systems.

[14]  Roberto Beraldi,et al.  A Biased Random Walk Routing Protocol for Wireless Sensor Networks: The Lukewarm Potato Protocol , 2010, IEEE Transactions on Mobile Computing.

[15]  Youxian Sun,et al.  Enhancing Real-Time Delivery in Wireless Sensor Networks With Two-Hop Information , 2009, IEEE Transactions on Industrial Informatics.

[16]  Paloma Martínez,et al.  Learning teaching strategies in an Adaptive and Intelligent Educational System through Reinforcement Learning , 2009, Applied Intelligence.

[17]  Wendi B. Heinzelman,et al.  Negotiation-Based Protocols for Disseminating Information in Wireless Sensor Networks , 2002, Wirel. Networks.

[18]  Abdul Hanan Abdullah,et al.  Overview of Data Routing Approaches for Wireless Sensor Networks , 2012, Sensors.

[19]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[20]  Yuh-Ren Tsai,et al.  Coverage-preserving routing protocols for randomly distributed wireless sensor networks , 2007, IEEE Transactions on Wireless Communications.

[21]  Gregory J. Pottie,et al.  Protocols for self-organization of a wireless sensor network , 2000, IEEE Wirel. Commun..

[22]  Wendi B. Heinzelman,et al.  Adaptive protocols for information dissemination in wireless sensor networks , 1999, MobiCom.

[23]  Yuh-Ren Tsai,et al.  Coverage-preserving routing protocols for randomly distributed wireless sensor networks , 2007, IEEE Transactions on Wireless Communications.

[24]  E. Rodrigues Gomes,et al.  Learning the IPA Market with Individual and Social Rewards , 2007, 2007 IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT'07).

[25]  Nicholas R. Jennings,et al.  Self-organized routing for wireless microsensor networks , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[26]  Deborah Estrin,et al.  Geographical and Energy Aware Routing: a recursive data dissemination protocol for wireless sensor networks , 2002 .

[27]  B. Etefia.  Routing Protocols for Wireless Sensor Networks , 2004 .

[28]  Jan M. Rabaey,et al.  Energy aware routing for low energy ad hoc sensor networks , 2002, 2002 IEEE Wireless Communications and Networking Conference Record. WCNC 2002 (Cat. No.02TH8609).

[29]  Arthur L. Liestman,et al.  A survey of gossiping and broadcasting in communication networks , 1988, Networks.

[30]  Ian F. Akyildiz,et al.  Wireless sensor networks: a survey , 2002, Comput. Networks.

[31]  Kin K. Leung,et al.  A dynamic clustering and energy efficient routing technique for sensor networks , 2007, IEEE Transactions on Wireless Communications.

[32]  Haojun Huang,et al.  Energy-aware interference-sensitive geographic routing in wireless sensor networks , 2011, IET Commun..

[33]  David E. Culler,et al.  A unifying link abstraction for wireless sensor networks , 2005, SenSys '05.

[34]  Abdalkarim Awad,et al.  Exploiting Virtual Coordinates for Improved Routing Performance in Sensor Networks , 2011, IEEE Transactions on Mobile Computing.

[35]  Rajashekhar C. Biradar,et al.  A survey on routing protocols in Wireless Sensor Networks , 2012, 2012 18th IEEE International Conference on Networks (ICON).

[36]  M. Winkler,et al.  Theoretical and practical aspects of military wireless sensor networks , 2023, Journal of Telecommunications and Information Technology.

[37]  Bülent Tavli,et al.  A survey of visual sensor network platforms , 2012, Multimedia Tools and Applications.

[38]  Ryszard Kowalczyk,et al.  Dynamic analysis of multiagent Q-learning with ε-greedy exploration , 2009, ICML '09.

[39]  Luca Mottola,et al.  MUSTER: Adaptive Energy-Aware Multisink Routing in Wireless Sensor Networks , 2011, IEEE Transactions on Mobile Computing.

[40]  Deborah Estrin,et al.  Geography-informed energy conservation for Ad Hoc routing , 2001, MobiCom '01.

[41]  Dong-Sung Kim,et al.  Enhancing Real-Time Delivery of Gradient Routing for Industrial Wireless Sensor Networks , 2012, IEEE Transactions on Industrial Informatics.