Exploiting Q-Learning in Extending the Network Lifetime of Wireless Sensor Networks with Holes

Geographic routing is one of the most popular routing protocols in wireless sensor networks (WSNs) due to its simplicity and efficiency. However, with the occurrence of holes, geographic routing incurs with the so-called local minimum problem that may lead to a long hole detour path as well as the traffic concentration around the hole boundary. In consequence, the network lifetime is shortened. In this paper, we aim at proposing a lightweight distributed geographic routing protocol, which can prolong the lifetime of WSNs under the hole occurrence. Our main idea is to exploiting Q-learning technique to estimate the distance from a node to the holes. The routing decision is then determined based on the residual energy of the nodes, their estimated distance to the holes, and their distance to the destination. The simulation experiments show that our protocol strongly outperforms state-of-the-art protocols in terms of the network lifetime, packet latency and energy consumption. Specifically, our proposed protocol extends the network lifetime by more than 12% compared to the existing protocols.

[1]  Moyses M. Lima,et al.  Geographic routing and hole bypass using long range sinks for wireless sensor networks , 2017, Ad Hoc Networks.

[2]  Yusheng Ji,et al.  Load balanced and constant stretch routing in the vicinity of holes in WSNs , 2018, 2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC).

[3]  Khanh-Van Nguyen,et al.  A Distributed Protocol for Detecting and Updating Hole Boundary in Wireless Sensor Networks , 2015, SoICT.

[4]  Mani Srivastava,et al.  Energy-aware wireless microsensor networks , 2002, IEEE Signal Process. Mag..

[5]  Mohsen Guizani,et al.  A Disaster Management-Oriented Path Planning for Mobile Anchor Node-Based Localization in Wireless Sensor Networks , 2020, IEEE Transactions on Emerging Topics in Computing.

[6]  Ye Tian,et al.  Efficient Hole Detour Scheme for Geographic Routing in Wireless Sensor Networks , 2008, VTC Spring 2008 - IEEE Vehicular Technology Conference.

[7]  Ye Tian,et al.  Energy-Efficient Data Dissemination Protocol for Detouring Routing Holes in Wireless Sensor Networks , 2008, 2008 IEEE International Conference on Communications.

[8]  Yusheng Ji,et al.  Constant stretch and load balanced routing protocol for bypassing multiple holes in wireless sensor networks , 2017, 2017 IEEE 16th International Symposium on Network Computing and Applications (NCA).

[9]  Huynh Thi Thanh Binh,et al.  Connectivity optimization problem in vehicular mobile Wireless Sensor Networks , 2016, 2016 International Conference on Computational Intelligence and Cybernetics.

[10]  Yusheng Ji,et al.  Distributed hole-bypassing protocol in WSNs with constant stretch and load balancing , 2017, Comput. Networks.

[11]  Yunsi Fei,et al.  QELAR: A Machine-Learning-Based Adaptive Routing Protocol for Energy-Efficient and Lifetime-Extended Underwater Sensor Networks , 2010, IEEE Transactions on Mobile Computing.

[12]  Leonidas J. Guibas,et al.  Locating and bypassing routing holes in sensor networks , 2004, IEEE INFOCOM 2004.

[13]  Huynh Thi Thanh Binh,et al.  Heuristic algorithm for target coverage with connectivity fault-tolerance problem in wireless sensor networks , 2016, 2016 Conference on Technologies and Applications of Artificial Intelligence (TAAI).

[14]  Geyong Min,et al.  Energy-Aware Dual-Path Geographic Routing to Bypass Routing Holes in Wireless Sensor Networks , 2018, IEEE Transactions on Mobile Computing.

[15]  Hyunseung Choo,et al.  Efficient hole bypass routing scheme using observer packets for geographic routing in wireless sensor networks , 2011, SIAP.

[16]  Khanh-Van Nguyen,et al.  Load balanced routing with constant stretch for wireless sensor network with holes , 2014, 2014 IEEE Ninth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP).

[17]  Myounggyu Won,et al.  CAMA: Efficient Modeling of the Capture Effect for Low-Power Wireless Networks , 2014, TOSN.

[18]  Sanjay Jha,et al.  The holes problem in wireless sensor networks: a survey , 2005, MOCO.

[19]  Young-Bae Ko,et al.  QGeo: Q-Learning-Based Geographic Ad Hoc Routing Protocol for Unmanned Robotic Networks , 2017, IEEE Communications Letters.

[20]  Deborah Estrin,et al.  GPS-less low-cost outdoor localization for very small devices , 2000, IEEE Wirel. Commun..

[21]  Nadeem Javaid,et al.  A Balanced Energy Consuming and Hole Alleviating Algorithm for Wireless Sensor Networks , 2017, 2017 31st International Conference on Advanced Information Networking and Applications Workshops (WAINA).

[22]  Narendra Singh Raghuwanshi,et al.  Wireless sensor networks for agriculture: The state-of-the-art in practice and future challenges , 2015, Comput. Electron. Agric..

[23]  Gregory J. Pottie,et al.  Wireless integrated network sensors , 2000, Commun. ACM.

[24]  Sang-Ha Kim,et al.  Void Avoidance Scheme for Real-Time Data Dissemination in Irregular Wireless Sensor Networks , 2016, 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA).

[25]  Matt Welsh,et al.  Simulating the power consumption of large-scale sensor network applications , 2004, SenSys '04.

[26]  Chih-Yung Chang,et al.  Active Route-Guiding Protocols for Resisting Obstacles in Wireless Sensor Networks , 2010, IEEE Transactions on Vehicular Technology.