Reinforcement Learning based Node Sleep or Wake-up Time Scheduling Algorithm for Wireless Sensor Network

A wireless sensor network is a collection of small sensor nodes that have limited energy and are usually not rechargeable. Because of this, the lifetime of wireless sensor networks has always been a challenging area. One of the basic problems of the network has been the ability of the nodes to effectively schedule the sleep and wake-up time to overcome this problem. The motivation behind node sleep or wake-up time scheduling is to take care of nodes in sleep mode for as long as possible (without losing data packet transfer efficiency) and thus extend their useful life. This research going to propose scheduling of nodes sleeps and wake-up time through reinforcement learning. This research is not based on the nodes' duty cycle strategy (which creates a compromise between data packet delivery and nodes energy saving delay) like other existing researches. It is based on the research of reinforcement learning which gives independence to each node to choose its own activity from the transmission of packets, tuning or sleep node in each time band which works in a decentralized way. The simulation results show the qualified performance of the proposed algorithm under different conditions. KeywordsSleep or wake-up scheduling, Wireless sensor network, Sensor node energy.

[1]  S. Chandrakala,et al.  Erratum to: MRL-SCSO: Multi-agent Reinforcement Learning-Based Self-Configuration and Self-Optimization Protocol for Unattended Wireless Sensor Networks , 2017, Wirel. Pers. Commun..

[2]  Gaddafi Abdul-Salaam,et al.  Energy-Efficient Data Reporting for Navigation in Position-Free Hybrid Wireless Sensor Networks , 2017, IEEE Sensors Journal.

[3]  ZhangJun,et al.  Distributed evolutionary algorithms and their models , 2015 .

[4]  Athanasios V. Vasilakos,et al.  EDAL: An Energy-Efficient, Delay-Aware, and Lifetime-Balancing Data Collection Protocol for Heterogeneous Wireless Sensor Networks , 2015, IEEE/ACM Transactions on Networking.

[5]  S. Chandrakala,et al.  MRL-SCSO: Multi-agent Reinforcement Learning-Based Self-Configuration and Self-Optimization Protocol for Unattended Wireless Sensor Networks , 2017, Wirel. Pers. Commun..

[6]  Deborah Estrin,et al.  Medium access control with coordinated adaptive sleeping for wireless sensor networks , 2004, IEEE/ACM Transactions on Networking.

[7]  David E. Culler,et al.  Versatile low power media access for wireless sensor networks , 2004, SenSys '04.

[8]  Mihail L. Sichitiu,et al.  An asynchronous scheduled MAC protocol for wireless sensor networks , 2013, Comput. Networks.

[9]  Damien Ernst,et al.  Reinforcement Learning for Electric Power System Decision and Control: Past Considerations and Perspectives , 2017 .

[10]  Cheng-Long Chuang,et al.  A Hybrid Memetic Framework for Coverage Optimization in Wireless Sensor Networks , 2015, IEEE Transactions on Cybernetics.

[11]  Xiaobo Sharon Hu,et al.  A Real-Time and Non-Cooperative Task Allocation Framework for Social Sensing Applications in Edge Computing Systems , 2018, 2018 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS).

[12]  Rong Zheng,et al.  Asynchronous wakeup for ad hoc networks , 2003, MobiHoc '03.

[13]  Sang-Jo Yoo,et al.  Dynamic resource allocation using reinforcement learning for LTE-U and WiFi in the unlicensed spectrum , 2017, 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN).

[14]  Otman A. Basir,et al.  Semi-Flocking Algorithm for Motion Control of Mobile Sensors in Large-Scale Surveillance Systems , 2015, IEEE Transactions on Cybernetics.

[15]  Ana L. C. Bazzan,et al.  Opportunities for multiagent systems and multiagent reinforcement learning in traffic control , 2009, Autonomous Agents and Multi-Agent Systems.

[16]  Qingfu Zhang,et al.  Distributed evolutionary algorithms and their models: A survey of the state-of-the-art , 2015, Appl. Soft Comput..

[17]  Lei Tang,et al.  EM-MAC: a dynamic multichannel energy-efficient MAC protocol for wireless sensor networks , 2011, MobiHoc '11.

[18]  Lei Tang,et al.  PW-MAC: An energy-efficient predictive-wakeup MAC protocol for wireless sensor networks , 2011, 2011 Proceedings IEEE INFOCOM.

[19]  Athanasios V. Vasilakos,et al.  Tight Performance Bounds of Multihop Fair Access for MAC Protocols in Wireless Sensor Networks and Underwater Sensor Networks , 2012, IEEE Transactions on Mobile Computing.

[20]  Joel Z. Leibo,et al.  Multi-agent Reinforcement Learning in Sequential Social Dilemmas , 2017, AAMAS.

[21]  Athanasios V. Vasilakos,et al.  Prediction-based data aggregation in wireless sensor networks: Combining grey model and Kalman Filter , 2011, Comput. Commun..

[22]  Joohwan Kim,et al.  Minimizing Delay and Maximizing Lifetime for Wireless Sensor Networks With Anycast , 2010, IEEE/ACM Transactions on Networking.

[23]  Tommi S. Jaakkola,et al.  Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms , 2000, Machine Learning.

[24]  Shu Du,et al.  DW-MAC: a low latency, energy efficient demand-wakeup MAC protocol for wireless sensor networks , 2008, MobiHoc '08.

[25]  Deborah Estrin,et al.  An energy-efficient MAC protocol for wireless sensor networks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[26]  Bhavna Ambudkar,et al.  Sleep Scheduling for Critical Event Monitoring in Wireless Sensor Networks , 2015 .

[27]  Omer Gurewitz,et al.  RI-MAC: a receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks , 2008, SenSys '08.

[28]  Athanasios V. Vasilakos,et al.  A Survey on Ambient Intelligence in Healthcare , 2013, Proceedings of the IEEE.

[29]  Xin-Ping Guan,et al.  Distributed optimal consensus filter for target tracking in heterogeneous sensor networks , 2011, 2011 8th Asian Control Conference (ASCC).

[30]  Huang Lee,et al.  Wakeup scheduling in wireless sensor networks , 2006, MobiHoc '06.

[31]  Yi Liang,et al.  A Survey on Topology Control in Wireless Sensor Networks: Taxonomy, Comparative Study, and Open Issues , 2008, Proceedings of the IEEE.

[32]  John S. Heidemann,et al.  Ultra-low duty cycle MAC with scheduled channel polling , 2006, SenSys '06.

[33]  Shimon Whiteson,et al.  Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning , 2017, ICML.

[34]  Xuxun Liu,et al.  A Deployment Strategy for Multiple Types of Requirements in Wireless Sensor Networks , 2015, IEEE Transactions on Cybernetics.

[35]  Tarek F. Abdelzaher,et al.  Towards optimal sleep scheduling in sensor networks for rare-event detection , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[36]  Binoy Ravindran,et al.  Heterogenous Quorum-Based Wake-Up Scheduling in Wireless Sensor Networks , 2010, IEEE Transactions on Computers.

[37]  Yu Zhao,et al.  Distributed average computation for multiple time‐varying signals with output measurements , 2016 .

[38]  Bo Fu,et al.  Bio-inspired Group Modeling and Analysis for Intruder Detection in Mobile Sensor/Robotic Networks , 2015, IEEE Transactions on Cybernetics.

[39]  Vijay K. Bhargava,et al.  Wireless sensor networks with energy harvesting technologies: a game-theoretic approach to optimal energy management , 2007, IEEE Wireless Communications.