Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks

A wireless sensor network (WSN) is composed of a large number of tiny sensor nodes. Sensor nodes are very resource-constrained, since nodes are often battery-operated and energy is a scarce resource. In this paper, a resource-aware task scheduling (RATS) method is proposed with better performance/resource consumption trade-off in a WSN. Particularly, RATS exploits an adversarial bandit solver method called exponential weight for exploration and exploitation (Exp3) for target tracking application of WSN. The proposed RATS method is compared and evaluated with the existing scheduling methods exploiting online learning: distributed independent reinforcement learning (DIRL), reinforcement learning (RL), and cooperative reinforcement learning (CRL), in terms of the tracking quality/energy consumption trade-off in a target tracking application. The communication overhead and computational effort of these methods are also computed. Simulation results show that the proposed RATS outperforms the existing methods DIRL and RL in terms of achieved tracking performance.

[1]  Chi-Sheng Shih,et al.  Collaborative resource allocation in wireless sensor networks , 2004, Proceedings. 16th Euromicro Conference on Real-Time Systems, 2004. ECRTS 2004..

[2]  Bernhard Rinner,et al.  Resource coordination in wireless sensor networks by cooperative reinforcement learning , 2012, 2012 IEEE International Conference on Pervasive Computing and Communications Workshops.

[3]  Athanasios V. Vasilakos,et al.  A new approach to the design of reinforcement schemes for learning automata: Stochastic estimator learning algorithm , 1995, Neurocomputing.

[4]  Athanasios V. Vasilakos,et al.  Hierarchical Data Aggregation Using Compressive Sensing (HDACS) in WSNs , 2015, ACM Trans. Sens. Networks.

[5]  Eylem Ekici,et al.  Energy-constrained task mapping and scheduling in wireless sensor networks , 2005, IEEE International Conference on Mobile Adhoc and Sensor Systems Conference, 2005..

[6]  Naixue Xiong,et al.  Design and Analysis of Self-Adapted Task Scheduling Strategies in Wireless Sensor Networks , 2011, Sensors.

[7]  Athanasios V. Vasilakos,et al.  Compressed data aggregation for energy efficient wireless sensor networks , 2011, 2011 8th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks.

[8]  Tamer A. ElBatt,et al.  Cooperative Q-learning techniques for distributed online power allocation in femtocell networks , 2015, Wirel. Commun. Mob. Comput..

[9]  Tony Greenfield,et al.  Theory and Problems of Probability and Statistics , 1982 .

[10]  Peter Auer,et al.  The Nonstochastic Multiarmed Bandit Problem , 2002, SIAM J. Comput..

[11]  Kechar Bouabdellah,et al.  Impact of Model Mobility in Ad Hoc Routing Protocols , 2012 .

[12]  Deborah Estrin,et al.  An energy-efficient MAC protocol for wireless sensor networks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[13]  Xin Yao,et al.  Socio-economic vision graph generation and handover in distributed smart camera networks , 2014, TOSN.

[14]  N. Singh,et al.  A Comparison of Utility-Based Information Management Policies in Sensor Networks , 2006, 2006 IEEE Systems and Information Engineering Design Symposium.

[15]  Bernhard Rinner,et al.  Performance Analysis of Resource-Aware Task Scheduling Methods in Wireless Sensor Networks , 2014, Int. J. Distributed Sens. Networks.

[16]  J. Brian Gray,et al.  Introduction to Linear Regression Analysis , 2002, Technometrics.

[17]  John W. Byers,et al.  Utility-based decision-making in wireless sensor networks , 2000, 2000 First Annual Workshop on Mobile and Ad Hoc Networking and Computing. MobiHOC (Cat. No.00EX444).

[18]  Kok-Lim Alvin Yau,et al.  Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues , 2012, J. Netw. Comput. Appl..

[19]  Athanasios V. Vasilakos,et al.  Physarum Optimization: A Biology-Inspired Algorithm for the Steiner Tree Problem in Networks , 2015, IEEE Transactions on Computers.

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Marcelo Finger,et al.  Advances in Artificial Intelligence - SBIA 2012 , 2012, Lecture Notes in Computer Science.

[22]  Athanasios V. Vasilakos,et al.  Spatial Reusability-Aware Routing in Multi-Hop Wireless Networks , 2016, IEEE Transactions on Computers.

[23]  Athanasios V. Vasilakos,et al.  Approximating Congestion + Dilation in Networks via "Quality of Routing" Games , 2012, IEEE Trans. Computers.

[24]  Bruce H. Krogh,et al.  Energy-efficient surveillance system using wireless sensor networks , 2004, MobiSys '04.

[25]  Stephen B. Wicker,et al.  On the Complexity of Distributed Self-Configuration in Wireless Networks , 2003, Telecommun. Syst..

[26]  Athanasios V. Vasilakos,et al.  CodePipe: An opportunistic feeding and routing protocol for reliable multicast with pipelined network coding , 2012, 2012 Proceedings IEEE INFOCOM.

[27]  Athanasios V. Vasilakos,et al.  A Biology-Based Algorithm to Minimal Exposure Problem of Wireless Sensor Networks , 2014, IEEE Transactions on Network and Service Management.

[28]  Jürgen Gross,et al.  Linear Regression , 2003 .

[29]  Mohan Kumar,et al.  Distributed Independent Reinforcement Learning (DIRL) Approach to Resource Management in Wireless Sensor Networks , 2007, 2007 IEEE Internatonal Conference on Mobile Adhoc and Sensor Systems.

[30]  Bernhard Rinner,et al.  Energy-aware task scheduling in wireless sensor networks based on cooperative reinforcement learning , 2014, 2014 IEEE International Conference on Communications Workshops (ICC).