A Reinforcement Learning-Based ToD Provisioning Dynamic Power Management for Sustainable Operation of Energy Harvesting Wireless Sensor Node

In this paper, a reinforcement learning-based throughput on demand (ToD) provisioning dynamic power management method (RLTDPM) is proposed for sustaining perpetual operation and satisfying the ToD requirements for today's energy harvesting wireless sensor node (EHWSN). The RLTDPM monitors the environmental state of the EHWS and adjusts their operational duty cycle under criteria of energy neutrality to meet the demanded throughput. Outcomes of these observation-adjustment interactions are then evaluated by feedback/reward that represents how well the ToD requests are met; subsequently, the observation-adjustment-evaluation process, so-called reinforcement learning, continues. After the learning process, the RLTDPM is able to autonomously adjust the duty cycle for satisfying the ToD requirement, and in doing so, sustain the perpetual operation of the EHWSN. Simulations of the proposed RLTDPM on a wireless sensor node powered by a battery and solar cell for image sensing tasks were performed. Experimental results demonstrate that the achieved demanded throughput is improved 10.7% for the most stringent ToD requirement, while the residual battery energy of the RLTDPM is improved 7.4% compared with an existing DPM algorithm for EHWSN with image sensing purpose.

[1]  Ekram Hossain,et al.  A Dynamic Programming Approach for QoS-Aware Power Management in Wireless Video Sensor Networks , 2009, IEEE Transactions on Vehicular Technology.

[2]  Sandy Irani,et al.  An overview of the competitive and adversarial approaches to designing dynamic power management strategies , 2005, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[3]  Peng-Yeng Yin,et al.  Maximum entropy-based optimal threshold selection using deterministic reinforcement learning with controlled randomization , 2002, Signal Process..

[4]  Micheal Drieberg,et al.  Deployment of MICAz mote for Wireless Sensor Network applications , 2011, 2011 IEEE International Conference on Computer Applications and Industrial Electronics (ICCAIE).

[5]  Allen C.-H. Wu,et al.  A predictive system shutdown method for energy saving of event-driven computation , 1997, 1997 Proceedings of IEEE International Conference on Computer Aided Design (ICCAD).

[6]  M. Cucumo,et al.  General calculation methods for solar trajectories , 1997 .

[7]  Eduardo Tovar,et al.  Improving Quality-of-Service in Wireless Sensor Networks by Mitigating “Hidden-Node Collisions” , 2009, IEEE Transactions on Industrial Informatics.

[8]  Anthony Rowe,et al.  Rate-Harmonized Scheduling and Its Applicability to Energy Management , 2010, IEEE Transactions on Industrial Informatics.

[9]  Leonard Kleinrock,et al.  QoS control for sensor networks , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[10]  Ivan Howitt,et al.  Wireless industrial sensor networks: framework for QoS assessment and QoS management. , 2006, ISA transactions.

[11]  Roy Chaoming Hsu,et al.  A Reinforcement Learning Agent for Dynamic Power Management in Embedded Systems , 2008 .

[12]  Mani B. Srivastava,et al.  Performance aware tasking for environmentally powered sensor networks , 2004, SIGMETRICS '04/Performance '04.

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  Christiana Honsberg,et al.  A New Generalized Detailed Balance Formulation to Calculate Solar Cell Efficiency Limits , 2001 .

[15]  Dakai Zhu,et al.  Reliability-Aware Energy Management for Periodic Real-Time Tasks , 2009, IEEE Trans. Computers.

[16]  Chang-Soo Kim,et al.  Energy management strategies of a fuel cell/battery hybrid system using fuzzy logics , 2005 .

[17]  Kuan-Chieh Wang,et al.  QoS-Aware Power Management for Energy Harvesting Wireless Sensor Network Utilizing Reinforcement Learning , 2009, 2009 International Conference on Computational Science and Engineering.

[18]  Luca Benini,et al.  A survey of design techniques for system-level dynamic power management , 2000, IEEE Trans. Very Large Scale Integr. Syst..

[19]  Mani B. Srivastava,et al.  Power management in energy harvesting sensor networks , 2007, TECS.

[20]  Jian-Jia Chen,et al.  Optimistic Reliability Aware Energy Management for Real-Time Tasks with Probabilistic Execution Times , 2008, 2008 Real-Time Systems Symposium.

[21]  Hai Jiang,et al.  Network Lifetime Maximization With Node Admission in Wireless Multimedia Sensor Networks , 2009, IEEE Transactions on Vehicular Technology.

[22]  D. Ernst,et al.  Power systems stability control: reinforcement learning framework , 2004, IEEE Transactions on Power Systems.

[23]  Chia-Feng Juang,et al.  Reinforcement Ant Optimized Fuzzy Controller for Mobile-Robot Wall-Following Control , 2009, IEEE Transactions on Industrial Electronics.

[24]  Luca Benini,et al.  Adaptive Power Management for Environmentally Powered Systems , 2010, IEEE Transactions on Computers.

[25]  Pramod K. Varshney,et al.  QoS Support in Wireless Sensor Networks: A Survey , 2004, International Conference on Wireless Networks.

[26]  Changjiu Zhou,et al.  Estimating Biped Gait Using Spline-Based Probability Distribution Function With Q-Learning , 2008, IEEE Transactions on Industrial Electronics.

[27]  Mani B. Srivastava,et al.  Harvesting aware power management for sensor networks , 2006, 2006 43rd ACM/IEEE Design Automation Conference.

[28]  Kao-Shing Hwang,et al.  Reinforcement Learning in Strategy Selection for a Coordinated Multirobot System , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[29]  Roy Chaoming Hsu,et al.  Reinforcement Learning-Based Dynamic Power Management for Energy Harvesting Wireless Sensor Network , 2009, IEA/AIE.

[30]  I. Reda,et al.  Solar position algorithm for solar radiation applications , 2004 .