Adaptive Power Management in Solar Energy Harvesting Sensor Node Using Reinforcement Learning

In this paper, we present an adaptive power manager for solar energy harvesting sensor nodes. We use a simplified model consisting of a solar panel, an ideal battery and a general sensor node with variable duty cycle. Our power manager uses Reinforcement Learning (RL), specifically SARSA(λ) learning, to train itself from historical data. Once trained, we show that our power manager is capable of adapting to changes in weather, climate, device parameters and battery degradation while ensuring near-optimal performance without depleting or overcharging its battery. Our approach uses a simple but novel general reward function and leverages the use of weather forecast data to enhance performance. We show that our method achieves near perfect energy neutral operation (ENO) with less than 6% root mean square deviation from ENO as compared to more than 23% deviation that occur when using other approaches.

[1]  Israel Koren,et al.  Event-driven adaptive duty-cycling in sensor networks , 2009, Int. J. Sens. Networks.

[2]  Taner Akkan,et al.  Power management for Wireless Sensor Networks in underground mining , 2016, 2016 24th Signal Processing and Communication Application Conference (SIU).

[3]  Roy Chaoming Hsu,et al.  Dynamic power management utilizing reinforcement learning with fuzzy reward for energy harvesting wireless sensor nodes , 2011, IECON 2011 - 37th Annual Conference of the IEEE Industrial Electronics Society.

[4]  Prashant J. Shenoy,et al.  Cloudy Computing: Leveraging Weather Forecasts in Energy Harvesting Sensor Systems , 2010, 2010 7th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON).

[5]  N.H. Malik,et al.  Analysis of adaptability of Reinforcement Learning approach , 2011, 2011 IEEE 14th International Multitopic Conference.

[6]  Hao-Li Wang,et al.  A Reinforcement Learning-Based ToD Provisioning Dynamic Power Management for Sustainable Operation of Energy Harvesting Wireless Sensor Node , 2014, IEEE Transactions on Emerging Topics in Computing.

[7]  Tzu-Hao Lin,et al.  Dynamic energy management of energy harvesting wireless sensor nodes using fuzzy inference system with reinforcement learning , 2015, 2015 IEEE 13th International Conference on Industrial Informatics (INDIN).

[8]  Deniz Gündüz,et al.  A Learning Theoretic Approach to Energy Harvesting Communication System Optimization , 2012, IEEE Transactions on Wireless Communications.

[9]  Anja Klein,et al.  Reinforcement learning for energy harvesting point-to-point communications , 2016, 2016 IEEE International Conference on Communications (ICC).

[10]  R.J. Williams,et al.  Reinforcement learning is direct adaptive optimal control , 1991, IEEE Control Systems.

[11]  Andrea Bonarini,et al.  A bird's eye view on reinforcement learning approaches for power management in WSNs , 2013, 6th Joint IFIP Wireless and Mobile Networking Conference (WMNC).

[12]  Mani B. Srivastava,et al.  Adaptive Duty Cycling for Energy Harvesting Systems , 2006, ISLPED'06 Proceedings of the 2006 International Symposium on Low Power Electronics and Design.

[13]  Mani B. Srivastava,et al.  Power management in energy harvesting sensor networks , 2007, TECS.

[14]  Leonardo Badia,et al.  Energy Management Policies for Harvesting-Based Wireless Sensor Devices with Battery Degradation , 2013, IEEE Transactions on Communications.

[15]  Pengfei Zhang,et al.  Adaptive Duty Cycling in Sensor Networks With Energy Harvesting Using Continuous-Time Markov Chain and Fluid Models , 2015, IEEE Journal on Selected Areas in Communications.

[16]  Deniz Gündüz,et al.  Designing intelligent energy harvesting communication systems , 2014, IEEE Communications Magazine.

[17]  Andrew G. Barto,et al.  Adaptive Control of Duty Cycling in Energy-Harvesting Wireless Sensor Networks , 2007, 2007 4th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks.

[18]  Purushottam Kulkarni,et al.  Energy Harvesting Sensor Nodes: Survey and Implications , 2011, IEEE Communications Surveys & Tutorials.

[19]  Mani B. Srivastava,et al.  Performance aware tasking for environmentally powered sensor networks , 2004, SIGMETRICS '04/Performance '04.

[20]  Hassaan Khaliq Qureshi,et al.  Energy management in Wireless Sensor Networks: A survey , 2015, Comput. Electr. Eng..

[21]  Luca Benini,et al.  Battery-Driven Dynamic Power Management , 2001, IEEE Des. Test Comput..

[22]  David E. Culler,et al.  Perpetual environmentally powered sensor networks , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[23]  Mani B. Srivastava,et al.  Design considerations for solar energy harvesting wireless embedded systems , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..