Residential Load Scheduling With Renewable Generation in the Smart Grid: A Reinforcement Learning Approach

The significance and need of demand response (DR) programs is realized by the utility as a means to reduce the additional production cost imposed by the accelerating energy demand. With the development in smart information and communication systems, the price-based DR programs can be effectively utilized for controlling the loads of smart residential buildings. Nowadays, the use of stochastic renewable energy sources like photovoltaic (PV) by a small domestic consumer is increasing. In this paper, a generalized model for the residential load scheduling or load commitment problem (LCP) in the presence of renewable sources for any type of tariff is presented. Reinforcement learning (RL) is an efficient tool that has been used to solve the decision making problem under uncertainty. An RL-based approach to solve the LCP is also proposed. The novelty of this paper lies in the introduction of a comprehensive model with implementable solution considering consumer comfort, stochastic renewable power, and tariff. Simulation experiments are conducted to test the efficacy and scalability of the proposed algorithm. The performance of the algorithm is investigated by considering a domestic consumer with schedulable and nonschedulable appliances along with a PV source. Guidelines are given for choosing the parameters of the load.

[1]  Hamid R. Tizhoosh,et al.  Application of reinforcement learning for segmentation of transrectal ultrasound images , 2008, BMC Medical Imaging.

[2]  Allen J. Wood,et al.  Power Generation, Operation, and Control , 1984 .

[3]  E. Caamaño-Martín,et al.  Neural network controller for Active Demand-Side Management with PV energy in the residential sector , 2012 .

[4]  Bhupendra R. Parekh,et al.  Dispersed Generation Enable Loss Reduction and Voltage Profile Improvement in Distribution Network—Case Study, Gujarat, India , 2014, IEEE Transactions on Power Systems.

[5]  John N. Tsitsiklis,et al.  Neuro-dynamic programming: an overview , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  Min Dong,et al.  Real-Time Residential-Side Joint Energy Storage Management and Load Scheduling With Renewable Integration , 2015, IEEE Transactions on Smart Grid.

[8]  E. A. Jasmin,et al.  Load scheduling with Maximum Demand using Binary Particle Swarm Optimization , 2015, 2015 International Conference on Technological Advancements in Power and Energy (TAP Energy).

[9]  W.G.J.H.M. van Sark,et al.  Review of barriers to the introduction of residential demand response: a case study in the Netherlands , 2017 .

[10]  Qiang Yang,et al.  Demand response under real-time pricing for domestic energy system with DGs , 2014, 2014 International Conference on Power System Technology.

[11]  Vincent W. S. Wong,et al.  Autonomous Demand-Side Management Based on Game-Theoretic Energy Consumption Scheduling for the Future Smart Grid , 2010, IEEE Transactions on Smart Grid.

[12]  Liang Xiao,et al.  Reinforcement Learning-based Energy Trading for Microgrids , 2018, ArXiv.

[13]  João P. S. Catalão,et al.  Optimal Household Appliances Scheduling Under Day-Ahead Pricing and Load-Shaping Demand Response Strategies , 2015, IEEE Transactions on Industrial Informatics.

[14]  Thillainathan Logenthiran,et al.  Demand Side Management in Smart Grid Using Heuristic Optimization , 2012, IEEE Transactions on Smart Grid.

[15]  S. L. Arun,et al.  Intelligent Residential Energy Management System for Dynamic Demand Response in Smart Buildings , 2018, IEEE Systems Journal.

[16]  Daniel Masa Bote,et al.  Neural network controller for active demand side management with PV energy in the residential sector , 2012 .

[17]  D. Ernst,et al.  Power systems stability control: reinforcement learning framework , 2004, IEEE Transactions on Power Systems.

[18]  Lingfeng Wang,et al.  Autonomous Appliance Scheduling for Household Energy Management , 2014, IEEE Transactions on Smart Grid.

[19]  Amjad Anvari-Moghaddam,et al.  Optimal Smart Home Energy Management Considering Energy Saving and a Comfortable Lifestyle , 2016, IEEE Transactions on Smart Grid.

[20]  Giorgio Rizzoni,et al.  Residential Demand Response: Dynamic Energy Management and Time-Varying Electricity Pricing , 2016, IEEE Transactions on Power Systems.

[21]  Nazar H. Malik,et al.  Learning Automata Algorithms for Load Scheduling , 2013 .

[22]  Andrew G. Barto,et al.  Elevator Group Control Using Multiple Reinforcement Learning Agents , 1998, Machine Learning.

[23]  Mohammed H. Albadi,et al.  A summary of demand response in electricity markets , 2008 .

[24]  M. Thathachar,et al.  Networks of Learning Automata: Techniques for Online Stochastic Optimization , 2003 .

[25]  J. Aghaei,et al.  Demand response in smart electricity grids equipped with renewable energy sources: A review , 2013 .

[26]  E.F. El-Saadany,et al.  Optimal Renewable Resources Mix for Distribution System Energy Loss Minimization , 2010, IEEE Transactions on Power Systems.

[27]  Mihaela van der Schaar,et al.  Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning , 2016, IEEE Transactions on Smart Grid.

[28]  Jan T. Bialasiewicz,et al.  Power-Electronic Systems for the Grid Integration of Renewable Energy Sources: A Survey , 2006, IEEE Transactions on Industrial Electronics.

[29]  Phani Chavali,et al.  Parallel Load Schedule Optimization With Renewable Distributed Generators in Smart Grids , 2013, IEEE Transactions on Smart Grid.

[30]  Jiangfeng Zhang,et al.  Optimal scheduling of household appliances for demand response , 2014 .

[31]  Joongheon Kim,et al.  Residential Demand Response for Renewable Energy Resources in Smart Grid Systems , 2017, IEEE Transactions on Industrial Informatics.

[32]  Miao Pan,et al.  Optimal Power Management of Residential Customers in the Smart Grid , 2012, IEEE Transactions on Parallel and Distributed Systems.

[33]  Leopoldo G. Franquelo,et al.  Grid-Connected Photovoltaic Systems: An Overview of Recent Research and Emerging PV Converter Technology , 2015, IEEE Industrial Electronics Magazine.

[34]  Gerald Tesauro,et al.  Temporal Difference Learning and TD-Gammon , 1995, J. Int. Comput. Games Assoc..

[35]  Xiao-Ping Zhang,et al.  Real-time Energy Control Approach for Smart Home Energy Management System , 2014 .

[36]  Lester L. Preiss,et al.  Integrating Demand-Side Management into Utility Planning , 1986, IEEE Transactions on Power Systems.