A Multi-Agent Reinforcement Learning-Based Data-Driven Method for Home Energy Management

This paper proposes a novel framework for home energy management (HEM) based on reinforcement learning in achieving efficient home-based demand response (DR). The concerned hour-ahead energy consumption scheduling problem is duly formulated as a finite Markov decision process (FMDP) with discrete time steps. To tackle this problem, a data-driven method based on neural network (NN) and ${Q}$ -learning algorithm is developed, which achieves superior performance on cost-effective schedules for HEM system. Specifically, real data of electricity price and solar photovoltaic (PV) generation are timely processed for uncertainty prediction by extreme learning machine (ELM) in the rolling time windows. The scheduling decisions of the household appliances and electric vehicles (EVs) can be subsequently obtained through the newly developed framework, of which the objective is dual, i.e., to minimize the electricity bill as well as the DR induced dissatisfaction. Simulations are performed on a residential house level with multiple home appliances, an EV and several PV panels. The test results demonstrate the effectiveness of the proposed data-driven based HEM framework.

[1]  Xiaofeng Yin,et al.  Stochastic Optimal Energy Management of Smart Home With PEV Energy Storage , 2018, IEEE Transactions on Smart Grid.

[2]  Xue Feng,et al.  Optimally coordinated dispatch of combined‐heat‐and‐electrical network with demand response , 2019, IET Generation, Transmission & Distribution.

[3]  E. A. Jasmin,et al.  Residential Load Scheduling With Renewable Generation in the Smart Grid: A Reinforcement Learning Approach , 2019, IEEE Systems Journal.

[4]  José R. Vázquez-Canteli,et al.  Reinforcement learning for demand response: A review of algorithms and modeling techniques , 2019, Applied Energy.

[5]  Seung Ho Hong,et al.  Demand Response for Home Energy Management Using Reinforcement Learning and Artificial Neural Network , 2019, IEEE Transactions on Smart Grid.

[6]  Jay Lee,et al.  Review and recent advances in battery health monitoring and prognostics technologies for electric vehicle (EV) safety and mobility , 2014 .

[7]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[8]  Luigi Atzori,et al.  Smart Home Energy Management Including Renewable Sources: A QoE-Driven Approach , 2018, IEEE Transactions on Smart Grid.

[9]  Sohrab Asgarpoor,et al.  Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid , 2018, IEEE Transactions on Power Systems.

[10]  Zhao Yang Dong,et al.  Hierarchical Energy Management System for Home Microgrids , 2019, IEEE Transactions on Smart Grid.

[11]  Mengmeng Yu,et al.  Incentive-based demand response considering hierarchical electricity market: A Stackelberg game approach , 2017 .

[12]  Haibo He,et al.  Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning , 2019, IEEE Transactions on Smart Grid.

[13]  H. Kappen Optimal control theory and the linear bellman equation , 2011 .

[14]  D. Pojani,et al.  Social housing and renewable energy: Community energy in a supporting role , 2018 .

[15]  Lei Wang,et al.  Chance Constrained Optimization in a Home Energy Management System , 2018, IEEE Transactions on Smart Grid.

[16]  Antonio Liotta,et al.  On-Line Building Energy Optimization Using Deep Reinforcement Learning , 2017, IEEE Transactions on Smart Grid.

[17]  Ashu Verma,et al.  Time-Coordinated Multienergy Management of Smart Buildings Under Uncertainties , 2019, IEEE Transactions on Industrial Informatics.

[18]  Weicong Kong,et al.  Optimal Home Energy Management System With Demand Charge Tariff and Appliance Operational Dependencies , 2020, IEEE Transactions on Smart Grid.

[19]  Xiao-Ping Zhang,et al.  Artificial intelligence based smart energy community management: A reinforcement learning approach , 2019, CSEE Journal of Power and Energy Systems.

[20]  Geoffrey R. Walker,et al.  Evaluating MPPT Converter Topologies Using a Matlab PV Model , 2000 .

[21]  Fangxing Li,et al.  Intelligent Multi-Microgrid Energy Management Based on Deep Neural Network and Model-Free Reinforcement Learning , 2020, IEEE Transactions on Smart Grid.

[22]  Seung Ho Hong,et al.  Demand Response Management for Industrial Facilities: A Deep Reinforcement Learning Approach , 2019, IEEE Access.

[23]  Scott Moura,et al.  PDE Modeling and Control of Electric Vehicle Fleets for Ancillary Services: A Discrete Charging Case , 2018, IEEE Transactions on Smart Grid.

[24]  Günther Palm,et al.  Value-Difference Based Exploration: Adaptive Control between Epsilon-Greedy and Softmax , 2011, KI.

[25]  Tao Jiang,et al.  Online Energy Management for a Sustainable Smart Home With an HVAC Load and Random Occupancy , 2017, IEEE Transactions on Smart Grid.

[26]  Taher Niknam,et al.  Probabilistic Load Forecasting Using an Improved Wavelet Neural Network Trained by Generalized Extreme Learning Machine , 2018, IEEE Transactions on Smart Grid.

[27]  M. Green Solar Cells : Operating Principles, Technology and System Applications , 1981 .

[28]  Seung Ho Hong,et al.  A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach , 2018, Applied Energy.

[29]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[30]  Pierluigi Siano,et al.  A novel strategy for optimal placement of locally controlled voltage regulators in traditional distribution systems , 2018 .

[31]  Kai Wang,et al.  Multi-step short-term wind speed forecasting approach based on multi-scale dominant ingredient chaotic analysis, improved hybrid GWO-SCA optimization and ELM , 2019, Energy Conversion and Management.

[32]  Hussain Shareef,et al.  Review on Home Energy Management System Considering Demand Responses, Smart Technologies, and Intelligent Controllers , 2018, IEEE Access.

[33]  Zhao Xu,et al.  Conditional Density Forecast of Electricity Price Based on Ensemble ELM and Logistic EMOS , 2019, IEEE Transactions on Smart Grid.

[34]  Archie C. Chapman,et al.  A Fast Technique for Smart Home Management: ADP With Temporal Difference Learning , 2018, IEEE Transactions on Smart Grid.

[35]  Bart De Schutter,et al.  Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning , 2017, IEEE Transactions on Smart Grid.