Deep reinforcement learning for home energy management system control

Abstract The use of machine learning techniques has been proven to be a viable solution for smart home energy management. These techniques autonomously control heating and domestic hot water systems, which are the most relevant loads in a dwelling, helping consumers to reduce energy consumption and also improving their comfort. Moreover, the number of houses equipped with renewable energy resources is increasing, and this is a key element for energy usage optimization, where coordinating loads and production can bring additional savings and reduce peak loads. In this regard, we propose the development of a deep reinforcement learning (DRL) algorithm for indoor and domestic hot water temperature control, aiming to reduce energy consumption by optimizing the usage of PV energy production. Furthermore, a methodology for a new dynamic indoor temperature setpoint definition is presented, thus allowing greater flexibility and savings. The results show that the proposed DRL algorithm combined with the dynamic setpoint achieved on average 8% of energy savings compared to a rule-based algorithm, reaching up to 16% of savings over the summer period. Moreover, the users’ comfort has not been compromised, as the algorithm is calibrated to not exceed more than 1% of the time out the specified temperature setpoints. Additional analysis shows that further savings could be achieved if the time out of comfort is increased, which could be agreed according to users’ needs. Regarding demand side management, the DRL control shows efficiency by anticipating and delaying actions for a PV self-consumption optimization, performing over 10% of load shifting. Finally, the renewable energy consumption is 9.5% higher for the DRL-based model compared to the rule-based, which means less energy consumed from the grid.

[1]  Hussain Shareef,et al.  Review on Home Energy Management System Considering Demand Responses, Smart Technologies, and Intelligent Controllers , 2018, IEEE Access.

[2]  Enda Barrett,et al.  Autonomous HVAC Control, A Reinforcement Learning Approach , 2015, ECML/PKDD.

[3]  Xingxing Zhang,et al.  A review of reinforcement learning methodologies for controlling occupant comfort in buildings , 2019, Sustainable Cities and Society.

[4]  Kuang-Chin Lu,et al.  Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithm , 2019, Building and Environment.

[5]  José R. Vázquez-Canteli,et al.  Reinforcement learning for demand response: A review of algorithms and modeling techniques , 2019, Applied Energy.

[6]  Tianshu Wei,et al.  Deep reinforcement learning for building HVAC control , 2017, 2017 54th ACM/EDAC/IEEE Design Automation Conference (DAC).

[7]  Gianfranco Rizzo,et al.  The control of indoor thermal comfort conditions: introducing a fuzzy adaptive controller , 2004 .

[8]  Jie Li,et al.  Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning , 2019, ArXiv.

[9]  Biao Huang,et al.  A Long-Short Term Memory Recurrent Neural Network Based Reinforcement Learning Controller for Office Heating Ventilation and Air Conditioning Systems , 2017 .

[10]  Enda Barrett,et al.  Transfer Learning Applied to Reinforcement Learning-Based HVAC Control , 2020, SN Computer Science.

[11]  T. Lawrence,et al.  The impact of increased cooling setpoint temperature during demand response events on occupant thermal comfort in commercial buildings: A review , 2018, Energy and Buildings.

[12]  Khee Poh Lam,et al.  Practical implementation and evaluation of deep reinforcement learning control for a radiant heating system , 2018, BuildSys@SenSys.

[13]  Johan Driesen,et al.  Deep Reinforcement Learning for Optimal Control of Space Heating , 2018, ArXiv.

[14]  William J. Batty,et al.  Fuzzy control strategies to provide cost and energy efficient high quality indoor environments in buildings with high occupant densities , 2003 .

[15]  Leslie K. Norford,et al.  Optimal control of HVAC and window systems for natural ventilation through reinforcement learning , 2018, Energy and Buildings.

[16]  Xin Li,et al.  Model-based and Data-driven Approaches for Building Automation and Control , 2018, 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[17]  Dae-Hyun Choi,et al.  Energy Management of Smart Home with Home Appliances, Energy Storage System and Electric Vehicle: A Hierarchical Deep Reinforcement Learning Approach , 2020, Sensors.

[18]  Michael J. Fell,et al.  Domestic demand-side response with heat pumps: controls and tariffs , 2019 .

[19]  Santiago Grijalva,et al.  A Review of Reinforcement Learning for Autonomous Building Energy Management , 2019, Comput. Electr. Eng..

[20]  P. O. Fanger,et al.  Thermal comfort: analysis and applications in environmental engineering, , 1972 .

[21]  Tianshu Wei.  Design And Management For Energy-Efficient Cyber-Physical Systems , 2018 .

[22]  Dasheng Lee,et al.  Energy savings by energy management systems: A review , 2016 .

[23]  Haibo He,et al.  Residential Energy Management with Deep Reinforcement Learning , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[24]  Tao Jiang,et al.  Deep Reinforcement Learning for Smart Home Energy Management , 2020, IEEE Internet of Things Journal.