Deep reinforcement learning for building HVAC control

Buildings account for nearly 40% of the total energy consumption in the United States, about half of which is used by the HVAC (heating, ventilation, and air conditioning) system. Intelligent scheduling of building HVAC systems has the potential to significantly reduce the energy cost. However, the traditional rule-based and model-based strategies are often inefficient in practice, due to the complexity in building thermal dynamics and heterogeneous environment disturbances. In this work, we develop a data-driven approach that leverages the deep reinforcement learning (DRL) technique, to intelligently learn the effective strategy for operating the building HVAC systems. We evaluate the performance of our DRL algorithm through simulations using the widely-adopted EnergyPlus tool. Experiments demonstrate that our DRL-based algorithm is more effective in energy cost reduction compared with the traditional rule-based approach, while maintaining the room temperature within desired range.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Michael Wetter,et al.  Co-simulation of building energy and control systems with the Building Controls Virtual Test Bed , 2011 .

[3]  Standard Ashrae.  Thermal Environmental Conditions for Human Occupancy , 1992 .

[4]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[5]  Enda Barrett,et al.  Autonomous HVAC Control, A Reinforcement Learning Approach , 2015, ECML/PKDD.

[6]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[7]  Giuseppe Tommaso Costanzo,et al.  Experimental analysis of data-driven control for a building heating system , 2015, ArXiv.

[8]  Daniel Urieli,et al.  A learning agent for heat-pump thermostat control , 2013, AAMAS.

[9]  Kalyan Veeramachaneni,et al.  Using reinforcement learning to optimize occupant comfort and energy usage in HVAC systems , 2014, J. Ambient Intell. Smart Environ..

[10]  Philip Haves,et al.  Model predictive control for the operation of building cooling systems , 2010, Proceedings of the 2010 American Control Conference.

[11]  Li Xia,et al.  A multi-grid reinforcement learning method for energy conservation and comfort of HVAC in buildings , 2015, 2015 IEEE International Conference on Automation Science and Engineering (CASE).

[12]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[13]  D. Nikovski,et al.  A Method for Computing Optimal SetPoint Schedule for HVAC Systems , 2013 .

[14]  Tianshu Wei,et al.  Co-scheduling of HVAC control, EV charging and battery usage for building energy efficiency , 2014, 2014 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[15]  Martin A. Riedmiller.  Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method , 2005, ECML.

[16]  Alberto Sangiovanni-Vincentelli,et al.  Model-Based Hierarchical Optimal Control Design for HVAC Systems , 2014 .

[17]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[18]  Gregor P. Henze,et al.  Evaluation of commercial building demand response potential using optimal short-term curtailment of heating, ventilation, and air-conditioning loads , 2014 .

[19]  Liming Xiang,et al.  Kernel-Based Reinforcement Learning , 2006, ICIC.

[20]  Pierre Geurts,et al.  Tree-Based Batch Mode Reinforcement Learning , 2005, J. Mach. Learn. Res..

[21]  Lei Yang,et al.  Reinforcement learning for optimal control of low exergy buildings , 2015 .

[22]  M Morari,et al.  Energy efficient building climate control using Stochastic Model Predictive Control and weather predictions , 2010, Proceedings of the 2010 American Control Conference.