Study on deep reinforcement learning techniques for building energy consumption forecasting

Abstract Reliable and accurate building energy consumption prediction is becoming increasingly pivotal in building energy management. Currently, data-driven approach has shown promising performances and gained lots of research attention due to its efficiency and flexibility. As a combination of reinforcement learning and deep learning, deep reinforcement learning (DRL) techniques are expected to solve nonlinear and complex issues. However, very little is known about DRL techniques in forecasting building energy consumption. Therefore, this paper presents a case study of an office building using three commonly-used DRL techniques to forecast building energy consumption, namely Asynchronous Advantage Actor-Critic (A3C), Deep Deterministic Policy Gradient (DDPG) and Recurrent Deterministic Policy Gradient (RDPG). The objective is to investigate the potential of DRL techniques in building energy consumption prediction field. A comprehensive comparison between DRL models and common supervised models is also provided. The results demonstrate that the proposed DDPG and RDPG models have obvious advantages in forecasting building energy consumption compared to common supervised models, while accounting for more computation time for model training. Their prediction performances measured by mean absolute error (MAE) can be improved by 16%-24% for single-step ahead prediction, and 19%-32% for multi-step ahead prediction. The results also indicate that A3C performs poor prediction accuracy and shows much slower convergence speed than DDPG and RDPG. However, A3C is still the most efficient technique among these three DRL methods. The findings are enlightening and the proposed DRL methodologies can be positively extended to other prediction problems, e.g., wind speed prediction and electricity load prediction.

[1]  Basilio Sierra,et al.  Data-driven prognostics using a combination of constrained K-means clustering, fuzzy modeling and LOF-based score , 2017, Neurocomputing.

[2]  Miriam A. M. Capretz,et al.  Energy Forecasting for Event Venues: Big Data and Prediction Accuracy , 2016 .

[3]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD 2000.

[4]  Guoyin Fu,et al.  Deep belief network based ensemble approach for cooling load forecasting of air-conditioning system , 2018 .

[5]  Shengwei Wang,et al.  A system-level fault detection and diagnosis method for low delta-T syndrome in the complex HVAC systems , 2016 .

[6]  Yang Zhao,et al.  Deep learning-based feature engineering methods for improved building energy prediction , 2019, Applied Energy.

[7]  Huanxin Chen,et al.  Energy consumption prediction for water-source heat pump system using pattern recognition-based algorithms , 2018 .

[8]  Antonio Colmenar-Santos,et al.  Solutions to reduce energy consumption in the management of large buildings , 2013 .

[9]  Xiaodong Cao,et al.  Building energy-consumption status worldwide and the state-of-the-art technologies for zero-energy buildings during the past decade , 2016 .

[10]  Ke Yan,et al.  Online fault detection methods for chillers combining extended kalman filter and recursive one-class SVM , 2017, Neurocomputing.

[11]  Flávio Miguel Varejão,et al.  Monthly energy consumption forecast: A deep learning approach , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[12]  Nora El-Gohary,et al.  A review of data-driven building energy consumption prediction studies , 2018 .

[13]  Shengwei Wang,et al.  Development of prediction models for next-day building energy consumption and peak power demand using data mining techniques , 2014 .

[14]  Fu Xiao,et al.  An interactive building power demand management strategy for facilitating smart grid optimization , 2014 .

[15]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[16]  Madeleine Gibescu,et al.  Deep learning for estimating building energy consumption , 2016 .

[17]  Jiejin Cai,et al.  Applying support vector machine to predict hourly cooling load in the building , 2009 .

[18]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[19]  Huanxin Chen,et al.  Optimization of support vector regression model based on outlier detection methods for predicting electricity consumption of a public building WSHP system , 2017 .

[20]  Benjamin C. M. Fung,et al.  A decision tree method for building energy demand modeling , 2010 .

[21]  Alberto Hernandez Neto,et al.  Comparison between detailed model simulation and artificial neural network for forecasting building energy consumption , 2008 .

[22]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[23]  Mohammad Yusri Hassan,et al.  A review on applications of ANN and SVM for building electrical energy consumption forecasting , 2014 .

[24]  Jiangyan Liu,et al.  Evaluation of the energy performance of variable refrigerant flow systems using dynamic energy benchmarks based on data mining techniques , 2017 .

[25]  D. Kolokotsa,et al.  Reinforcement learning for energy conservation and comfort in buildings , 2007 .

[26]  Shengwei Wang,et al.  Development and validation of an effective and robust chiller sequence control strategy using data-driven models , 2016 .

[27]  Yisheng Lv,et al.  Data driven parallel prediction of building energy consumption using generative adversarial nets , 2019, Energy and Buildings.

[28]  Wolfram Burgard,et al.  Learning driving styles for autonomous vehicles from demonstration , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[29]  Vivek Srikumar,et al.  Predicting electricity consumption for commercial and residential buildings using deep recurrent neural networks , 2018 .

[30]  Tao Liu,et al.  A novel deep reinforcement learning based methodology for short-term HVAC system energy consumption prediction , 2019, International Journal of Refrigeration.

[31]  Chongqing Kang,et al.  Load profiling and its application to demand response: A review , 2015 .

[32]  Bogdan Gabrys,et al.  Meta-learning for time series forecasting and forecast combination , 2010, Neurocomputing.

[33]  Minoru Asada,et al.  Purposive behavior acquisition for a real robot by vision-based reinforcement learning , 1995, Machine Learning.

[34]  Wei Feng,et al.  China's energy consumption in the building sector: A Statistical Yearbook-Energy Balance Sheet based splitting method , 2018, Journal of Cleaner Production.

[35]  TaeChoong Chung,et al.  Controlling bicycle using deep deterministic policy gradient algorithm , 2017, 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[36]  Tao Liu,et al.  Modal decomposition based ensemble learning for ground source heat pump systems load forecasting , 2019, Energy and Buildings.

[37]  Yueren Wang,et al.  A novel ensemble learning approach to support building energy use prediction , 2018 .

[38]  John N. Tsitsiklis,et al.  Actor-Critic Algorithms , 1999, NIPS.

[39]  Fu Xiao,et al.  A short-term building cooling load prediction method using deep learning algorithms , 2017 .

[40]  Lei Yang,et al.  Reinforcement learning for optimal control of low exergy buildings , 2015 .