Unsupervised energy prediction in a Smart Grid context using reinforcement cross-building transfer learning

Abstract In a future Smart Grid context, increasing challenges in managing the stochastic local energy supply and demand are expected. This increased the need of more accurate energy prediction methods in order to support further complex decision-making processes. Although many methods aiming to predict the energy consumption exist, all these require labelled data, such as historical or simulated data. Still, such datasets are not always available under the emerging Smart Grid transition and complex people behaviour. Our approach goes beyond the state-of-the-art energy prediction methods in that it does not require labelled data. Firstly, two reinforcement learning algorithms are investigated in order to model the building energy consumption. Secondly, as a main theoretical contribution, a Deep Belief Network (DBN) is incorporated into each of these algorithms, making them suitable for continuous states. Thirdly, the proposed methods yield a cross-building transfer that can target new behaviour of existing buildings (due to changes in their structure or installations), as well as completely new types of buildings. The methods are developed in the MATLAB® environment and tested on a real database recorded over seven years, with hourly resolution. Experimental results demonstrate that the energy prediction accuracy in terms of RMSE has been significantly improved in 91.42% of the cases after using a DBN for automatically extracting high-level features from the unlabelled data, compared to the equivalent methods without the DBN pre-processing.

[1]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[2]  J. Zico Kolter,et al.  Large-scale probabilistic forecasting in energy systems using sparse Gaussian conditional random fields , 2013, 52nd IEEE Conference on Decision and Control.

[3]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[4]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[5]  Geoffrey E. Hinton,et al.  Reinforcement Learning with Factored States and Actions , 2004, J. Mach. Learn. Res..

[6]  Tony N.T. Lam,et al.  Artificial neural networks for energy analysis of office buildings with daylighting , 2010 .

[7]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[8]  M. Smith,et al.  A comparison of periodic autoregressive and dynamic factor models in intraday energy demand forecasting , 2013 .

[9]  Andrea Costa,et al.  Building operation and energy performance: Monitoring, analysis and optimisation toolkit , 2013 .

[10]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[11]  Anastasios G. Bakirtzis,et al.  Comparison of advanced power system operations models for large-scale renewable integration , 2015 .

[12]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[13]  Soteris A. Kalogirou,et al.  Artificial neural networks in energy applications in buildings , 2006 .

[14]  Lv Jinhu,et al.  A Novel Hybrid Approach of KPCA and SVM for Building Cooling Load Prediction , 2010, 2010 Third International Conference on Knowledge Discovery and Data Mining.

[15]  Yee Whye Teh,et al.  Actor-Critic Reinforcement Learning with Energy-Based Policies , 2012, EWRL.

[16]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[17]  Liu Yang,et al.  Thermal comfort and building energy consumption implications - A review , 2014 .

[18]  Haitham Bou-Ammar,et al.  Automatically Mapped Transfer between Reinforcement Learning Tasks via Three-Way Restricted Boltzmann Machines , 2013, ECML/PKDD.

[19]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[20]  Anastasios I. Dounis,et al.  Artificial intelligence for energy conservation in buildings , 2010 .

[21]  Geoffrey E. Hinton,et al.  Two Distributed-State Models For Generating High-Dimensional Time Series , 2011, J. Mach. Learn. Res..

[22]  Wil L. Kling,et al.  Comparison of machine learning methods for estimating energy consumption in buildings , 2014, 2014 International Conference on Probabilistic Methods Applied to Power Systems (PMAPS).

[23]  F. Massey.  The Kolmogorov-Smirnov Test for Goodness of Fit , 1951 .

[24]  Wei-Chiang Hong,et al.  Electric load forecasting by support vector model , 2009 .

[25]  Ruslan Salakhutdinov,et al.  Learning Deep Boltzmann Machines using Adaptive MCMC , 2010, ICML.

[26]  S. J. Devlin,et al.  Robust estimation and outlier detection with correlation coefficients , 1975 .

[27]  L. Suganthi,et al.  Energy models for demand forecasting—A review , 2012 .

[28]  Bart De Schutter,et al.  Approximate reinforcement learning: An overview , 2011, 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[29]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[30]  Sylvain Robert,et al.  State of the art in building modelling and energy performances prediction: A review , 2013 .

[31]  Changyin Sun,et al.  Next Day Load Forecasting Using SVM , 2005, ISNN.

[32]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[33]  V. Ismet Ugursal,et al.  Comparison of neural network, conditional demand analysis, and engineering approaches for modeling end-use energy consumption in the residential sector , 2008 .

[34]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[35]  Damien Ernst,et al.  Learning Exploration/Exploitation Strategies for Single Trajectory Reinforcement Learning , 2012, EWRL.

[36]  Marco Wiering,et al.  Reinforcement Learning , 2014, Adaptation, Learning, and Optimization.

[37]  A. Miraoui,et al.  A Comparison of Smart Grid Technologies and Progresses in Europe and the U.S. , 2012, IEEE Transactions on Industry Applications.

[38]  Louis Wehenkel,et al.  Reinforcement Learning Versus Model Predictive Control: A Comparison on a Power System Problem , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  Andrew G. Barto,et al.  Improving Elevator Performance Using Reinforcement Learning , 1995, NIPS.

[40]  Frédéric Magoulès,et al.  A review on the prediction of building energy consumption , 2012 .