Reinforcement Learning based Dynamic Model Selection for Short-Term Load Forecasting

With the growing prevalence of smart grid technology, short-term load forecasting (STLF) becomes particularly important in power system operations. There is a large collection of methods developed for STLF, but selecting a suitable method under varying conditions is still challenging. This paper develops a novel reinforcement learning based dynamic model selection (DMS) method for STLF. A forecasting model pool is first built, including ten state-of-the-art machine learning based forecasting models. Then a Q-learning agent learns the optimal policy of selecting the best forecasting model for the next time step, based on the model performance. The optimal DMS policy is applied to select the best model at each time step with a moving window. Numerical simulations on two-year load and weather data show that the Q-learning algorithm converges fast, resulting in effective and efficient DMS. The developed STLF model with Q-learning based DMS improves the forecasting accuracy by approximately 50%, compared to the state-of-the-art machine learning based STLF models.

[1]  Jose I. Bilbao,et al.  A review and analysis of regression and machine learning models on commercial building electricity load forecasting , 2017 .

[2]  Jie Zhang,et al.  Characterizing forecastability of wind sites in the United States , 2019, Renewable Energy.

[3]  Jie Zhang,et al.  A data-driven multi-model methodology with deep feature selection for short-term wind forecasting , 2017 .

[4]  Wei-Jen Lee,et al.  Improve the unit commitment scheduling by using the neural network based short term load forecasting , 2004, Conference, 2004 IEEE Industrial and Commercial Power Systems Technical.

[5]  Jie Zhang,et al.  Short-term global horizontal irradiance forecasting based on sky imaging and pattern recognition , 2017, 2017 IEEE Power & Energy Society General Meeting.

[6]  Jie Zhang,et al.  Short-Term Load Forecasting With Different Aggregation Strategies , 2018, DAC 2018.

[7]  Ran Li,et al.  Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN , 2018, IEEE Transactions on Smart Grid.

[8]  Miltiadis Alamaniotis,et al.  Evolutionary Multiobjective Optimization of Kernel-Based Very-Short-Term Load Forecasting , 2012, IEEE Transactions on Power Systems.

[9]  Haibo He,et al.  Q-Learning-Based Vulnerability Analysis of Smart Grid Against Sequential Topology Attacks , 2017, IEEE Transactions on Information Forensics and Security.

[10]  Jie Zhang,et al.  An Unsupervised Clustering-Based Short-Term Solar Forecasting Methodology Using Multi-Model Machine Learning Blending , 2018, ArXiv.

[11]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[12]  Jie Zhang,et al.  Characterizing and analyzing ramping events in wind power, solar power, load, and netload , 2017 .

[13]  Weilin Li,et al.  Short-term electrical load forecasting using the Support Vector Regression (SVR) model to calculate the demand response baseline for office buildings , 2017 .

[14]  Xin Wang,et al.  Factors that Impact the Accuracy of Clustering-Based Load Forecasting , 2015, IEEE Transactions on Industry Applications.

[15]  Abbas Khosravi,et al.  A review on artificial intelligence based load demand forecasting techniques for smart grid and buildings , 2015 .