Sliding window-based LightGBM model for electric load forecasting using anomaly repair

Smart grids have attracted much attention recently for their potential to reduce power system operating and management costs. Smart grid core components include energy storage, renewable energy source(s), and smart meters. Smart meters collect diverse data regarding smart grid operation, which can lead to inefficient operation if the meter data are damaged or tampered with during collection or transmission. Therefore, it is important to identify abnormalities in smart grid data and process them accordingly. Various anomaly detection models have been proposed using statistical methods, but they cannot detect some anomaly patterns accurately, and the models generally did not consider repair strategies for the detected anomalies. Anomaly repair should be included with model training to improve forecasting performance. This paper proposes a robust sliding window-based LightGBM model for short-term load forecasting using anomaly detection and repair. We first show how to detect anomalies using a variational autoencoder and then how they can be repaired using a random forest method. Finally, we verify that the proposed sliding window-based LightGBM achieves superior forecasting performance in combination with anomaly detection and repair.

[1]  Charu C. Aggarwal,et al.  Outlier Detection with Autoencoder Ensembles , 2017, SDM.

[2]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[3]  Miriam A. M. Capretz,et al.  Energy Forecasting for Event Venues: Big Data and Prediction Accuracy , 2016 .

[4]  Miriam A. M. Capretz,et al.  An ensemble learning framework for anomaly detection in building energy consumption , 2017 .

[5]  Yangkang Chen,et al.  Dealiased Seismic Data Interpolation Using Seislet Transform With Low-Frequency Constraint , 2015, IEEE Geoscience and Remote Sensing Letters.

[6]  J. Scott Armstrong,et al.  Combining forecasts: The end of the beginning or the beginning of the end? , 1989 .

[7]  Alberto Leon-Garcia,et al.  OpenAMI: Software-Defined AMI Load Balancing , 2018, IEEE Internet of Things Journal.

[8]  Ping-Huan Kuo,et al.  A High Precision Artificial Neural Networks Model for Short-Term Energy Load Forecasting , 2018 .

[9]  Àngela Nebot,et al.  Short-term electric load forecasting using computational intelligence methods , 2013, 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[10]  Philip S. Yu,et al.  Time Series Data Cleaning: From Anomaly Detection to Anomaly Repairing , 2017, Proc. VLDB Endow..

[11]  Jui-Sheng Chou,et al.  Smart grid data analytics framework for increasing energy savings in residential buildings , 2016 .

[12]  Victoria J. Hodge,et al.  Wireless Sensor Networks for Condition Monitoring in the Railway Industry: A Survey , 2015, IEEE Transactions on Intelligent Transportation Systems.

[13]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD 2000.

[14]  S Jagannathan Real-time big data analytics architecture for remote sensing application , 2016, 2016 International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES).

[15]  Mohammad Masoud Javidi,et al.  Estimates of residential building energy consumption using a multi-verse optimizer-based support vector machine with k-fold cross-validation , 2019, Evol. Syst..

[16]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[17]  Qingbo Yang,et al.  A Survey of Anomaly Detection Methods in Networks , 2009, 2009 International Symposium on Computer Network and Multimedia Technology.

[18]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[19]  Olfa Kanoun,et al.  Sensor technology advances and future trends , 2004, IEEE Transactions on Instrumentation and Measurement.

[20]  Eenjun Hwang,et al.  2-Stage Electric Load Forecasting Scheme for Day-Ahead CCHP Scheduling , 2019, 2019 IEEE 13th International Conference on Power Electronics and Drive Systems (PEDS).

[21]  Steven C. Hillmer,et al.  Modeling Time Series with Calendar Variation , 1983 .

[22]  Joaquim Melendez,et al.  Short-term load forecasting in a non-residential building contrasting models and attributes , 2015 .

[23]  Vanish Talwar,et al.  Statistical techniques for online anomaly detection in data centers , 2011, 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops.

[24]  Sung Wook Baik,et al.  A Two-Stage Industrial Load Forecasting Scheme for Day-Ahead Combined Cooling, Heating and Power Scheduling , 2020, Energies.

[25]  Tao Hong,et al.  Relative Humidity for Load Forecasting Models , 2018, IEEE Transactions on Smart Grid.

[26]  P. Pasquina,et al.  Sensor technology for smart homes. , 2011, Maturitas.

[27]  Chengdong Xu,et al.  Interpolation of Missing Temperature Data at Meteorological Stations Using P-BSHADE , 2013 .

[28]  Simin Nadjm-Tehrani,et al.  Embedded Cyber-Physical Anomaly Detection in Smart Meters , 2012, CRITIS.

[29]  Jun Luo,et al.  Energy-theft detection issues for advanced metering infrastructure in smart grid , 2014, Tsinghua Science and Technology.

[30]  Zhiwen Yu,et al.  A framework based on sparse representation model for time series prediction in smart city , 2020, Frontiers Comput. Sci..

[31]  Nadeem Javaid,et al.  Short Term Load Forecasting Using XGBoost , 2019, AINA Workshops.

[32]  Jianhui Wang,et al.  A Hierarchical Framework for Smart Grid Anomaly Detection Using Large-Scale Smart Meter Data , 2018, IEEE Transactions on Smart Grid.

[33]  Christian Habermann,et al.  Multidimensional Spline Interpolation: Theory and Applications , 2007 .

[34]  Gu-Yeon Wei,et al.  Survey of Hardware Systems for Wireless Sensor Networks , 2008, J. Low Power Electron..

[35]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[36]  Seungmin Rho,et al.  Combination of short-term load forecasting models based on a stacking ensemble approach , 2020 .

[37]  Sung Wook Baik,et al.  Bagging Ensemble of Multilayer Perceptrons for Missing Electricity Consumption Data Imputation , 2020, Sensors.

[38]  Richard J. Povinelli,et al.  Data Improving in Time Series Using ARX and ANN Models , 2017, IEEE Transactions on Power Systems.

[39]  M. Kramer Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[40]  Pu Wang,et al.  Electric load forecasting with recency effect: A big data approach , 2016 .