Effects of data cleansing on load prediction algorithms

The rollout of advanced metering infrastructure that is planned in many countries worldwide will lead to a massive inflow of data from moderately reliable sensory equipment. In principle, this will make intelligent and automated planning and operation possible at an increasingly finer scale in the electric grid. However, errors can creep into the meter data, either from faulty sensors or during transmission from the meters to the database. This work studies the role of data cleansing as a preprocessing step for short-term (24-hour) power load prediction. We focus on cleansing and prediction at several levels of granularity, from the transmission level via distribution substations down to single households. We believe that preprocessing filters such as cleansing should lead to more robustness and/or precision in the subsequent processing step. However, load cleansing frameworks tend to make the popular assumption of normally and independently distributed noise in the time series. We show that this is incorrect at the diurnal level, due to the characteristic pattern of power consumption, with two peak loads during daytime and a nighttime trough. Moreover, we present empirical evidence that a preprocessing step based on this assumption fails to contribute positively to the performance of the subsequent prediction step. To rectify this problem, we suggest to subtract the average power load consumption in a given period before cleansing. We present empirical evidence that this improves the robustness and efficiency of load cleansing as a preprocessing step. Data cleansing and load prediction is performed by a system that searches out parameters using an evolutionary approach.

[1]  Hong-Tzer Yang,et al.  Identification of ARMAX model for short term load forecasting: an evolutionary programming approach , 1995 .

[2]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[3]  J. O. Ramsay,et al.  Functional Data Analysis (Springer Series in Statistics) , 1997 .

[4]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[5]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[6]  Hanne Marit Dalen,et al.  Formålsfordeling av husholdningenes elektrisitetsforbruk i 2006 : utvikling over tid 1990 - 2006 , 2009 .

[7]  Saeid Nahavandi,et al.  Construction of Optimal Prediction Intervals for Load Forecasting Problems , 2010, IEEE Transactions on Power Systems.

[8]  John G. Harris,et al.  Automatic speech recognition using a predictive echo state network classifier , 2007, Neural Networks.

[9]  Hemen Showkati,et al.  Short Term Load Forecasting using Echo State Networks , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[10]  Fionn Murtagh,et al.  Wavelet-based nonlinear multiscale decomposition model for electricity load forecasting , 2006, Neurocomputing.

[11]  Dipti Srinivasan,et al.  Parallel neural network-fuzzy expert system strategy for short-term load forecasting: system implementation and performance evaluation , 1999 .

[12]  Andrei Z. Morch,et al.  Results of monitoring of amr systems in Norway: Analysis of metered data and definition of the performance parameters , 2009 .

[13]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[14]  Hesham K. Alfares,et al.  Electric load forecasting: Literature survey and classification of methods , 2002, Int. J. Syst. Sci..

[15]  Farshid Keynia,et al.  Short-Term Load Forecast of Microgrids by a New Bilevel Prediction Strategy , 2010, IEEE Transactions on Smart Grid.

[16]  Kwang-Ho Kim,et al.  Implementation of hybrid short-term load forecasting system using artificial neural networks and fuzzy expert systems , 1995 .

[17]  Nima Amjady,et al.  Short-term hourly load forecasting using time-series modeling with peak load estimation capability , 2001 .

[18]  Jiguo Cao,et al.  Automated Load Curve Data Cleansing in Power Systems , 2010, IEEE Transactions on Smart Grid.

[19]  Gonzalo Mateos,et al.  Robust Nonparametric Regression via Sparsity Control With Application to Load Curve Data Cleansing , 2011, IEEE Transactions on Signal Processing.

[20]  S. Koopman,et al.  An Hourly Periodic State Space Model for Modelling French National Electricity Load , 2007 .

[21]  B. Silverman,et al.  Functional Data Analysis , 1997 .

[22]  Bodil Merethe Larsen,et al.  Formålsfordeling av husholdningenes elektrisitetsforbruk i 1990 og 2001 , 2005 .

[23]  Fionn Murtagh,et al.  Wavelet-based combined signal filtering and prediction , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.