A SOM-based hybrid linear-neural model for short-term load forecasting

In this paper, a short-term load forecasting method is considered, which is based upon a flexible smooth transition autoregressive (STAR) model. The described model is a linear model with time varying coefficients, which are the outputs of a single hidden layer feedforward neural network. The hidden layer is responsible for partitioning the input space into multiple sub-spaces through multivariate thresholds and smooth transition between the sub-spaces. In this paper, we propose a new method to smartly initialize the weights of the hidden layer of the neural network before its training. A self-organizing map (SOM) network is applied to split the historical data dynamics into clusters, and the Ho-Kashyap algorithm is then used to obtain the separating planes' equations. Applied to the electricity markets, the proposed method is better able to model the smooth transitions between the different regimes, which are present in the load demand series because of market effects and season effects. We use data from three electricity markets to compare the prediction accuracy of the proposed method with traditional benchmarks and other recent models, and find our results to be competitive.

[1]  Paul Zarchan,et al.  Fundamentals of Kalman Filtering: A Practical Approach , 2001 .

[2]  Leonardo Franco,et al.  Neural Network Architecture Selection: Can Function Complexity Help? , 2009, Neural Processing Letters.

[3]  Marcelo C. Medeiros,et al.  A flexible coefficient smooth transition time series model , 2005, IEEE Transactions on Neural Networks.

[4]  H. Yoo,et al.  Short term load forecasting using a self-supervised adaptive neural network , 1999 .

[5]  Maxwell Stevenson,et al.  Filtering and Forecasting Spot Electricity Prices in the Increasingly Deregulated Australian Electricity Market , 2001 .

[6]  Terry Robinson.  Electricity pool prices: a case study in nonlinear time-series modelling , 2000 .

[7]  Mercedes Fernández-Redondo,et al.  A comparison among weight initialization methods for multilayer feedforward networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[8]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[9]  Augusto Montisci,et al.  Geometrical synthesis of MLP neural networks , 2008, Neurocomputing.

[10]  Shu Du,et al.  Short-Term Load Forecasting Using System-Type Neural Network Architecture , 2009, 2009 GSW Proceedings.

[11]  Mercedes Fernández-Redondo,et al.  Weight initialization methods for multilayer feedforward , 2001, ESANN.

[12]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[13]  H. Tong.  On a threshold model , 1978 .

[14]  Emile Fiesler,et al.  High-order and multilayer perceptron initialization , 1997, IEEE Trans. Neural Networks.

[15]  S. Huang,et al.  Short-term load forecasting using threshold autoregressive models , 1997 .

[16]  Michel Verleysen,et al.  Time series forecasting: Obtaining long term trends with self-organizing maps , 2005, Pattern Recognit. Lett..

[17]  W. Charytoniuk,et al.  Nonparametric regression based short-term load forecasting , 1998 .

[18]  T. Teräsvirta.  Specification, Estimation, and Evaluation of Smooth Transition Autoregressive Models , 1994 .

[19]  T. Hesterberg,et al.  A regression-based approach to short-term system load forecasting , 1989, Conference Papers Power Industry Computer Application Conference.

[20]  Michel Verleysen,et al.  Double quantization of the regressor space for long-term time series prediction: method and proof of stability , 2004, Neural Networks.

[21]  S. J. Kiartzis,et al.  A neural network short term load forecasting model for the Greek power system , 1996 .

[22]  Pedro Paulo Balestrassi,et al.  Electricity demand and spot price forecasting using evolutionary computation combined with chaotic nonlinear dynamic model , 2010 .

[23]  David G. Stork,et al.  Pattern Classification , 1973 .

[24]  H. Kantz,et al.  Nonlinear time series analysis , 1997 .

[25]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[26]  Shuyang Du,et al.  Short-Term Load Forecasting Using Semigroup Based System-Type Neural Network , 2009, 2009 15th International Conference on Intelligent System Applications to Power Systems.

[27]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[28]  Vojislav Kecman,et al.  Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models , 2001 .

[29]  Lutz Prechelt,et al.  Early Stopping-But When? , 1996, Neural Networks: Tricks of the Trade.

[30]  Ljubo B. Vlacic,et al.  Learning and Soft Computing, Support Vector Machines, Neural Networks, and Fuzzy Logic Models, Vojislav Kecman; MIT Press, Cambridge, MA, 2001, ISBN 0-262-11255-8, 2001, pp 578 , 2002, Neurocomputing.

[31]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[32]  Marcelo C. Medeiros,et al.  A hybrid linear-neural model for time series forecasting , 2000, IEEE Trans. Neural Networks Learn. Syst..

[33]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[34]  Rudy Setiono,et al.  Feedforward Neural Network Construction Using Cross Validation , 2001, Neural Computation.

[35]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[36]  Shyh-Jier Huang,et al.  Short-term load forecasting via ARMA model identification including non-Gaussian process considerations , 2003 .

[37]  Stephan Trenn,et al.  Multilayer Perceptrons: Approximation Order and Necessary Number of Hidden Units , 2008, IEEE Transactions on Neural Networks.

[38]  Howell Tong,et al.  Threshold autoregression, limit cycles and cyclical data- with discussion , 1980 .

[39]  John F. Kolen,et al.  Backpropagation is Sensitive to Initial Conditions , 1990, Complex Syst..

[40]  João Cesar M. Mota,et al.  Nonstationary Time Series Prediction Using Local Models Based on Competitive Neural Networks , 2004, IEA/AIE.

[41]  Chi Hau Chen,et al.  Pattern recognition and signal processing , 1978 .

[42]  Raymond Turner,et al.  Specification , 2011, Minds and Machines.

[43]  M. G. Currie.  An optimized filter architecture incorporating a neural net , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[44]  Esa Alhoniemi,et al.  Clustering of the self-organizing map , 2000, IEEE Trans. Neural Networks Learn. Syst..

[45]  Paul Newbold,et al.  Unit roots and smooth transitions , 1998 .

[46]  Remy Cottet,et al.  Bayesian Modeling and Forecasting of Intraday Electricity Load , 2003 .

[47]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[48]  T. Funabashi,et al.  One-Hour-Ahead Load Forecasting Using Neural Networks , 2002 .

[49]  Reinaldo Castro Souza,et al.  A smooth transition periodic autoregressive (STPAR) model for short-term load forecasting , 2008 .

[50]  Timo Teräsvirta,et al.  Testing linearity against smooth transition autoregressive models , 1988 .

[51]  Kwang Y. Lee,et al.  Short-Term Load Forecasting Using System-Type Neural Network Architecture , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[52]  H. Tong,et al.  ON ESTIMATING THRESHOLDS IN AUTOREGRESSIVE MODELS , 1986 .