Meteorological time series forecasting with pruned multi-layer perceptron and two-stage Levenberg-Marquardt method

A Multi-Layer Perceptron (MLP) defines a family of artificial neural networks often used in TS modeling and forecasting. Because of its "black box" aspect, many researchers refuse to use it. Moreover, the optimization (often based on the exhaustive approach where "all" configurations are tested) and learning phases of this artificial intelligence tool (often based on the Levenberg-Marquardt algorithm; LMA) are weaknesses of this approach (exhaustively and local minima). These two tasks must be repeated depending on the knowledge of each new problem studied, making the process, long, laborious and not systematically robust. In this paper a pruning process is proposed. This method allows, during the training phase, to carry out an inputs selecting method activating (or not) inter-nodes connections in order to verify if forecasting is improved. We propose to use iteratively the popular damped least-squares method to activate inputs and neurons. A first pass is applied to 10% of the learning sample to determine weights significantly different from 0 and delete other. Then a classical batch process based on LMA is used with the new MLP. The validation is done using 25 measured meteorological TS and cross-comparing the prediction results of the classical LMA and the 2-stage LMA.

[1]  Yang Li,et al.  Adaptive neural network control for a class of strict feedback non-linear time delay systems , 2014, Int. J. Model. Identif. Control..

[2]  Gilles Notton,et al.  Numerical weather prediction or stochastic modeling: an objective criterion of choice for the global radiation forecasting , 2014, ArXiv.

[3]  Mohamed M. Ismail Ali,et al.  Efficiency optimisation with PI gain adaptation of field-oriented control applied on five phase induction motor using AI technique , 2013, Int. J. Model. Identif. Control..

[4]  Cyril Voyant,et al.  Multi-horizon solar radiation forecasting for Mediterranean locations using time series models , 2013, ArXiv.

[5]  Qi Hua,et al.  Neural network and PSO-based structural approximation analysis for blade of wind turbine , 2013, Int. J. Model. Identif. Control..

[6]  Cyril Voyant,et al.  Numerical Weather Prediction (NWP) and hybrid ARMA/ANN model to predict global radiation , 2012, ArXiv.

[7]  Efstathios Paparoditis,et al.  Bootstrap methods for dependent data: A review , 2011 .

[8]  Yi-Qing Ni,et al.  Damage detection utilising the artificial neural network methods to a benchmark structure , 2011 .

[9]  A. Troccoli,et al.  Seasonal climate forecasting , 2010 .

[10]  Nikolaos Kourentzes,et al.  Feature selection for time series prediction - A combined filter and wrapper approach for neural networks , 2010, Neurocomputing.

[11]  Haiyan Lu,et al.  Combined modeling for electric load forecasting with adaptive particle swarm optimization , 2010 .

[12]  Cyril Voyant,et al.  Predictability of PV power grid performance on insular sites without weather stations: use of artificial neural networks , 2009, ArXiv.

[13]  Soteris A. Kalogirou,et al.  Artificial intelligence techniques for sizing photovoltaic systems: A review , 2009 .

[14]  Jinyan Fan,et al.  A note on the Levenberg-Marquardt parameter , 2009, Appl. Math. Comput..

[15]  G. Riddihough.  Damage Detection , 2008, Science Signaling.

[16]  Philippe Lauret,et al.  Bayesian neural network approach to short time load forecasting , 2008 .

[17]  Kwok-wing Chau,et al.  A hybrid adaptive time-delay neural network model for multi-step-ahead prediction of sunspot activity , 2006 .

[18]  Vasilis D. Valavanis,et al.  Time series analysis and forecasting techniques applied on loliginid and ommastrephid landings in Greek waters , 2006 .

[19]  Fernando Morgado Dias,et al.  A sliding window solution for the on-line implementation of the Levenberg-Marquardt algorithm , 2006, Eng. Appl. Artif. Intell..

[20]  ChangKyoo Yoo,et al.  Generalized damped least squares algorithm , 2003, Comput. Chem. Eng..

[21]  Michael Lewis,et al.  Time Series Analysis , 1994, Statistics for Environmental Science and Management.

[22]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[23]  Jean-Pierre Petit,et al.  On the convergence of standard and damped least squares methods , 1976 .

[24]  Rob J Hyndman,et al.  25 years of time series forecasting , 2006 .

[25]  S.F. Crone,,et al.  Stepwise Selection of Artificial Neural Network Models for Time Series Prediction , 2005 .

[26]  猛 小林,et al.  Bayesian Neural Network によるヒト白血病患者のクラス分類と事後確率による評価 , 2005 .

[27]  M. C. Deo,et al.  Forecasting wind with neural networks , 2003 .