Comparing and Combining MLP and NEAT for Time Series Forecasting

Neural networks are one of the widely-used time series forecasting methods in time series applications. Among different neural network architectures and learning algorithms, the most popular choice is the feedforward Multilayer Perceptron (MLP). However, it suffers from some drawbacks such as getting trapped in local minima, human intervention during the stage of training, and limitations in architecture design. The aims of this study were twofold. The first was to employ NeuroEvolution of Augmenting Topologies (NEAT), which has many successful applications in numerous fields. In this paper, we applied it to time series forecasting for the first time and compared its performance with that of the MLP. The second aim was to analyse the performance resulting from the pairwise combination of these methods. In general, the results suggested that the forecasts from the NEAT algorithm were more accurate than those of the MLP. The results also showed that pairwise combined forecasts in general were better than single forecasts. The best forecasts of all were obtained by pairwise combination of MLP and NEAT.

[1]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[2]  Claas de Groot,et al.  Analysis of univariate time series with connectionist nets: A case study of two classical examples , 1991, Neurocomputing.

[3]  Serkan Aras,et al.  A new model selection strategy in time series forecasting with artificial neural networks: IHTS , 2016, Neurocomputing.

[4]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[5]  Vassilis S. Kodogiannis,et al.  Forecasting Financial Time Series using Neural Network and Fuzzy System-based Techniques , 2002, Neural Computing & Applications.

[6]  W. Enders.  Applied Econometric Time Series , 1994 .

[7]  C. Granger.  Invited review combining forecasts—twenty years later , 1989 .

[8]  David Horn,et al.  Combined Neural Networks for Time Series Analysis , 1993, NIPS.

[9]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[10]  J. Gooijer,et al.  Some recent developments in non-linear time series modelling, testing, and forecasting☆ , 1992 .

[11]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[12]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[13]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[14]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[15]  B. L. Welch.  The generalisation of student's problems when several different population variances are involved. , 1947, Biometrika.

[16]  Risto Miikkulainen,et al.  Competitive Coevolution through Evolutionary Complexification , 2011, J. Artif. Intell. Res..

[17]  César Hervás-Martínez,et al.  COVNET: a cooperative coevolutionary model for evolving artificial neural networks , 2003, IEEE Trans. Neural Networks.

[18]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[19]  Guoqiang Peter Zhang,et al.  An investigation of neural networks for linear time-series forecasting , 2001, Comput. Oper. Res..

[20]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[21]  Jens Ove Riis,et al.  A hybrid econometric—neural network modeling approach for sales forecasting , 1996 .

[22]  R. Clemen.  Combining forecasts: A review and annotated bibliography , 1989 .

[23]  Risto Miikkulainen,et al.  Neuroevolution of an automobile crash warning system , 2005, GECCO '05.

[24]  Ron S. Kenett,et al.  Statistics for Business and Economics , 1973 .

[25]  Hak-Keung Lam,et al.  Tuning of the structure and parameters of a neural network using an improved genetic algorithm , 2003, IEEE Trans. Neural Networks.

[26]  Risto Miikkulainen,et al.  Coevolving Strategies for General Game Playing , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[27]  Daihai He,et al.  Chaotic oscillations and cycles in multi-trophic ecological systems. , 2007, Journal of theoretical biology.

[28]  Mehdi Khashei,et al.  An artificial neural network (p, d, q) model for timeseries forecasting , 2010, Expert Syst. Appl..

[29]  Krzysztof J. Cios,et al.  Time series forecasting by combining RBF networks, certainty factors, and the Box-Jenkins model , 1996, Neurocomputing.

[30]  J. Scott Armstrong,et al.  Combining forecasts: The end of the beginning or the beginning of the end? , 1989 .

[31]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[32]  F. E. Satterthwaite.  An approximate distribution of estimates of variance components. , 1946, Biometrics.

[33]  D. J. Reid.  Combining Three Estimates of Gross Domestic Product , 1968 .

[34]  Milton S. Boyd,et al.  Designing a neural network for forecasting financial and economic time series , 1996, Neurocomputing.

[35]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[36]  C. Granger,et al.  Experience with Forecasting Univariate Time Series and the Combination of Forecasts , 1974 .

[37]  Fang-Mei Tseng,et al.  Combining neural network model with seasonal time series ARIMA model , 2002 .

[38]  G. Keppel,et al.  Design and Analysis: A Researcher's Handbook , 1976 .

[39]  Arnold Zellner,et al.  To combine or not to combine? Issues of combining forecasts , 1992 .

[40]  Guoping Xia,et al.  An investigation and comparison of artificial neural network and time series models for Chinese food grain price forecasting , 2007, Neurocomputing.

[41]  W. Li,et al.  On a mixture autoregressive model , 2000 .

[42]  J. David Fuller,et al.  Back propagation in time‐series forecasting , 1995 .

[43]  Michael Y. Hu,et al.  A simulation study of artificial neural networks for nonlinear time-series forecasting , 2001, Comput. Oper. Res..

[44]  M. B. Priestley,et al.  Non-linear and non-stationary time series analysis , 1990 .

[45]  J. Stock,et al.  A Comparison of Linear and Nonlinear Univariate Models for Forecasting Macroeconomic Time Series , 1998 .

[46]  J. Scott Armstrong,et al.  On the Selection of Error Measures for Comparisons Among Forecasting Methods , 2005 .

[47]  Robert L. Winkler,et al.  The accuracy of extrapolation (time series) methods: Results of a forecasting competition , 1982 .

[48]  L. Cooper,et al.  When Networks Disagree: Ensemble Methods for Hybrid Neural Networks , 1992 .

[49]  Manoochehr Ghiassi,et al.  A dynamic architecture for artificial neural networks , 2005, Neurocomputing.

[50]  Shiro Usui,et al.  Mutation-based genetic neural network , 2005, IEEE Transactions on Neural Networks.

[51]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[52]  Konstantinos Nikolopoulos,et al.  Forecasting with cue information: A comparison of multiple regression with alternative forecasting approaches , 2007, Eur. J. Oper. Res..

[53]  Derek W. Bunn,et al.  Review of guidelines for the use of combined forecasts , 2000, Eur. J. Oper. Res..

[54]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[55]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .