Time series forecasting by evolving artificial neural networks with genetic algorithms, differential evolution and estimation of distribution algorithm

Time series forecasting is an important tool to support both individual and organizational decisions (e.g. planning production resources). In recent years, a large literature has evolved on the use of evolutionary artificial neural networks (EANN) in many forecasting applications. Evolving neural networks are particularly appealing because of their ability to model an unspecified nonlinear relationship between time series variables. In this work, two new approaches of a previous system, automatic design of artificial neural networks (ADANN) applied to forecast time series, are tackled. In ADANN, the automatic process to design artificial neural networks was carried out by a genetic algorithm (GA). This paper evaluates three methods to evolve neural networks architectures, one carried out with genetic algorithm, a second one carried out with differential evolution algorithm (DE) and the last one using estimation of distribution algorithms (EDA). A comparative study among these three methods with a set of referenced time series will be shown. In this paper, we also compare ADANN forecasting ability against a forecasting tool called Forecast Pro® (FP) software, using five benchmark time series. The object of this study is to try to improve the final forecasting getting an accurate system.

[1]  Frédéric Gruau,et al.  Genetic synthesis of Boolean neural networks with a cell rewriting developmental process , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[2]  Xiaodong Li,et al.  Choosing Leaders for Multi-objective PSO Algorithms Using Differential Evolution , 2008, SEAL.

[3]  Araceli Sanchis,et al.  ADANN: automatic design of artificial neural networks , 2008, GECCO '08.

[4]  Amir F. Atiya,et al.  A new Bayesian formulation for Holt's exponential smoothing , 2009 .

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Ajith Abraham,et al.  An ensemble of neural networks for weather forecasting , 2004, Neural Computing & Applications.

[7]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[8]  Pedro Larrañaga,et al.  Adjusting Weights in Artificial Neural Networks using Evolutionary Algorithms , 2002, Estimation of Distribution Algorithms.

[9]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[10]  Timur Ash,et al.  Dynamic node creation in backpropagation networks , 1989 .

[11]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[12]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[13]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[14]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[15]  Jiwen Dong,et al.  Time-series prediction using a local linear wavelet neural network , 2006, Neurocomputing.

[16]  Tiago Alessandro Espínola Ferreira,et al.  A New Intelligent System Methodology for Time Series Forecasting with Artificial Neural Networks , 2008, Neural Processing Letters.

[17]  Nikolaos Kourentzes,et al.  Feature selection for time series prediction - A combined filter and wrapper approach for neural networks , 2010, Neurocomputing.

[18]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[19]  Juan Julián Merelo Guervós,et al.  Evolving RBF neural networks for time-series forecasting with EvRBF , 2004, Inf. Sci..

[20]  Fi-John Chang,et al.  Evolutionary artificial neural networks for hydrological systems forecasting , 2009 .

[21]  Martin Stepnicka,et al.  A linguistic approach to time series modeling with the help of F-transform , 2011, Fuzzy Sets Syst..

[22]  H. M. Abdul,et al.  Neural Networks Training Based on Differential Evolution Algorithm Compared with Other Architectures for Weather Forecasting34 , 2009 .

[23]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[24]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[25]  Baikunth Nath,et al.  A fusion model of HMM, ANN and GA for stock market forecasting , 2007, Expert Syst. Appl..

[26]  Juan R. Rabuñal,et al.  Artificial Neural Networks in Real-Life Applications , 2005 .

[27]  D. B. Fogel,et al.  Evolving neural networks , 1990, Biological Cybernetics.

[28]  Araceli Sanchis,et al.  Time series forecasting by evolving artificial neural networks using genetic algorithms and estimation of distribution algorithms , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[29]  Paulo Cortez,et al.  Sensitivity analysis for time lag selection to forecast seasonal time series using Neural Networks and Support Vector Machines , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[30]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[31]  Steven C. Wheelwright,et al.  Forecasting methods and applications. , 1979 .

[32]  Larry J. Eshelman,et al.  Using genetic search to exploit the emergent behavior of neural networks , 1990 .

[33]  J. Scott Armstrong,et al.  Long-Range Forecasting. , 1979 .

[34]  Xiaodong Li,et al.  Time series forecasting by evolving artificial neural networks using genetic algorithms and differential evolution , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[35]  Tony White,et al.  The application of antigenic search techniques to time series forecasting , 2005, GECCO '05.

[36]  S.F. Crone,,et al.  Stepwise Selection of Artificial Neural Network Models for Time Series Prediction , 2005 .

[37]  J. G. Settle,et al.  Long‐Range Forecasting , 1979 .

[38]  Mikko Kolehmainen,et al.  Evolving the neural network model for forecasting air pollution time series , 2004, Eng. Appl. Artif. Intell..

[39]  Tiago Alessandro Espínola Ferreira,et al.  Hybrid differential evolutionary system for financial time series forecasting , 2007, 2007 IEEE Congress on Evolutionary Computation.

[40]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[41]  Miguel Rocha,et al.  Time Series Forecasting by Evolutionary Neural Networks , 2005 .

[42]  José Neves,et al.  Evolving Time Series Forecasting ARMA Models , 2004, J. Heuristics.

[43]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .