Forecasting with genetically programmed polynomial neural networks

Abstract Recent literature on nonlinear models has shown genetic programming to be a potential tool for forecasters. A special type of genetically programmed model, namely polynomial neural networks, is addressed. Their outputs are polynomials and, as such, they are open boxes that are amenable to comprehension, analysis, and interpretation. This paper presents a polynomial neural network forecasting system, PGP, which has three innovative features: polynomial block reformulation, local ridge regression for weight estimation, and regularized weight subset selection for pruning that uses a least absolute shrinkage and selection operator. The relative performance of this system to other established forecasting procedures is the focus of this research and is illustrated by three empirical studies. Overall, the results are very promising and indicate areas for further research.

[1]  William Remus,et al.  Neural Networks for Time-Series Forecasting , 2001 .

[2]  Franklin Allen,et al.  Using genetic algorithms to find technical trading rules , 1999 .

[3]  Derek W. Bunn,et al.  Large neural networks for electricity load forecasting: Are they overfitted? , 2005 .

[4]  Achilleas Zapranis,et al.  Principles of Neural Model Identification, Selection and Adequacy: With Applications to Financial Econometrics , 1999 .

[5]  Chris Chatfield,et al.  Time series forecasting with neural networks: a comparative study using the air line data , 2008 .

[6]  Rajkumar Venkatesan,et al.  A genetic algorithms approach to growth phase forecasting of wireless subscribers , 2002 .

[7]  Martin D. Fraser,et al.  Network models for control and processing , 2000 .

[8]  J. Scott Armstrong,et al.  Principles of forecasting , 2001 .

[9]  Curtis F. Gerald.  Applied numerical analysis , 1970 .

[10]  Malcolm I. Heywood,et al.  A framework for improved training of Sigma-Pi networks , 1995, IEEE Trans. Neural Networks.

[11]  Mark J. L. Orr,et al.  Regularization in the Selection of Radial Basis Function Centers , 1995, Neural Computation.

[12]  C. Granger,et al.  Modelling Nonlinear Economic Relationships , 1995 .

[13]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[14]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[15]  W. Charytoniuk,et al.  Very short-term load forecasting using artificial neural networks , 2000 .

[16]  Ludwig Kanzler,et al.  Very Fast and Correctly Sized Estimation of the Bds Statistic , 1999 .

[17]  Donald E. Brown,et al.  Induction and polynomial networks , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[18]  B. LeBaron,et al.  Nonlinear Dynamics, Chaos, and Instability: Statistical Theory and Economic Evidence , 1991 .

[19]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[20]  Carlos E. Pedreira,et al.  Neural networks for short-term load forecasting: a review and evaluation , 2001 .

[21]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[22]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[23]  M. Schetzen.  The Volterra and Wiener Theories of Nonlinear Systems , 1980 .

[24]  B. LeBaron,et al.  A test for independence based on the correlation dimension , 1996 .

[25]  Hitoshi Iba,et al.  Regularization approach to inductive genetic programming , 2001, IEEE Trans. Evol. Comput..

[26]  M. Kaboudan.  Genetic Programming Prediction of Stock Prices , 2000 .

[27]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[28]  Vasilis Z. Marmarelis,et al.  Volterra models and three-layer perceptrons , 1997, IEEE Trans. Neural Networks.

[29]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[30]  Georges A. Darbellay,et al.  Forecasting the short-term demand for electricity: Do neural networks stand a better chance? , 2000 .

[31]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[32]  Shu-Heng Chen,et al.  Evolutionary Artificial Neural Networks and Genetic Programming: A Comparative Study Based on Financial Data , 1997, ICANNGA.

[33]  A. Barron,et al.  Discussion: Multivariate Adaptive Regression Splines , 1991 .

[34]  P. D. Lima,et al.  On the robustness of nonlinearity tests to moment condition failure , 1997 .

[35]  François E. Cellier,et al.  Artificial Neural Networks and Genetic Algorithms , 1991 .

[36]  G. Wahba.  Spline models for observational data , 1990 .