Predicting sun spots using a layered perceptron neural network

Interest in neural networks has expanded rapidly in recent years. Selecting the best structure for a given task, however, remains a critical issue in neural-network design. Although the performance of a network clearly depends on its structure, the procedure for selecting the optimal structure has not been thoroughly investigated, it is well known that the number of hidden units must be sufficient to discriminate each observation correctly. A large number of hidden units requires extensive computational time for training and often times prediction results may not be as accurate as expected. This study attempts to apply the principal component analysis (PCA) to determine the structure of a multilayered neural network for time series forecasting problems. The main focus is to determine the number of hidden units for a multilayered feedforward network. One empirical experiment with sunspot data is used to demonstrate the usefulness of the proposed approach.

[1]  Jocelyn Sietsma,et al.  Creating artificial neural networks that generalize , 1991, Neural Networks.

[2]  Richard P. Brent,et al.  Fast training algorithms for multilayer neural nets , 1991, IEEE Trans. Neural Networks.

[3]  R.J.F. Dow,et al.  Neural net pruning-why and how , 1988, IEEE 1988 International Conference on Neural Networks.

[4]  Lorien Y. Pratt,et al.  Comparing Biases for Minimal Network Construction with Back-Propagation , 1988, NIPS.

[5]  M. Gutierrez,et al.  Estimating hidden unit number for two-layer perceptrons , 1989, International 1989 Joint Conference on Neural Networks.

[6]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[7]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[8]  D. Rumelhart,et al.  Generalization by weight-elimination applied to currency exchange rate prediction , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[9]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[10]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[11]  David B. Fogel.  An information criterion for optimal neural network selection , 1991, IEEE Trans. Neural Networks.

[12]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[13]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[14]  Chung-Jen Ho.  On Multi-Layered Connectionist Models: Adding Layers vs. Increasing Width , 1989, IJCAI.

[15]  Beat Kleiner,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[16]  D. F. Morrison,et al.  Multivariate Statistical Methods , 1968 .

[17]  Mark D. Plumbley.  Lyapunov functions for convergence of principal component algorithms , 1995, Neural Networks.

[18]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[19]  Hervé Bourlard,et al.  Generalization and Parameter Estimation in Feedforward Netws: Some Experiments , 1989, NIPS.

[20]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[21]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[22]  S. Y. Kung,et al.  An algebraic projection analysis for optimal hidden units size and learning rates in back-propagation learning , 1988, IEEE 1988 International Conference on Neural Networks.

[23]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Michael I. Jordan.  Supervised learning and systems with excess degrees of freedom , 1988 .

[25]  H. White,et al.  An additional hidden unit test for neglected nonlinearity in multilayer feedforward networks , 1989, International 1989 Joint Conference on Neural Networks.

[26]  Esther Levin,et al.  A recurrent neural network: Limitations and training , 1990, Neural Networks.

[27]  Yoshio Hirose,et al.  Backpropagation algorithm which varies the number of hidden units , 1989, International 1989 Joint Conference on Neural Networks.

[28]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[29]  H. Akaike.  A new look at the statistical model identification , 1974 .

[30]  Yves Chauvin,et al.  A Back-Propagation Algorithm with Optimal Use of Hidden Units , 1988, NIPS.

[31]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.