An improved time series prediction by applying the layer-by-layer learning method to FIR neural networks

Abstract The FIR neural network model was recently proposed for time series prediction and gave good results. However, the learning algorithm used for the FIR network is a kind of gradient descent method and hence inherits all the well-known problems of the method. Recently a new learning algorithm called the optimization layer by layer was proposed for the regular multilayer perceptron network, and showed a great improvement in the learning time as well as the performance of the network. In this paper we develop a new learning algorithm for the FIR neural network model by applying the idea of the optimization layer by layer to the model. The results of the experiment, using two popular time series prediction problems, show that the new algorithm is far better in learning time and more accurate in prediction performance than the original learning algorithm.

[1]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[2]  Paul A. Fishwick,et al.  Time series forecasting using neural networks vs. Box- Jenkins methodology , 1991, Simul..

[3]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[4]  Eric A. Wan,et al.  Time series prediction by using a connectionist network with internal delay lines , 1993 .

[5]  Ah Chung Tsoi,et al.  FIR and IIR Synapses, a New Neural Network Architecture for Time Series Modeling , 1991, Neural Computation.

[6]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[7]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[8]  S. Ergezinger,et al.  An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer , 1995, IEEE Trans. Neural Networks.

[9]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[10]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[11]  D Kleinfeld,et al.  Sequential state generation by model neural networks. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[13]  Kishan G. Mehrotra,et al.  Forecasting the behavior of multivariate time series using neural networks , 1992, Neural Networks.

[14]  Jing Peng,et al.  An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories , 1990, Neural Computation.

[15]  Herz.  Global analysis of parallel analog networks with retarded feedback. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[16]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[17]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[18]  Eric A. Wan,et al.  Temporal backpropagation for FIR neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[19]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[20]  Michael R. Davenport,et al.  Continuous-time temporal back-propagation with adaptable time delays , 1993, IEEE Trans. Neural Networks.

[21]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[22]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..