Time-Series Prediction

Publisher Summary An artificial neural network is a highly connected array of elementary processors called neurons. Neural networks have been extensively investigated and have been successfully applied to a variety of areas; one of these areas is time-series prediction. This chapter deals with neural networks using a supervised learning procedure that can learn a nonlinear relationship between an input and the corresponding output based on the desired or target output. Such neural networks can learn the nonlinear relationship by using the past value of the time series as the input and the desired output, and can implicitly construct the underlying model required for time-series prediction. This chapter discusses three types of neural networks: multilayer perceptrons, finite impulse response (FIR) multilayer perceptrons, and recurrent neural networks. The multilayer perceptron consists of the input layer of distribution nodes, one or more hidden layer of computation nodes, and an output layer of computation nodes. The input signal into the input layer passes through the hidden layers to the output layer in a forward direction, on a layer-by-layer basis. FIR multilayer perceptrons can be constructed by replacing synaptic weights with FIR synaptic filters in the structure of the standard multilayer perceptrons. The recurrent neural networks use an ordinary model of a neuron, but the networks develop a temporal processing capability through feedback built into the architecture. This chapter examines the aforementioned neural networks using training algorithms and applications.

[1]  R. Shumway.  Applied Statistical Time Series Analysis , 1988 .

[2]  George Sugihara,et al.  Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series , 1990, Nature.

[3]  Andreas S. Weigend,et al.  The Future of Time Series: Learning and Understanding , 1993 .

[4]  Eric A. Wan.  Temporal Backpropagation: An Efficient Algorithm for Finite Impulse Response Neural Networks , 1991 .

[5]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[6]  B. Irie,et al.  Capabilities of three-layered perceptrons , 1988, IEEE 1988 International Conference on Neural Networks.

[7]  Les Atlas,et al.  Recurrent neural networks and time series prediction , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[8]  Robert J. Marks,et al.  Electric load forecasting using an artificial neural network , 1991 .

[9]  Les E. Atlas,et al.  Recurrent Networks and NARMA Modeling , 1991, NIPS.

[10]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[11]  Kishan G. Mehrotra,et al.  Forecasting the behavior of multivariate time series using neural networks , 1992, Neural Networks.

[12]  Hisashi Shimodaira.  A method for selecting similar learning data in the prediction of time series using neural networks , 1996 .

[13]  Barak A. Pearlmutter.  Learning state space trajectories in recurrent neural networks : a preliminary report. , 1988 .

[14]  L. K. Li.  Approximation theory and recurrent networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[15]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[16]  Eric A. Wan,et al.  Temporal backpropagation for FIR neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[17]  Daniel M. Wolpert,et al.  Detecting chaos with neural networks , 1990, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[18]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[19]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[20]  Paul A. Fishwick,et al.  Time series forecasting using neural networks vs. Box- Jenkins methodology , 1991, Simul..

[21]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[22]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[23]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[24]  Hisashi Shimodaira.  A method of selecting learning data in the prediction of time series with explanatory variables using neural networks , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[25]  J. Yorke,et al.  Recurrent outbreaks of measles, chickenpox and mumps. I. Seasonal variation in contact rates. , 1973, American journal of epidemiology.

[26]  George G. Karady,et al.  Advancement in the application of neural networks for short-term load forecasting , 1992 .

[27]  G. G. Karady,et al.  Conceptual approach to the application of neural network for short-term load forecasting , 1990, IEEE International Symposium on Circuits and Systems.

[28]  T. Onoda.  Next day's peak load forecasting using an artificial neural network , 1993, [1993] Proceedings of the Second International Forum on Applications of Neural Networks to Power Systems.

[29]  A. N. Sharkovskiĭ.  Dynamic systems and turbulence , 1989 .

[30]  James P. Crutchfield,et al.  Geometry from a Time Series , 1980 .

[31]  S. Y. Kung,et al.  An algebraic projection analysis for optimal hidden units size and learning rates in back-propagation learning , 1988, IEEE 1988 International Conference on Neural Networks.

[32]  Thierry Catfolis,et al.  A method for improving the real-time recurrent learning algorithm , 1993, Neural Networks.

[33]  Thomas Jackson,et al.  Neural Computing - An Introduction , 1990 .

[34]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[35]  Garrison W. Cottrell,et al.  Learning in recurrent finite difference networks , 1995, Int. J. Neural Syst..

[36]  R. Hecht-Nielsen,et al.  Application of feedforward and recurrent neural networks to chemical plant predictive modeling , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[37]  Jacques de Villiers,et al.  Backpropagation neural nets with one and two hidden layers , 1993, IEEE Trans. Neural Networks.

[38]  Yih-Fang Huang,et al.  Bounds on the number of hidden neurons in multilayer perceptrons , 1991, IEEE Trans. Neural Networks.