A Neural Net Model for Prediction

Abstract In this article we introduce a neural net designed for nonlinear statistical prediction. The net is based on a stochastic model featuring a multilayer feedforward architecture with random connections between units and noisy response functions. A Bayesian inferential procedure for this model, based on the Kalman filter, is derived. The resulting learning algorithm generalizes the so-called onedimensional Newton method, an updating algorithm currently popular in the neural net literature. A numerical study concerning the prediction of a noisy chaotic time series is presented, and the greater predictive accuracy of the new algorithm with respect to the Newton algorithm is exhibited.

[1]  A. Lapedes,et al.  Nonlinear Signal Processing Using Neural Networks , 1987 .

[2]  M. B. Priestley,et al.  Non-linear and non-stationary time series analysis , 1990 .

[3]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[4]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[5]  Richard J. Meinhold,et al.  Robustification of Kalman Filter Models , 1989 .

[6]  R. Berk,et al.  Limiting Behavior of Posterior Distributions when the Model is Incorrect , 1966 .

[7]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[8]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[9]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[10]  Masanao Aoki,et al.  State Space Modeling of Time Series , 1987 .

[11]  M. Priestley.  STATE‐DEPENDENT MODELS: A GENERAL APPROACH TO NON‐LINEAR TIME SERIES ANALYSIS , 1980 .

[12]  P. S. Lewis,et al.  Function approximation and time series prediction with neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[13]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[14]  M. West,et al.  Dynamic Generalized Linear Models and Bayesian Forecasting , 1985 .

[15]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[16]  James L. McClelland,et al.  James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.

[17]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[18]  P. D. Jong.  The Diffuse Kalman Filter , 1991 .

[19]  J. Doyne Farmer,et al.  Exploiting Chaos to Predict the Future and Reduce Noise , 1989 .

[20]  Martin Casdagli,et al.  An analytic approach to practical state space reconstruction , 1992 .

[21]  H. White.  Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models , 1989 .