Dynamical recurrent neural networks -- towards environmental time series prediction

Dynamical Recurrent Neural Networks (DRNN) (Aussem 1995a) are a class of fully recurrent networks obtained by modeling synapses as autoregressive filters. By virtue of their internal dynamic, these networks approximate the underlying law governing the time series by a system of nonlinear difference equations of internal variables. They therefore provide history-sensitive forecasts without having to be explicitly fed with external memory. The model is trained by a local and recursive error propagation algorithm called temporal-recurrent-backpropagation. The efficiency of the procedure benefits from the exponential decay of the gradient terms backpropagated through the adjoint network. We assess the predictive ability of the DRNN model with meterological and astronomical time series recorded around the candidate observation sites for the future VLT telescope. The hope is that reliable environmental forecasts provided with the model will allow the modern telescopes to be preset, a few hours in advance, in the most suited instrumental mode. In this perspective, the model is first appraised on precipitation measurements with traditional nonlinear AR and ARMA techniques using feedforward networks. Then we tackle a complex problem, namely the prediction of astronomical seeing, known to be a very erratic time series. A fuzzy coding approach is used to reduce the complexity of the underlying laws governing the seeing. Then, a fuzzy correspondence analysis is carried out to explore the internal relationships in the data. Based on a carefully selected set of meteorological variables at the same time-point, a nonlinear multiple regression, termed nowcasting (Murtagh et al. 1993, 1995), is carried out on the fuzzily coded seeing records. The DRNN is shown to outperform the fuzzy k-nearest neighbors method.

[1]  Kevin J. Lang.  A time delay neural network architecture for speech recognition , 1989 .

[2]  Marc S. Sarazin,et al.  Dynamical Recurrent Neural Networks and Pattern Recognition Methods for Time Series Prediction - Application to Seeing and Temperature Forecasting in the Context of Eso's VLT Astronomical Weather Station , 1994 .

[3]  S. W. Piche,et al.  Steepest descent algorithms for neural network controllers and filters , 1994, IEEE Trans. Neural Networks.

[4]  Marc S. Sarazin,et al.  NOWCASTING ASTRONOMICAL SEEING: A STUDY OF ESO LA SILLA AND PARANAL , 1993 .

[5]  Ah Chung Tsoi,et al.  Locally recurrent globally feedforward networks: a critical review of architectures , 1994, IEEE Trans. Neural Networks.

[6]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[7]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[8]  Andreas S. Weigend,et al.  The Future of Time Series: Learning and Understanding , 1993 .

[9]  Marc S. Sarazin,et al.  NOWCASTING ASTRONOMICAL SEEING: TOWARDS AN OPERATIONAL APPROACH , 1995 .

[10]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[11]  L. B. Almeida,et al.  BACKPROPAGATION IN PERCEPTRONS WITH FEEDBACK , 2022 .

[12]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[13]  Ronald J. Williams,et al.  Experimental Analysis of the Real-time Recurrent Learning Algorithm , 1989 .

[14]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[15]  Mw Hirsch,et al.  Chaos In Dynamical Systems , 2016 .

[16]  E K Hege,et al.  Evidence of a chaotic attractor in star-wander data. , 1991, Optics letters.

[17]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[18]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[19]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[20]  Sadaaki Miyamoto,et al.  Fuzzy Sets in Information Retrieval and Cluster Analysis , 1990, Theory and Decision Library.

[21]  Marc S. Sarazin,et al.  Statistical Prediction of Astronomical Seeing and of Telescope Thermal Environment , 1992 .

[22]  Fionn Murtagh,et al.  Fuzzy astronomical seeing nowcasts with a dynamical and recurrent connectionist network , 1996, Neurocomputing.

[23]  U. R. Prasad,et al.  Back propagation through adjoints for the identification of nonlinear dynamic systems using recurrent neural models , 1994, IEEE Trans. Neural Networks.

[24]  R. R. Leighton,et al.  The autoregressive backpropagation algorithm , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.