Refining accuracy of environmental data prediction by MoG neural networks

Abstract The prediction of future values of environmental data sequences is mandatory to the cost-effective management of available resources. Consequently, the possibility to improve the prediction accuracy is a very important goal to be pursued. We propose in the present paper two possible approaches for refining the prediction accuracy on real data sequences. Both these approaches make use of Mixture of Gaussian neural networks for the solution of suitable function approximation problems. The first approach pursues the regularization of the learning process based on the reconstructed state of the context delivering the sequence; the second one is based on the particular chaotic nature of the prediction error.

[1]  Halbert White,et al.  Learning in Artificial Neural Networks: A Statistical Perspective , 1989, Neural Computation.

[2]  RefenesApostolos Nicholas,et al.  Stock performance modeling using neural networks , 1994 .

[3]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[4]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[5]  Vladimir Cherkassky,et al.  Comparison of adaptive methods for function estimation from samples , 1996, IEEE Trans. Neural Networks.

[6]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[7]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[8]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[9]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[10]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[11]  Thomas M. Cover,et al.  Estimation by the nearest neighbor rule , 1968, IEEE Trans. Inf. Theory.

[12]  D. N. Geary.  Mixture Models: Inference and Applications to Clustering , 1989 .

[13]  Henry D. I. Abarbanel,et al.  Analysis of Observed Chaotic Data , 1995 .

[14]  Fraser,et al.  Independent coordinates for strange attractors from mutual information. , 1986, Physical review. A, General physics.

[15]  Jerome H. Friedman,et al.  An Overview of Predictive Learning and Function Approximation , 1994 .

[16]  J. Príncipe,et al.  Local dynamic modeling with self-organizing maps and applications to nonlinear system identification and control , 1998, Proc. IEEE.

[17]  Bart Kosko,et al.  Fuzzy Systems as Universal Approximators , 1994, IEEE Trans. Computers.

[18]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[19]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[20]  Brown,et al.  Computing the Lyapunov spectrum of a dynamical system from an observed time series. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[21]  Simon Haykin,et al.  Making sense of a complex world , 1998 .

[22]  H. Abarbanel,et al.  Local false nearest neighbors and dynamical dimensions from observed chaotic data. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[23]  Geoffrey E. Hinton,et al.  SMEM Algorithm for Mixture Models , 1998, Neural Computation.

[24]  K. Rose.  Deterministic annealing for clustering, compression, classification, regression, and related optimization problems , 1998, Proc. IEEE.

[25]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[26]  Anastasios A. Tsonis,et al.  Singular spectrum analysis , 1996 .

[27]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[28]  Chris Chatfield,et al.  Time‐series forecasting , 2000 .

[29]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[30]  H. Abarbanel,et al.  Determining embedding dimension for phase-space reconstruction using a geometrical construction. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[31]  Kenneth Rose,et al.  A Deterministic Annealing Approach for Parsimonious Design of Piecewise Regression Models , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Naonori Ueda,et al.  Deterministic annealing EM algorithm , 1998, Neural Networks.

[33]  I. H. Öğüş,et al.  NATO ASI Series , 1997 .

[34]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[35]  Simon Haykin,et al.  Neural networks , 1994 .

[36]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[37]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[38]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[39]  Achilleas Zapranis,et al.  Stock performance modeling using neural networks: A comparative study with regression models , 1994, Neural Networks.

[40]  T. W. Frison,et al.  Obtaining order in a world of chaos [signal processing] , 1998, IEEE Signal Process. Mag..

[41]  David E. Booth.  Time Series: Forecasting, Simulation, Applications , 1993 .

[42]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[43]  Simon Haykin,et al.  Detection of signals in chaos , 1995, Proc. IEEE.

[44]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[45]  R. Mañé,et al.  On the dimension of the compact invariant sets of certain non-linear maps , 1981 .

[46]  Craig B. Borkowf,et al.  Time-Series Forecasting , 2002, Technometrics.

[47]  Zoubin Ghahramani,et al.  Solving inverse problems using an EM approach to density estimation , 1993 .

[48]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[49]  Antonello Rizzi,et al.  A constructive EM approach to density estimation for learning , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[50]  A. Rizzi,et al.  Constructive MoG neural networks for pollution data forecasting , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[51]  Song-Shyong Chen,et al.  Robust TSK fuzzy modeling for function approximation with outliers , 2001, IEEE Trans. Fuzzy Syst..

[52]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[53]  Vladimir Cherkassky,et al.  Constrained topological mapping for nonparametric regression analysis , 1991, Neural Networks.

[54]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[55]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[56]  S. Haykin,et al.  Making sense of a complex world [chaotic events modeling] , 1998, IEEE Signal Process. Mag..

[57]  Steve McLaughlin,et al.  How to extract Lyapunov exponents from short and noisy time series , 1997, IEEE Trans. Signal Process..

[58]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[59]  C. Bishop.  Mixture density networks , 1994 .

[60]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[61]  R. Vautard,et al.  Singular-spectrum analysis: a toolkit for short, noisy chaotic signals , 1992 .

[62]  Francesco Masulli,et al.  Computational Intelligence in Hydroinformatics: A Review , 1999 .