Modelling Chaotic Systems with Neural Networks: Application to Seismic Event Predicting in Gold Mines

This thesis explores the use of neural networks for predicting difficult, real-world time series. We first establish and demonstrate methods for characterising, modelling and predicting well-known systems. The real-world system we explore is seismic event data obtained from a South African gold mine. We show that this data is chaotic. After preprocessing the raw data, we show that neural networks are able to predict seismic activity reasonably well.

[1]  Zoran Obradovic,et al.  Data mining techniques for designing neural network time series predictors , 2000 .

[2]  Mingzhou Ding,et al.  Estimating correlation dimension from a chaotic time series: when does plateau onset occur? , 1993 .

[3]  Martin A. Riedmiller,et al.  Fast Network Pruning and Feature Extraction by using the Unit-OBS Algorithm , 1996, NIPS.

[4]  Ying-Cheng Lai,et al.  Controlling chaos , 1994 .

[5]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[6]  C. Lee Giles,et al.  An analysis of noise in recurrent neural networks: convergence and generalization , 1996, IEEE Trans. Neural Networks.

[7]  Masumi Ishikawa.  Structural learning and rule discovery , 2000 .

[8]  George Bolt,et al.  Investigating Fault Tolerance in ArtificialNeural Networks , 1991 .

[9]  Ott,et al.  Experimental observation of a strange nonchaotic attractor. , 1990, Physical review letters.

[10]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[11]  Dirk Hoyer,et al.  Nonlinear Analysis of the Cardiorespiratory Coordination in a Newborn Piglet , 1998 .

[12]  J. I. Minnix.  Fault tolerance of the backpropagation neural network trained on noisy inputs , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[13]  H. Schuster,et al.  Proper choice of the time delay for the analysis of chaotic time series , 1989 .

[14]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[15]  Alan F. Murray,et al.  Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training , 1994, IEEE Trans. Neural Networks.

[16]  P. Grassberger,et al.  Characterization of Strange Attractors , 1983 .

[17]  D. Kugiumtzis.  State space reconstruction parameters in the analysis of chaotic time series—the role of the time window length , 1996, comp-gas/9602002.

[18]  Lutz Prechelt,et al.  Connection pruning with static and adaptive pruning schedules , 1997, Neurocomputing.

[19]  Hava T. Siegelmann,et al.  Computational capabilities of recurrent NARX neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[20]  Edward Ott,et al.  Controlling chaos , 2006, Scholarpedia.

[21]  Fraser,et al.  Independent coordinates for strange attractors from mutual information. , 1986, Physical review. A, General physics.

[22]  David Ruelle,et al.  Deterministic chaos: the science and the fiction , 1995 .

[23]  秦 浩起,et al.  Characterization of Strange Attractor (カオスとその周辺(基研長期研究会報告)) , 1987 .

[24]  K. Pyragas,et al.  Transmission of Signals via Synchronization of Chaotic Time-Delay Systems , 1998 .

[25]  Eduardo Sontag,et al.  Turing computability with neural nets , 1991 .

[26]  Henry D I Abarbanel,et al.  False neighbors and false strands: a reliable minimum embedding dimension algorithm. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  A. Wolf,et al.  Determining Lyapunov exponents from a time series , 1985 .

[28]  Eckmann,et al.  Liapunov exponents from time series. , 1986, Physical review. A, General physics.

[29]  Hübner,et al.  Dimensions and entropies of chaotic intensity pulsations in a single-mode far-infrared NH3 laser. , 1989, Physical review. A, General physics.

[30]  David Lerner,et al.  Monitoring changing dynamics with correlation integrals: case study of an epileptic seizure , 1996 .

[31]  James L. McClelland,et al.  James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.

[32]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[33]  J. Yorke,et al.  Dimension of chaotic attractors , 1982 .

[34]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[35]  Katsuhiko Ogata,et al.  Modern Control Engineering , 1970 .

[36]  Leonidas D. Iasemidis,et al.  The evolution with time of the spatial distribution of the largest Lyapunov exponent on the human epileptic cortex , 1991 .

[37]  Nicholas B. Tufillaro,et al.  Experimental approach to nonlinear dynamics and chaos , 1992, Studies in nonlinearity.

[38]  Rudy Moddemeijer,et al.  A statistic to estimate the variance of the histogram-based mutual information estimator based on dependent pairs of observations , 1999, Signal Process..

[39]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[40]  C. Lee Giles,et al.  Learning Chaotic Attractors by Neural Networks , 2000, Neural Computation.

[41]  James Theiler,et al.  On the evidence for how-dimensional chaos in an epileptic electroencephalogram , 1995 .

[42]  Dipankar Dasgupta,et al.  Novelty detection in time series data using ideas from immunology , 1996 .

[43]  C. J. Harris,et al.  Neural Networks for Modelling and Control , 1994 .

[44]  Brun,et al.  Control of NMR-laser chaos in high-dimensional embedding space. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[45]  Hanspeter Herzel,et al.  Detecting Bifurcations in Voice Signals , 1998 .

[46]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[47]  George E. Backus,et al.  Moment Tensors and other Phenomenological Descriptions of Seismic Sources—I. Continuous Displacements , 1976 .

[48]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[49]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[50]  Emile Fiesler,et al.  Neural Networks with Adaptive Learning Rate and Momentum Terms , 1995 .

[51]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[52]  Jude W. Shavlik,et al.  Understanding Time-Series Networks: A Case Study in Rule Extraction , 1997, Int. J. Neural Syst..

[53]  Steven M. Boker,et al.  Nonlinear Analysis of Perceptual-Motor Coupling in the Development of Postural Control , 1998 .

[54]  O. Rössler.  An equation for continuous chaos , 1976 .

[55]  C. Lee Giles,et al.  What Size Neural Network Gives Optimal Generalization? Convergence Properties of Backpropagation , 1998 .

[56]  G. P. King,et al.  Extracting qualitative dynamics from experimental data , 1986 .

[57]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[58]  Y. Shimshoni.  Classifying Seismic Signals by Integrating Ensembles of Neural Networks , 1996 .

[59]  C. Sparrow.  The Lorenz Equations: Bifurcations, Chaos, and Strange Attractors , 1982 .

[60]  Eric A. Wan,et al.  Time series prediction by using a connectionist network with internal delay lines , 1993 .

[61]  C. Diks,et al.  Detecting differences between delay vector distributions. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[62]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[63]  Tom Heskes,et al.  Partial Retraining: A New Approach to Input Relevance Determination , 1999, Int. J. Neural Syst..

[64]  Irwin W. Sandberg,et al.  Uniform approximation of multidimensional myopic maps , 1997 .