Annealed Competition of Experts for a Segmentation and Classification of Switching Dynamics

We present a method for the unsupervised segmentation of data streams originating from different unknown sources that alternate in time. We use an architecture consisting of competing neural networks. Memory is included to resolve ambiguities of input-output relations. To obtain maximal specialization, the competition is adiabatically increased during training. Our method achieves almost perfect identification and segmentation in the case of switching chaotic dynamics where input manifolds overlap and input-output relations are ambiguous. Only a small dataset is needed for the training procedure. Applications to time series from complex systems demonstrate the potential relevance of our approach for time series analysis and short-term prediction.

[1]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[2]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[3]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[4]  Lawrence R. Rabiner,et al.  A tutorial on Hidden Markov Models , 1986 .

[5]  Kaneko.  Chaotic but regular posi-nega switch among coded attractors by cluster-size variation. , 1989, Physical review letters.

[6]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[7]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[8]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[9]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[10]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[11]  K. Pawelzik,et al.  Optimal Embeddings of Chaotic Attractors from Topological Considerations , 1991 .

[12]  Gain Scheduling: Potential Hazards and Possible Remedies , 1992, 1991 American Control Conference.

[13]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[14]  Yoshua Bengio,et al.  Credit Assignment through Time: Alternatives to Backpropagation , 1993, NIPS.

[15]  Steven J. Nowlan,et al.  Mixtures of Controllers for Jump Linear and Non-Linear Plants , 1993, NIPS.

[16]  Klaus Pawelzik,et al.  Segmentation and Identification of Switching Dynamics with Competing Neural Networks , 1994 .

[17]  Alan L. Yuille,et al.  Statistical Physics, Mixtures of Distributions, and the EM Algorithm , 1994, Neural Computation.

[18]  Rudower Chaussee,et al.  Competing Predictors Segment andIdentify Switching , 1994 .

[19]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[20]  Klaus Pawelzik,et al.  Detecting Coherence in Neuronal Data , 1994 .

[21]  Klaus-Robert Müller,et al.  Analysis of switching dynamics with competing neural networks , 1995 .