Stability and chaos of LMSER PCA learning algorithm

LMSER PCA algorithm is a principal components analysis algorithm. It is used to extract principal components on-line from input data. The algorithm has both stability and chaotic dynamic behavior under some conditions. This paper studies the local stability of the LMSER PCA algorithm via a corresponding deterministic discrete time system. Conditions for local stability are derived. The paper also explores the chaotic behavior of this algorithm. It shows that the LMSER PCA algorithm can produce chaos. Waveform plots, Lyapunov exponents and bifurcation diagrams are presented to illustrate the existence of chaotic behavior of this algorithm.

[1]  H. N. Agiza,et al.  On the Analysis of Stability, Bifurcation, Chaos and Chaos Control of Kopel Map , 1999 .

[2]  Pedro J. Zufiria,et al.  On the discrete-time dynamics of the basic Hebbian neural network node , 2002, IEEE Trans. Neural Networks.

[3]  Xiaofeng Liao,et al.  Generating chaos by Oja's rule , 2003, Neurocomputing.

[4]  Lixin Tian,et al.  Bifurcation analysis and linear control of the Newton–Leipnik system , 2006 .

[5]  Zhang Yi,et al.  Convergence Analysis of Recurrent Neural Networks , 2003, Network Theory and Applications.

[6]  Zhujun Jing,et al.  Bifurcation and chaos in discrete-time predator–prey system , 2006 .

[7]  Misha Tsodyks,et al.  Chaos in neural networks with dynamic synapses , 2000, Neurocomputing.

[8]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[9]  Vwani P. Roychowdhury,et al.  Algorithms for accelerated convergence of adaptive PCA , 2000, IEEE Trans. Neural Networks Learn. Syst..

[10]  Mao Ye,et al.  Convergence analysis of a deterministic discrete time system of feng's MCA learning algorithm , 2005, IEEE Transactions on Signal Processing.

[11]  Ralf Möller,et al.  Coupled principal component analysis , 2004, IEEE Transactions on Neural Networks.

[12]  K. Aihara,et al.  Chaos and asymptotical stability in discrete-time neural networks , 1997, chao-dyn/9701020.

[13]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[14]  Qingfu Zhang,et al.  On the discrete-time dynamics of a PCA learning algorithm , 2003, Neurocomputing.

[15]  Pedro J. Zufiria,et al.  Generalized neural networks for spectral analysis: dynamics and Liapunov functions , 2004, Neural Networks.

[16]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[17]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[18]  S.Y. Kung,et al.  Adaptive Principal component EXtraction (APEX) and applications , 1994, IEEE Trans. Signal Process..

[19]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .

[20]  M. V. Jankovic,et al.  A new simple ∞OH neuron model as a biologically plausible principal component analyzer , 2003, IEEE Trans. Neural Networks.