A Family of Fuzzy Learning Algorithms for Robust Principal Component Analysis Neural Networks

In this paper, we analyze Xu and Yuille's robust principal component analysis (RPCA) learning algorithms by means of the distance measurement in space. Based on the analysis, a family of fuzzy RPCA learning algorithms is proposed, which is robust against outliers. These algorithms can explicitly be understood from the viewpoint of fuzzy set theory, though Xu and Yuille's algorithms were proposed based on a statistical physics approach. In the proposed algorithms, an adaptive learning procedure overcomes the difficulty of selection of learning parameters in Xu and Yuille's algorithms. Furthermore, the robustness of proposed algorithms is investigated by using the theory of influence functions. Simulations are carried out to illustrate the robustness of these algorithms.

[1]  Hong Chen,et al.  An on-line unsupervised learning machine for adaptive feature extraction , 1993, 1993 IEEE International Symposium on Circuits and Systems.

[2]  S.Y. Kung,et al.  Adaptive Principal component EXtraction (APEX) and applications , 1994, IEEE Trans. Signal Process..

[3]  Michael J. Black,et al.  On the unification of line processes , 1996 .

[4]  Zhang Yi,et al.  Convergence analysis of Xu's LMSER learning algorithm via deterministic discrete time system method , 2006, Neurocomputing.

[5]  Ralf Möller,et al.  Coupled principal component analysis , 2004, IEEE Transactions on Neural Networks.

[6]  Sheng-De Wang,et al.  Robust algorithms for principal component analysis , 1999, Pattern Recognit. Lett..

[7]  Zhang Yi,et al.  Global convergence of Oja's PCA learning algorithm with a non-zero-approaching adaptive learning rate , 2006, Theor. Comput. Sci..

[8]  Mahmood R. Azimi-Sadjadi,et al.  Principal component extraction using recursive least squares learning , 1995, IEEE Trans. Neural Networks.

[9]  V. Yohai,et al.  Robust Statistics: Theory and Methods , 2006 .

[10]  Mao Ye,et al.  Convergence analysis of a deterministic discrete time system of feng's MCA learning algorithm , 2005, IEEE Transactions on Signal Processing.

[11]  Hidetomo Ichihashi,et al.  Regularized linear fuzzy clustering and probabilistic PCA mixture models , 2005, IEEE Transactions on Fuzzy Systems.

[12]  Hidemitsu Ogawa,et al.  Modulated Hebb-Oja learning Rule-a method for principal subspace analysis , 2006, IEEE Transactions on Neural Networks.

[13]  Pedro J. Zufiria,et al.  On the discrete-time dynamics of the basic Hebbian neural network node , 2002, IEEE Trans. Neural Networks.

[14]  Chanchal Chatterjee,et al.  Adaptive algorithms for first principal eigenvector computation , 2005, Neural Networks.

[15]  Junzo Watada,et al.  Fuzzy Principal Component Analysis and Its Application , 1997 .

[16]  Zhang Yi,et al.  Determination of the Number of Principal Directions in a Biologically Plausible PCA Model , 2007, IEEE Transactions on Neural Networks.

[17]  Tai-Ning Yang,et al.  Fuzzy auto-associative neural networks for principal component extraction of noisy data , 2000, IEEE Trans. Neural Networks Learn. Syst..

[18]  Andrzej Cichocki,et al.  Adaptive learning algorithm for principal component analysis with partial data , 1996 .

[19]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[20]  Jianbo Gao,et al.  Principal component analysis of 1/fα noise , 2003 .

[21]  Hidetomo Ichihashi,et al.  Component-wise robust linear fuzzy clustering for collaborative filtering , 2004, Int. J. Approx. Reason..

[22]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[23]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[24]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[25]  Shinto Eguchi,et al.  The Influence Function of Principal Component Analysis by Self-Organizing Rule , 1998, Neural Computation.

[26]  Alan L. Yuille,et al.  Robust principal component analysis by self-organizing rules based on statistical physics approach , 1995, IEEE Trans. Neural Networks.

[27]  F. Critchley.  Influence in principal components analysis , 1985 .

[28]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[29]  Andrzej Cichocki,et al.  Robust estimation of principal components by using neural network learning algorithms , 1993 .

[30]  Michael G. Strintzis,et al.  Optimal linear compression under unreliable representation and robust PCA neural models , 1999, IEEE Trans. Neural Networks.

[31]  Zheng Bao,et al.  Robust recursive least squares learning algorithm for principal component analysis , 2000, IEEE Trans. Neural Networks Learn. Syst..

[32]  Pedro J. Zufiria,et al.  Generalized neural networks for spectral analysis: dynamics and Liapunov functions , 2004, Neural Networks.

[33]  Vwani P. Roychowdhury,et al.  Algorithms for accelerated convergence of adaptive PCA , 2000, IEEE Trans. Neural Networks Learn. Syst..

[34]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[35]  Thierry Denoeux,et al.  Principal component analysis of fuzzy data using autoassociative neural networks , 2004, IEEE Transactions on Fuzzy Systems.

[36]  Zhang Yi,et al.  Global Convergence of GHA Learning Algorithm With Nonzero-Approaching Adaptive Learning Rates , 2007, IEEE Transactions on Neural Networks.