Robust kernel discriminant analysis using fuzzy memberships

Linear discriminant analysis (LDA) is a simple but widely used algorithm in the area of pattern recognition. However, it has some shortcomings in that it is sensitive to outliers and limited to linearly separable cases. To solve these problems, in this paper, a non-linear robust variant of LDA, called robust kernel fuzzy discriminant analysis (RKFDA) is proposed. RKFDA uses fuzzy memberships to reduce the effect of outliers and adopts kernel methods to accommodate non-linearly separable cases. There have been other attempts to solve the problems of LDA, including attempts using kernels. However, RKFDA, encompassing previous methods, is the most general one. Furthermore, theoretical analysis and experimental results show that RKFDA is superior to other existing methods in solving the problems.

[1]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[2]  James M. Keller,et al.  A possibilistic fuzzy c-means clustering algorithm , 2005, IEEE Transactions on Fuzzy Systems.

[3]  Jian Yang,et al.  KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[5]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[6]  I. Jolliffe.  Principal Component Analysis , 2002 .

[7]  Jieping Ye,et al.  Efficient Kernel Discriminant Analysis via QR Decomposition , 2004, NIPS.

[8]  Paul D. Gader,et al.  2009 Special Issue: RKF-PCA: Robust kernel fuzzy PCA , 2009 .

[9]  Haesun Park,et al.  Generalizing discriminant analysis using the generalized singular value decomposition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jian-hui Jiang,et al.  Fuzzy linear discriminant analysis for chemical data sets , 1999 .

[11]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[12]  Haesun Park,et al.  Nonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[13]  Xiao-Hong Wu,et al.  Fuzzy discriminant analysis with kernel methods , 2006, Pattern Recognit..

[14]  Zhihua Zhang,et al.  A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis , 2009, ECML/PKDD.

[15]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[16]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[17]  Bok-Suk Shin,et al.  Segmentation of Scanned Insect Footprints Using ART2 for Threshold Selection , 2007, PSIVT.

[18]  Jing-Yu Yang,et al.  Face recognition using Complete Fuzzy LDA , 2008, 2008 19th International Conference on Pattern Recognition.

[19]  Jieping Ye,et al.  Discriminant kernel and regularization parameter learning via semidefinite programming , 2007, ICML '07.

[20]  J. Bendat,et al.  Random Data: Analysis and Measurement Procedures , 1971 .

[21]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[22]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[23]  J. Bendat,et al.  Random Data: Analysis and Measurement Procedures , 1987 .

[24]  Jieping Ye,et al.  Learning the kernel matrix in discriminant analysis via quadratically constrained quadratic programming , 2007, KDD '07.

[25]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[26]  Takio Kurita,et al.  Robust De-noising by Kernel PCA , 2002, ICANN.

[27]  Ahmed S. Abutableb.  Automatic thresholding of gray-level pictures using two-dimensional entropy , 1989 .

[28]  Reshma Khemchandani,et al.  Learning the optimal kernel for Fisher discriminant analysis via second order cone programming , 2010, Eur. J. Oper. Res..

[29]  Jieping Ye,et al.  A two-stage linear discriminant analysis via QR-decomposition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Bok-Suk Shin,et al.  Effective feature extraction by trace transform for insect footprint recognition , 2008, 2008 3rd International Conference on Bio-Inspired Computing: Theories and Applications.

[31]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[32]  Paul D. Gader,et al.  Fuzzy SVM for noisy data: A robust membership calculation method , 2009, 2009 IEEE International Conference on Fuzzy Systems.