An Application of Weighted Kernel Fuzzy Discriminant Analysis

In this paper, a new method for feature extraction and recognition called based on QR decomposition weighted kernel fuzzy discriminant analysis (WKFDA/QR) is proposed to deal with nonlinear separable problem. Since QR decomposition on a small size matrix is adopted. A superiority of the proposed methods is its computational efficiency and can avoid the singularity. In the proposed method, the membership degree is incorporated into the definition of between-class and within-class scatter matrixes to get fuzzy between-class and within-class scatter matrixes. Under different distances and different kernel functions, we compare WKFDA/QR, kernel discriminant analysis (KDA) and fuzzy discriminant analysis (FDA) three algorithms by means of the classification rate. In addition, we also compare WKFDA/QR with KDA and FDA under the parameters of weighted function and kernel function. Experiments on ORL and FERET two real-world data sets are performed to test and evaluate the effectiveness of the proposed algorithms and the effect of weights on classification accuracy. The results show that the effect of weighted schemes is very significantly.

[1]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[2]  David G. Stork,et al.  Pattern Classification , 1973 .

[3]  John W. Sammon,et al.  An Optimal Set of Discriminant Vectors , 1975, IEEE Transactions on Computers.

[4]  L. Duchene,et al.  An Optimal Transformation for Discriminant and Principal Component Analysis , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[6]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Jing-Yu Yang,et al.  Optimal discriminant plane for a small number of samples and design method of classifier on the plane , 1991, Pattern Recognit..

[8]  R. Tibshirani,et al.  Flexible Discriminant Analysis by Optimal Scoring , 1994 .

[9]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[10]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[13]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[14]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[15]  Nikhil R. Pal,et al.  Two efficient connectionist schemes for structure preserving dimensionality reduction , 1998, IEEE Trans. Neural Networks.

[16]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[17]  Ravi Kothari,et al.  Fractional-Step Dimensionality Reduction , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[19]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[20]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[21]  Jingyu Yang,et al.  Optimal FLD algorithm for facial feature extraction , 2001, SPIE Optics East.

[22]  Jing-Yu Yang,et al.  A theorem on the uncorrelated optimal discriminant vectors , 2001, Pattern Recognit..

[23]  Chih-Ming Chen,et al.  An efficient fuzzy classifier with feature selection based on fuzzy entropy , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[24]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Jing-Yu Yang,et al.  Face recognition based on the uncorrelated discriminant transformation , 2001, Pattern Recognit..

[26]  Ming-Hsuan Yang,et al.  Kernel Eigenfaces vs. Kernel Fisherfaces: Face recognition using kernel methods , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[27]  Konstantinos N. Plataniotis,et al.  Face recognition using kernel direct discriminant analysis algorithms , 2003, IEEE Trans. Neural Networks.

[28]  Jian Yang,et al.  Why can LDA be performed in PCA transformed space? , 2003, Pattern Recognit..

[29]  David Zhang,et al.  Improvements on the uncorrelated optimal discriminant vectors , 2003, Pattern Recognit..

[30]  Gunnar Rätsch,et al.  Constructing Descriptive and Discriminative Nonlinear Features: Rayleigh Coefficients in Kernel Feature Spaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Jieping Ye,et al.  LDA/QR: an efficient and effective dimension reduction algorithm and its theoretical foundation , 2004, Pattern Recognit..

[32]  A kernel fractional-step nonlinear discriminant analysis for pattern recognition , 2004, ICPR 2004.

[33]  Jian Yang,et al.  KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Witold Pedrycz,et al.  Face recognition using a fuzzy fisherface classifier , 2005, Pattern Recognit..

[35]  Xiao-Hong Wu,et al.  Fuzzy discriminant analysis with kernel methods , 2006, Pattern Recognit..

[36]  Xiaojun Wu,et al.  Fuzzy Kernel Fisher Discriminant Algorithm with Application to Face Recognition , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[37]  Xin Yang,et al.  Improved-LDA based face recognition using both facial global and local information , 2006, Pattern Recognit. Lett..

[38]  Yuntao Qian,et al.  Face recognition using a kernel fractional-step discriminant analysis algorithm , 2007, Pattern Recognit..

[39]  Xiuping Liu,et al.  A new fuzzy approach for handling class labels in canonical correlation analysis , 2008, Neurocomputing.

[40]  Lei Zhang,et al.  Feature extraction using fuzzy inverse FDA , 2009, Neurocomputing.

[41]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[42]  Zhenmin Tang,et al.  A modification of kernel discriminant analysis for high-dimensional data - with application to face recognition , 2010, Signal Process..

[43]  Fan Li-ya.  The Impact on the Classification Results from Distances in Weighted PCA and LDA , 2010 .

[44]  Jianqiang Gao,et al.  Kernel-based weighted discriminant analysis with QR decomposition and its application to face recognition , 2011 .

[45]  Jianqiang Gao,et al.  Solving the face recognition problem using QR factorization , 2012 .

[46]  Fan Li-ya.  THE IMPACT ON THE FACE RECOGNITION FROM DISTANCES IN FUZZY LINEAR DISCRIMINANT ANALYSIS , 2012 .

[47]  Jianqiang Gao,et al.  Median null(Sw)-based method for face feature recognition , 2013, Appl. Math. Comput..