A practical application of kernel-based fuzzy discriminant analysis

Abstract A novel method for feature extraction and recognition called Kernel Fuzzy Discriminant Analysis (KFDA) is proposed in this paper to deal with recognition problems, e.g., for images. The KFDA method is obtained by combining the advantages of fuzzy methods and a kernel trick. Based on the orthogonal-triangular decomposition of a matrix and Singular Value Decomposition (SVD), two different variants, KFDA/QR and KFDA/SVD, of KFDA are obtained. In the proposed method, the membership degree is incorporated into the definition of between-class and within-class scatter matrices to get fuzzy between-class and within-class scatter matrices. The membership degree is obtained by combining the measures of features of samples data. In addition, the effects of employing different measures is investigated from a pure mathematical point of view, and the t-test statistical method is used for comparing the robustness of the learning algorithm. Experimental results on ORL and FERET face databases show that KFDA/QR and KFDA/SVD are more effective and feasible than Fuzzy Discriminant Analysis (FDA) and Kernel Discriminant Analysis (KDA) in terms of the mean correct recognition rate.

[1]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[2]  Ahmet Arslan,et al.  A NOVEL HYBRID APPROACH TO ESTIMATING MISSING VALUES IN DATABASES USING K-NEAREST NEIGHBORS AND NEURAL NETWORKS , 2012 .

[3]  Dao-Qing Dai,et al.  Improved discriminate analysis for high-dimensional data and its application to face recognition , 2007, Pattern Recognit..

[4]  Jian Yang,et al.  A reformative kernel Fisher discriminant algorithm and its application to face recognition , 2006, Neurocomputing.

[5]  Jian Yang,et al.  Why can LDA be performed in PCA transformed space? , 2003, Pattern Recognit..

[6]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[7]  Xiaoping Xue,et al.  CONSTRUCTING KERNELS BY FUZZY RULES FOR SUPPORT VECTOR REGRESSIONS , 2012 .

[8]  Ewa Swiercz.  Classification in the Gabor time-frequency domain of non-stationary signals embedded in heavy noise with unknown statistical distribution , 2010, Int. J. Appl. Math. Comput. Sci..

[9]  Jian Yang,et al.  KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[11]  Bartosz Krawczyk,et al.  Combined classifier based on feature space partitioning , 2012, Int. J. Appl. Math. Comput. Sci..

[12]  Jing-Yu Yang,et al.  Optimal discriminant plane for a small number of samples and design method of classifier on the plane , 1991, Pattern Recognit..

[13]  Nikhil R. Pal,et al.  Two efficient connectionist schemes for structure preserving dimensionality reduction , 1998, IEEE Trans. Neural Networks.

[14]  R. Tibshirani,et al.  Flexible Discriminant Analysis by Optimal Scoring , 1994 .

[15]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[17]  Witold Pedrycz,et al.  Face recognition using a fuzzy fisherface classifier , 2005, Pattern Recognit..

[18]  J. Pahasa,et al.  PSO BASED KERNEL PRINCIPAL COMPONENT ANALYSIS AND MULTI-CLASS SUPPORT VECTOR MACHINE FOR POWER QUALITY PROBLEM CLASSIFICATION , 2011 .

[19]  David Zhang,et al.  Face recognition using FLDA with single training image per person , 2008, Appl. Math. Comput..

[20]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Jianqiang Gao,et al.  Median null(Sw)-based method for face feature recognition , 2013, Appl. Math. Comput..

[22]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[23]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[24]  Jianqiang Gao,et al.  Kernel-based weighted discriminant analysis with QR decomposition and its application to face recognition , 2011 .

[25]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[26]  Xiaojun Wu,et al.  Fuzzy Kernel Fisher Discriminant Algorithm with Application to Face Recognition , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[27]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[28]  Robert P. W. Duin,et al.  Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[30]  Jingyu Yang,et al.  Optimal FLD algorithm for facial feature extraction , 2001, SPIE Optics East.

[31]  Xiao-Hong Wu,et al.  Fuzzy discriminant analysis with kernel methods , 2006, Pattern Recognit..

[32]  Chih-Ming Chen,et al.  An efficient fuzzy classifier with feature selection based on fuzzy entropy , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[33]  Jianqiang Gao,et al.  Solving the face recognition problem using QR factorization , 2012 .

[34]  Dao-Qing Dai,et al.  Inverse Fisher discriminate criteria for small sample size problem and its application to face recognition , 2005, Pattern Recognit..

[35]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[36]  Xiuping Liu,et al.  A new fuzzy approach for handling class labels in canonical correlation analysis , 2008, Neurocomputing.

[37]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[38]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[39]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  David G. Stork,et al.  Pattern Classification , 1973 .

[41]  Mehmet Koç,et al.  A new solution to one sample problem in face recognition using FLDA , 2011, Appl. Math. Comput..

[42]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[43]  Lei Zhang,et al.  Feature extraction using fuzzy inverse FDA , 2009, Neurocomputing.

[44]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[45]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[46]  E. Newport,et al.  Science Current Directions in Psychological Statistical Learning : from Acquiring Specific Items to Forming General Rules on Behalf Of: Association for Psychological Science , 2022 .

[47]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[48]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .