Learning kernel parameters for kernel Fisher discriminant analysis

Abstract Kernel Fisher discriminant analysis (KFDA) faces the problem of kernel parameters selection. A novel KFDA kernel parameters optimization criterion is presented for maximizing the uniformity of class-pair separabilities and class separability in kernel space simultaneously. The presented criterion is also applied to the kernel parameters selection of spectral regression kernel discriminant analysis (SRKDA). Minimum distance classifier, k nearest neighbor ( k NN) classifier, and naive Bayes classifier are used to evaluate the feature extraction performance. Experiments on fourteen benchmark multiclass data sets show that, comparing with the criterion for merely maximizing the class separability in kernel space, the presented criterion can search the optimum KFDA kernel parameters more accurately, and do better in SRKDA kernel parameters selection.

[1]  Neil D. Lawrence,et al.  Optimising Kernel Parameters and Regularisation Coefficients for Non-linear Discriminant Analysis , 2006, J. Mach. Learn. Res..

[2]  Lei Wang,et al.  Learning kernel parameters by using class separability measure , 2002 .

[3]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[4]  Jiawei Han,et al.  SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis , 2008, IEEE Transactions on Knowledge and Data Engineering.

[5]  Feiping Nie,et al.  Semi-supervised sub-manifold discriminant analysis , 2008, Pattern Recognit. Lett..

[6]  Liefeng Bo,et al.  Feature Scaling for Kernel Fisher Discriminant Analysis Using Leave-One-Out Cross Validation , 2006 .

[7]  Robert P. W. Duin,et al.  Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Pawel Chudzian,et al.  Evaluation measures for kernel optimization , 2012, Pattern Recognit. Lett..

[9]  Lei Wang,et al.  A Kernel-Induced Space Selection Approach to Model Selection in KLDA , 2008, IEEE Transactions on Neural Networks.

[10]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[11]  Jian-Huang Lai,et al.  Kernel subspace LDA with optimized kernel parameters on face recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[12]  Carlos Santa Cruz,et al.  On the equivalence of Kernel Fisher discriminant analysis and Kernel Quadratic Programming Feature Selection , 2011, Pattern Recognit. Lett..

[13]  Aleix M. Martínez,et al.  Kernel Optimization in Discriminant Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Jiawei Han,et al.  Speed up kernel discriminant analysis , 2011, The VLDB Journal.

[15]  Jian-Huang Lai,et al.  Choosing Parameters of Kernel Subspace LDA for Recognition of Face Images Under Pose and Illumination Variations , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  M. Omair Ahmad,et al.  Optimizing the kernel in the empirical feature space , 2005, IEEE Transactions on Neural Networks.

[17]  Boubakeur Boufama,et al.  Non-parametric Fisher's discriminant analysis with kernels for data classification , 2013, Pattern Recognit. Lett..

[18]  Jie Wang,et al.  Gaussian kernel optimization for pattern classification , 2009, Pattern Recognit..

[19]  Tu Bao Ho,et al.  An efficient kernel matrix evaluation measure , 2008, Pattern Recognit..