Fisher+Kernel criterion for discriminant analysis

We simultaneously approach two tasks of nonlinear discriminant analysis and kernel selection problem by proposing a unified criterion, Fisher+Kernel criterion. In addition, an efficient procedure is derived to optimize this new criterion in an iterative manner. More specifically, original input vector is first transformed into a higher dimensional feature matrix through a battery of nonlinear mappings involved in different kernels. Then, based on the feature matrices, FKC is presented within two coupled projection spaces: one projection space is used to search for the optimal combinations of kernels; while the other encodes the optimal nonlinear discriminating projection directions. Our proposed method is a unified framework for both kernel selection and nonlinear discriminant analysis. Besides, the algorithm potentially alleviates overfitting problem existing in traditional KDA and has no singularity problems in most cases. The effectiveness of our proposed algorithm is validated by extensive face recognition experiments on several datasets.

[1]  Konstantinos N. Plataniotis,et al.  Face recognition using kernel direct discriminant analysis algorithms , 2003, IEEE Trans. Neural Networks.

[2]  Ming-Hsuan Yang,et al.  Kernel Eigenfaces vs. Kernel Fisherfaces: Face recognition using kernel methods , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[3]  Massimiliano Pontil,et al.  Leave One Out Error, Stability, and Generalization of Voting Combinations of Classifiers , 2004, Machine Learning.

[4]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[5]  Tony Jebara,et al.  A Kernel Between Sets of Vectors , 2003, ICML.

[6]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[7]  Haesun Park,et al.  Generalizing discriminant analysis using the generalized singular value decomposition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Hanqing Lu,et al.  Improving kernel Fisher discriminant analysis for face recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[10]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[11]  Koby Crammer,et al.  Kernel Design Using Boosting , 2002, NIPS.

[12]  Murat Dundar,et al.  A fast iterative algorithm for fisher discriminant using heterogeneous kernels , 2004, ICML.

[13]  Jian-Huang Lai,et al.  Kernel subspace LDA with optimized kernel parameters on face recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[14]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[15]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..