Constructing Sparse KFDA Using Pre-image Reconstruction

Kernel Fisher Discriminant Analysis (KFDA) improves greatly the classification accuracy of FDA via using kernel trick. However, the final solution of KFDA is expressed as an expansion of all training examples, which seriously undermines the classification efficiency, especially in real-time applications. This paper proposes a novel framework to construct sparse KFDA using pre-image reconstruction. The proposed method (PR-KFDA) appends greedily the pre-image of the residual between the current approximate model and the original KFDA model in feature space with the local optimal Fisher coefficients to acquire sparse representation of KFDA solution. Experimental results show that PR-KFDA can reduce the solution of KFDA effectively while maintaining comparable test accuracy.

[1]  Licheng Jiao,et al.  Kernel Uncorrelated Discriminant Analysis for Radar Target Recognition , 2006, ICONIP.

[2]  Gunnar Rätsch,et al.  Input space versus feature space in kernel-based methods , 1999, IEEE Trans. Neural Networks.

[3]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[4]  Yong Wang,et al.  Sparse Kernel Fisher Discriminant Analysis , 2005, ISNN.

[5]  Mario Köppen,et al.  Advances in Neuro-Information Processing, 15th International Conference, ICONIP 2008, Auckland, New Zealand, November 25-28, 2008, Revised Selected Papers, Part I , 2009, International Conference on Neural Information Processing.

[6]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[7]  Ivor W. Tsang,et al.  The pre-image problem in kernel methods , 2003, IEEE Transactions on Neural Networks.

[8]  Bernhard Schölkopf,et al.  Fast Approximation of Support Vector Kernel Expansions, and an Interpretation of Clustering as Approximation in Feature Spaces , 1998, DAGM-Symposium.

[9]  Zhang Yi,et al.  Advances in Neural Networks - ISNN 2005, Second International Symposium on Neural Networks, Chongqing, China, May 30 - June 1, 2005, Proceedings, Part II , 2005, ISNN.

[10]  Pascal Vincent,et al.  Kernel Matching Pursuit , 2002, Machine Learning.

[11]  Tom Diethe,et al.  Matching Pursuit Kernel Fisher Discriminant Analysis , 2009, AISTATS.

[12]  Abdolhossein Sarrafzadeh,et al.  Multi-layered Hand and Face Tracking for Real-Time Gesture Recognition , 2008, ICONIP.

[13]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[14]  Sungzoon Cho,et al.  Invariance of neighborhood relation under input space to feature space mapping , 2005, Pattern Recognit. Lett..

[15]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[16]  Bernhard Schölkopf,et al.  An improved training algorithm for kernel Fisher discriminants , 2001, AISTATS.

[17]  Kurt Stadlthanner,et al.  KPCA denoising and the pre-image problem revisited , 2008, Digit. Signal Process..

[18]  Paul Honeine,et al.  A Closed-form Solution for the Pre-image Problem in Kernel-based Machines , 2011, J. Signal Process. Syst..

[19]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .